Run a local model with Ollama

Use a self-hosted LLM for AI API testing: point Bandura at Ollama or LM Studio over the OpenAI-compatible API. No key, no payload leaves your machine.

Bandura’s OpenAI-compatible provider talks to any /chat/completions endpoint, and a local runtime is just another one of those. Point it at Ollama or LM Studio on localhost and every AI feature (chat, flow generation, and ai-action nodes) runs against a model on your own hardware.

Two things follow from that, and they’re the reason to do it:

No API key. A local runtime authenticates with nothing. Leave the key field empty.
No traffic leaves the machine. Your prompts carry your endpoints, your headers, and your captured response bodies. Against http://localhost, none of that is sent anywhere, which matters if the API under test is internal, regulated, or under NDA.

What you give up in exchange is real, and it’s mostly tool-calling reliability: local LLM API testing sets out which capabilities hold up on a 7B model and which don’t, before you install anything.

Ollama, start to finish

Install Ollama and pull a model:
```
ollama pull llama3.1
```
Ollama serves an OpenAI-compatible API on port 11434 while it’s running.
In Bandura, open Settings (⌘,) → AI provider.
Set Provider to OpenAI-compatible.
Set Base URL to http://localhost:11434/v1.
Set Model to the tag you pulled: llama3.1, qwen2.5-coder, whatever you’re serving. The field is free text; it isn’t limited to the suggestions.
Leave the API key empty.
Click Test connection. It sends one small message and reports what came back, so a wrong port or an un-pulled model shows up now rather than mid-run.

That’s the whole setup. Chat, generation, and ai-action nodes now go to localhost.

LM Studio

Identical, with a different port: start LM Studio’s local server, load a model, and set the base URL to http://localhost:1234/v1. The model id is the one LM Studio lists for the loaded model.

Headless: the CLI, CI, and MCP

The same endpoint works for bandura run and the MCP server; they read the environment rather than Settings:

export BANDURA_AI_BASE_URL=http://localhost:11434/v1
export BANDURA_AI_MODEL=llama3.1
bandura run flows/checkout.aether

No key variable is needed. A base URL on its own is enough to select the OpenAI-compatible adapter. The full variable list is in Connect an AI provider.

Worth noting for CI: a self-hosted runner can serve the model to the job on the same host, so an ai-action suite runs with no third-party API in the pipeline at all.

What to expect from a local model

Being honest about the trade: Anthropic remains the default and the best-tested path, and a 7B model on a laptop is not Claude Opus.

Tool use is required for the agentic chat. The chat works by calling tools (listing flows, reading them, running one). A model without reliable tool-calling support will chat but won’t act. llama3.1, qwen2.5-coder and similar tool-capable tags are the ones to reach for.
Flow generation is stricter than chat. Generated flows are schema-validated before they touch disk, so a weaker model tends to fail the validation rather than write a broken file. That’s the right failure, but expect more retries than you’d get from a frontier model.
ai-action output still isn’t deterministic, locally or otherwise. Keep pass/fail decisions in assertion nodes with concrete check expressions.

You can switch back at any time: keys are stored per provider, so moving between a local model and Anthropic is one dropdown, not a re-paste.

If it won’t connect

Three causes account for nearly all of it: the base URL is missing its /v1 prefix (Bandura appends /chat/completions to whatever you enter), the server isn’t running, or the tag isn’t pulled. curl http://localhost:11434/v1/models distinguishes the three in one command. The full message-by-message list is on AI provider troubleshooting.

Stuck, or found a mistake on this page? Email [email protected]. A human reads it.