Run flows in CI

Pipeline recipes for GitHub Actions, GitLab CI, and any runner: secrets, environments, JUnit reports, spec drift gates, headless ai-action nodes.

A Bandura suite is files plus one command, so CI integration is mostly about wiring secrets and collecting a report. This page is the copy-paste layer on top of the CLI reference. If you’re still weighing whether to move the suite into the pipeline at all, API testing in CI covers the exit-code contract, the reporters, and why there’s no hosted runner to rent.

The shape of every recipe below is the same. Every pinned snippet on this page uses one version, 1.0.0-rc.6; substitute whatever npm view @bandura/cli dist-tags reports when you set the pipeline up:

npx @bandura/[email protected] run "flows/**/*.aether" -r junit,github -o report.xml

Two things to carry into every example below:

Quote the glob. Unquoted, your shell expands it before bandura sees it, and a shell that expands ** differently (or not at all) silently changes which flows run.
Pin the version. These samples pin the version stated above; substitute whatever you’re on. An unpinned npx @bandura/cli resolves the latest tag, which moves with every release, so a pipeline that worked yesterday can pick up a different runner tomorrow without anything in your repo changing. Pin an exact version in CI and upgrade it deliberately. (Installing the CLI.)

Exit codes are the contract

CI needs to tell “the tests failed” apart from “the tool broke”, and these codes are stable:

Code	Meaning	What CI should do
`0`	All flows passed	Green
`1`	A test failure	Red. This is a real finding
`2`	Usage/config error	Red. Bad flags, no flows matched, or a declared variable is unset
`3`	A file doesn’t parse	Red. Your repo is wrong
`4`	Internal error	Red. Tell us

GitHub Actions

The direct version is one step, annotations on the PR, and a JUnit file for the test dashboard:

name: API tests
on: [push, pull_request]

jobs:
  api:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v5
        with:
          node-version: "20"

      - name: Run flows
        run: npx @bandura/[email protected] run "flows/**/*.aether" -r junit,github -o report.xml
        env:
          # Flows read these as env.API_KEY / env.BASE_URL. Never hard-code them in .aether files.
          API_KEY: ${{ secrets.API_KEY }}
          BASE_URL: ${{ vars.STAGING_BASE_URL }}

      - name: Publish the report
        if: always()
        uses: actions/upload-artifact@v5
        with:
          name: bandura-report
          path: report.xml

if: always() matters. Without it a failing run uploads nothing, which is exactly the run you wanted the report from.

That report is an artifact anyone with repository access can download, which is why reports are redacted by default: the resolved Authorization header a request actually sent does not survive into the file. Bodies and failure messages are deliberately left intact, so read what redaction covers before you treat an uploaded report as safe to share more widely. --no-redact exists for local debugging and should not appear in a pipeline.

A composite action that wraps the same binary in three lines is written but not published yet, so there is deliberately no snippet for it here. bandura-io/run@v1 does not resolve today, and a workflow you can copy that fails on its first run with “Unable to resolve action” is worse than no workflow at all. When it publishes it will always write JUnit alongside whatever reporters you ask for and expose exit-code and junit-path as outputs, and checkout will stay your job. Until then the run: step above is the supported form, and it is what “Export to CI/CD” generates.

GitLab CI

api-tests:
  image: node:20
  script:
    - npx @bandura/[email protected] run "flows/**/*.aether" -r junit -o report.xml
  variables:
    BASE_URL: https://staging.api.example.com
  # API_KEY comes from the project's masked CI/CD variables, never from this file.
  artifacts:
    when: always
    reports:
      junit: report.xml

Anything else

The recipe doesn’t change for Jenkins, CircleCI, Buildkite, or a cron job on a box:

npm install -g @bandura/cli
bandura run "flows/**/*.aether" -r junit -o report.xml

The CLI ships as a self-contained native binary, so once it’s installed there’s no Node runtime on the run path at all.

Secrets and environments

Two mechanisms, and they compose:

env.* reads the process environment plus a dotenv file (--env-file, default .env). Inject secrets as environment variables in your CI settings; the values never enter the repo.
--env <name> loads a named environment’s non-secret values from bandura.json: base URLs, tenant ids, feature flags. Secrets stay out of the manifest by design.

# Non-secret config from the manifest, secrets from the CI environment
bandura run -e staging "flows/**/*.aether"

# Or override one value without touching any file
bandura run "flows/**/*.aether" --var baseUrl=https://staging.api.example.com

A one-off --var beats every other source, which makes it the right tool for a matrix job that runs the same suite against three hosts.

A missing secret stops the run, before it runs

If bandura.json declares requiredEnv, bandura run checks those names before executing anything and exits 2 listing the ones that are not set:

2 required environment variables are not set. bandura.json declares them under requiredEnv:
  API_KEY     Service key for the orders API
  BASE_URL

This is the single most common CI failure, and it is worth the gate because of what happens without it: one mistyped secret name runs the whole suite and reports forty assertion failures against a 401, with no message anywhere naming the actual cause.

Two details. The check reads the resolved environment, so a name supplied by the manifest’s own environments.<name> block counts as satisfied. And an empty value counts as missing, because interpolation renders an absent value as the empty string with no error, so API_KEY= produces exactly the same silent 401 as no API_KEY at all.

Only the declared list is enforced. A gate that inferred its own list would fail open on the day someone added a variable and forgot to declare it, so requiredEnv is worth keeping current: bandura init seeds it by scanning your flows for ${{ env.* }} references.

Making it fast, and making it stop

# Four flows at a time; each flow still runs its own nodes in order
bandura run "flows/**/*.aether" --concurrency 4

# Stop at the first failing flow: a smoke gate, not a full report
bandura run "smoke/**/*.aether" --bail

# A hard ceiling per flow, so a hung endpoint can't hold the runner
bandura run "flows/**/*.aether" --timeout 120000

Output is buffered back into input order, so a concurrent run’s report is identical to a serial one’s. Reports stay diffable.

Gates worth adding before the tests

Parse everything, run nothing. Seconds, no network. Good as a pre-commit hook or the first job in the pipeline:

bandura validate "flows/**/*.aether"

Lint the flows. Static analysis over the same files: nodes nothing routes to, loops with no body, certificate verification switched off, literal-looking credentials that belong in env, and flows with no assertions at all. --severity picks what fails the job.

bandura lint "flows/**/*.aether" --severity warning

A parse error is never a suppressible finding here. It exits 3 at every severity, because a build whose flows no longer load must not go green.

Fail if the committed OpenAPI spec has drifted behind the flows. export openapi is deterministic and offline, so --check is a legitimate CI gate rather than a flaky one:

bandura export openapi --check

Gating on latency, not just correctness

--repeat runs each flow N times and reports percentiles; --threshold turns that into a pass or fail:

bandura run flows/login.aether --repeat 30 --concurrency 5 --threshold "p95<500"

A missed threshold exits 1, the same code a failed assertion uses, even when every individual run passed. This is lite local load testing rather than a replacement for a real load tool, and it is best pointed at a stable environment: run it against a busy shared staging box and you will be gating on someone else’s traffic. Full details.

Publishing docs from the same flows

bandura docs generate writes readable documentation from the flows the tests already run, so it cannot describe an endpoint the suite does not exercise:

bandura docs generate --format html --out public/docs

It is deterministic and offline, which makes it safe to regenerate on every build and commit or publish the result. Full details.

AI nodes in a pipeline

Flows containing ai-action nodes run headlessly. There’s no Settings view in CI, so the provider comes from the environment. Pick the recipe that matches your setup. The full variable table is in Connect an AI provider.

Anthropic, with the original variables unchanged:

- run: npx @bandura/[email protected] run "flows/**/*.aether" -r junit -o report.xml
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
    ANTHROPIC_MODEL: claude-sonnet-5 # optional; omit for the default

Any OpenAI-compatible endpoint: OpenAI, Groq, OpenRouter, Together. The base URL alone selects the adapter, so BANDURA_AI_PROVIDER is rarely needed:

- run: npx @bandura/[email protected] run "flows/**/*.aether" -r junit -o report.xml
  env:
    BANDURA_AI_BASE_URL: https://api.groq.com/openai/v1
    BANDURA_AI_API_KEY: ${{ secrets.GROQ_API_KEY }}
    BANDURA_AI_MODEL: llama-3.3-70b-versatile

A shell already configured for OpenAI needs nothing new. OPENAI_API_KEY, OPENAI_BASE_URL and OPENAI_MODEL are read as fallbacks:

env:
  OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
  OPENAI_BASE_URL: https://api.openai.com/v1
  OPENAI_MODEL: gpt-4o

A model on the runner itself, so no third-party API is in the pipeline at all. This needs a self-hosted runner with Ollama serving locally, and no key:

- run: |
    ollama serve &
    until curl -s -o /dev/null http://localhost:11434; do sleep 0.5; done
    ollama pull llama3.1
    npx @bandura/[email protected] run "flows/**/*.aether" -r junit -o report.xml
  env:
    BANDURA_AI_BASE_URL: http://localhost:11434/v1
    BANDURA_AI_MODEL: llama3.1

Nothing configured at all is a supported state, not a broken one: ai-action nodes fail with a message saying so and every other node type runs normally. A suite with one AI node doesn’t become a suite that can’t run in CI. If that’s your intent, keep the AI node’s flow out of the CI glob instead. Remember, too, that model output isn’t deterministic, so pass/fail decisions belong in assertion nodes.

If AI runs fail only in CI, work through AI provider troubleshooting. The causes are the same as on the desktop, minus the Test connection button.

Testing against a mock instead of a live API

When there’s no staging environment to point at, serve one from the flows themselves:

bandura mock "flows/**/*.aether" -p 4010 &
# Wait for the listener before pointing the suite at it.
until curl -s -o /dev/null http://localhost:4010; do sleep 0.2; done
bandura run "flows/**/*.aether" --var baseUrl=http://localhost:4010

Pass an explicit -p in CI. The default is a free port the OS picks, which you can’t hard-code into the next command.

Same OpenAPI document behind both, so the mock can’t disagree with the spec you export. Its honest limit is that it’s only as detailed as your assertions.

The bandura CLI covers every command, flag and reporter.
Share flows with Git covers what belongs in the repo.
Variables & environments explains where a value comes from, and in what order.

Stuck, or found a mistake on this page? Email [email protected]. A human reads it.