Automators

flowproof

OPEN SOURCE · v0.11.0 · APACHE-2.0

Your agent behaved today.
Prove it’ll behave tomorrow.

AI agents are starting to do real work in production: raising SAP orders, updating records, closing tickets. Flowproof tests that an agent did only the job it was given — and did not delete, send or approve what it must not.

Record the agent doing its job once. Every commit after that replays the recording and checks what the agent did: which systems it used, what it told them, in what order — and what it never touched. No model, no API key, nothing billed.

View on GitHub Read the docs
Python pip install flowproof
Node npx flowproof

An agent can go wrong in two places. Watch both.

An agent can decide the wrong thing, or do the wrong thing to your system. A test suite can drive SAP but cannot see the agent’s decisions. An eval tool sees the decisions but cannot drive SAP. Flowproof records both in one run, under one list of rules.

Record once. Replay with no model at all.

Recording runs your agent against a real model and saves every question and answer. Every later run replays the saved answers: no API key, no model call, nothing billed. So the suite is free on every commit, and when a test goes red, your agent changed — not the model.

If a rule stops being checked, the build stops.

A rule the agent must obey — never refund without approval, never delete a customer — gets a fixed name, written in the test as control:. flowproof audit lists every rule, whether it held, and the recording that proves it. Add --since and it compares against an earlier run, then fails the build if a rule started failing or stopped being checked.

In CI flowproof audit ./flows --since "$BASE_RUN"

There are three answers: pass, fail, and capability-error — we could not check this one. A missing recording gives a capability-error naming it. "Could not check" is never written down as "it held".

The gate is tested in both directions: red when a rule fails or is removed, green when nothing is wrong. See how flowproof tests flowproof.

“It can’t” and “it didn’t” are different promises.

One means the machine blocks the agent. The other means we watched, and it behaved. Both are worth having. We keep them separate — in the product, and in the two boxes below.

Containment

It can’t.

List the addresses the agent may reach in allow_egress, then check it held with assert_no_egress. On Linux the operating system enforces the list: anything else is refused. Name a tool under mcp: with a fixed result: and the real tool never runs — not while recording, not while replaying. None of this depends on the model or on a line in a prompt. 

agent:  command: node assistant.mjs  allow_egress:    - api.acme.internal:443steps:  - assert_no_egress

The lock exists on Linux. On macOS and Windows the run says "not contained" and assert_no_egress fails, rather than passing quietly.

Behaviour

It didn’t — checked on every commit.

assert_no_tool_call proves the agent did not use a forbidden tool in the situation you recorded. The recording pins every argument to the byte, so any change turns the build red. This is evidence that nothing has changed — not proof that nothing can. 

steps:  - prompt: Refund order 4471 for me.  - assert_no_tool_call: issue_refund

A sentence in a prompt is not a control — put tests like this behind a real lock. assert_no_tool_call ships with a test proving it can actually fail.

Flowproof certifies nothing. It gives each rule a name that does not move: the control: id. Map that id to your own framework in your own catalogue. Rename or move the file and the id still points at the same rule.

All of it is Apache-2.0.

No cut-down free edition, no paid unlock. Everything runs without talking to us, and everything it produces is yours. We sell help getting it into your business.

In the repository

  • The engine, the command line, the Python API and the MCP server
  • All six kinds of system: web, Windows, SAP GUI, Citrix, HTTP, and the agent itself
  • The network lock on Linux, and assert_no_egress to prove it held
  • flowproof audit, and --since to fail the build on a regression
  • Suites, repairs you review before applying, JUnit XML and HTML reports

What we charge for

  • Setup on your landscape: your SAP systems, your pipeline, your rules
  • Keeping your run evidence, still findable later
  • Support through your certification programme

Also in the box

  • On-screen checks: elements, text, pixels, with regions you can mask
  • Cookie security flags checked, never a cookie value
  • Database and API checks; faked network replies
  • Secrets passed as ${VAR}, blanked in every screenshot
  • assert_no_secret_leak on agent, page and API output
  • Retries, setup and cleanup steps, one manifest per suite
  • App changed? A repair you review as a diff
  • flowproof capture: what a tool sends, byte for byte

From Selenium to Flowproof.

Selenium, Robot Framework, Playwright — each fit how software was built in its time. Flowproof is for the time when agents do part of the work, and someone has to show what they did.

Timeline: Selenium (2004), Robot Framework (2008), Playwright (2020), Flowproof (2026)
  1. 2004 Selenium
  2. 2008 Robot Framework
  3. 2020 Playwright
  4. 2026 Flowproof

$ flowproof audit ./flows --since $BASE_RUN

View on GitHub Talk to us about a pilot
Docs Agent testing Authoring grammar When not to adopt it llms-full.txt PyPI npm

Flowproof is an Apache-2.0 open-source project by Automators. It is early, and things may still change between versions.