Multi-Agent Operations Pipeline

Applied AI · stateful agents · human-in-the-loop

Copy that ships only after a human says yes.

This service turns a product SKU and a campaign brief into channel marketing copy for a Natural Health Product: a web title and product page, five marketplace bullets, a social post, an image prompt, the mandatory disclaimers and the licence number. A supervisor validates the request, a tool reads the SKU from PostgreSQL, a synthesis agent drafts, and a compliance validator audits the draft against Health Canada NHP rules: a model's verdict, plus deterministic checks that a model cannot talk its way past. A draft with violations goes back to the synthesis agent, a bounded number of times. Then the graph pauses on a checkpoint and waits for a person.

The only mutation is behind the gate. Approval or an edit resumes the graph and fires one HMAC-signed webhook, delivered with retries and written to an audit log. A rejection ends the thread with nothing sent. The recorded runs below show all of it, as the API returned it.

LangGraph StateGraph · FastAPI · PostgreSQL checkpoints (langgraph-checkpoint-postgres) · Pydantic v2 · free OpenRouter models · HMAC-SHA256 webhook

Open the live demo Code on GitHub

The live demo runs on free hosting and sleeps between visits. Opening it wakes it, which takes about a minute.

Runs at zero marginal cost: free OpenRouter models for the two agents, serverless PostgreSQL for the checkpoints, the webhook delivered to the service's own receiver.

Recorded runs

Real runs of the service, saved as the API returned them.

The recorded runs are read from recorded/runs.json; when that file is not there yet, or JavaScript is off, this sentence is all there is.

How a run flows

Six nodes, one cycle, one breakpoint. The graph is compiled with interrupt_before=["webhook_dispatch"], so every thread stops on a checkpoint before the only node that touches the outside world.

The pipeline graph Start, supervisor, inventory SQL agent, copy synthesis agent, compliance validator, human approval gate, then an interrupt before the webhook dispatch node and the end. The compliance validator sends violations back to the synthesis agent while revisions remain. The supervisor and the inventory agent can end the run early. inventory already loaded (replay) violations and revision_count < max_revisions compliant, or escalated END: invalid request END: SKU unknown or out of stock interrupt_before a human decides Supervisordeterministic Inventory SQL Agenttool Copy Synthesis Agentllm Compliance Validatorllm + checks Human Approval Gatedeterministic Webhook Dispatchmutation
  1. 1

    Supervisor

    Validates the request (SKU code, channels, brief, revision budget), resets the revision counter and routes: to the inventory agent on a fresh thread, straight to synthesis when the inventory is already loaded, to END when the request is invalid.

    graph/nodes.py

  2. 2

    Inventory SQL Agent

    One parameterized, read-only lookup of the SKU. No model writes SQL: the tool's arguments are validated by Pydantic and the query is fixed. An unknown SKU or a stock of zero ends the run here, before any model is called.

    graph/tools.py

  3. 3

    Copy Synthesis Agent

    A model drafts every field under a strict JSON schema, grounded in the licence data. On a revision it receives the previous draft and the exact violations to fix. The brief travels in tags with a random suffix and is declared untrusted.

    graph/prompts.py

  4. 4

    Compliance Validator

    A model audits the draft against the rules, then Python checks what can be checked deterministically: the NPN is shown, no prohibited claim appears, amounts next to an ingredient are the licensed ones, every mandatory risk statement is there word for word. A finding from the checks is a violation whatever the model said.

    compliance/rules.py

  5. 5

    Human Approval Gate

    Marks the thread as waiting; the graph stops right after it on a checkpoint in PostgreSQL. The thread can be resumed by another process, hours later: the API reads the checkpoint, applies the reviewer's decision to the state and continues.

    graph/builder.py

  6. 6

    Webhook Dispatch

    Refuses to send unless the review status is approved or edited, validates the copy once more, signs the body with HMAC-SHA256 over the timestamp and the bytes, delivers with retries on 429, 5xx and timeouts, and records every attempt. A rejection is written as if this node had run, so it never executes.

    services/webhook_dispatcher.py

The rules

Modelled on Health Canada's framework for Natural Health Products; a demo, not legal advice.

high severity

Show the Natural Product Number

A licensed product is identified by its NPN. The product page and the social post must show it exactly as licensed, and the draft must report the number it displayed. Checked deterministically.

high severity

Make only the claims the licence authorizes

Every health claim in the copy must be one of the product's authorized claims, or a plain restatement of one. New benefits, even plausible ones, are not allowed. The model's judgement, with the claims listed in the prompt.

high severity

No treatment, cure or prevention claims for diseases

Advertising to the general public may not present the product as a treatment, cure, preventive or substitute for medical care for a disease. Words such as cure, treat, heal, reverse, antibiotic and kills infections are caught by regular expressions as well.

medium severity

State ingredient amounts exactly as licensed

An amount written next to an ingredient must be the licensed amount per serving, or the total across strains for a multi-strain product. The checks compare every amount within 60 characters of an ingredient name with the licence.

high severity

Include every mandatory risk statement verbatim

The cautions and warnings the licence requires must be in the draft's disclaimers word for word. Paraphrases do not count: the texts are normalized and compared.

medium severity

No guarantees, no safety absolutes, no approval claims

Results cannot be guaranteed, the product cannot be called safe for everyone or free of side effects, "clinically proven" needs evidence the licence does not provide, and licensing is not an endorsement by Health Canada.

Why a DAG is not enough

The cycle

A pipeline that runs once cannot correct itself. Here the compliance validator's findings become the next draft's instructions, and the graph loops back to the synthesis agent while revision_count < max_revisions. The budget is a hard bound: when it is spent, violations go to a person instead of around again.

The checkpoint

Every node run is saved to PostgreSQL by langgraph-checkpoint-postgres. A thread that stopped yesterday can be read, resumed and finished today, by another process: the state is not in anyone's memory.

The interrupt

The graph is compiled with interrupt_before on the webhook node. The run stops with next == ("webhook_dispatch",); the review route updates the state (approved, edited, or rejected as if the node had run) and resumes. The node itself refuses to send without an approval in the state.

Deterministic tools

What can be checked by code is checked by code. The inventory lookup is a parameterized query, not model-written SQL, and the compliance checks (NPN, prohibited patterns, amounts, verbatim warnings) run after the model's audit and add to it. The audit trail says how many findings came from which.

The chain of free models

The two agents call free OpenRouter models through an ordered chain: the next model is tried only when a call fails (a rate limit, an upstream error, a withdrawal, output that does not validate), never on a wrong answer. Free models are withdrawn often, so with no models named in the settings the service looks after the chain itself. At startup it reads the chain this project's scheduled test published (one small file on a branch of the repository, no key needed). After a request that a model of the chain did not answer, it asks OpenRouter's own page for that model; if the answer is "Model not found", the model leaves the chain and the free place is filled by a test of the free models with this project's own three cases: a draft, an audit that must catch planted violations, and an audit that must pass a compliant draft. The scheduled test runs the same cases on GitHub Actions and publishes what passes. Three rules: repair, never rank; a chain holds at most three places; a model must pass every case.

What it runs on

PartThis demo (free)Drop-in alternative
OrchestrationLangGraph StateGraph with conditional edges, a cycle and interrupt_beforeThe same graph; the nodes are plain async functions
APIFastAPI, Pydantic v2 request and response models, a body limit before parsingAny ASGI host
Checkpoints and inventoryPostgreSQL on Neon, through one psycopg pool shared with the checkpointerAny PostgreSQL 14+; CHECKPOINTER=memory for a laptop
ModelsAn ordered chain of free OpenRouter models, with fallback when one failsClaude through the Anthropic SDK (LLM_PROVIDER=anthropic), OpenAI or any Chat Completions endpoint (openai_compatible)
WebhookHMAC-SHA256 signed, delivered to the service's own receiver (WEBHOOK_TARGET=self)Any HTTPS endpoint (WEBHOOK_TARGET=url), or a dry run
HostingRender free web service, Neon serverless PostgreSQL, GitHub Pages for this pageDocker Compose (PostgreSQL 16 and the API), any container host

The public demo is deliberately closed in three ways: runs are rate limited per visitor and capped per day, the number of self-correction loops is capped, and the webhook can only reach the service's own receiver.