Most AI planning tools stop at the document. You feed in an idea, and out comes a beautifully-formatted spec — a plan, a set of requirements, a technical design. Then you close the tab, because the thing you actually wanted was software, and no spec compiles.
AgentFlow is my attempt to close that last gap. It is a self-hostable web app that chains the eight artefacts a real delivery needs — Idea → Plan → Gap Analysis → BRD → TRD → Design → Build → Ship — with an AI generating each one in the context of the last, a human gate between every stage, and a Build stage that emits real source files which download as a repository you can install and run. It routes work across five cloud LLM APIs and three locally-installed AI CLIs, so a developer already paying for a Claude or Gemini subscription can run the whole pipeline at zero marginal token cost. It has a second, headless face too: a local gateway can trigger an entire idea→repo run over HTTP with no browser at all. I built it after hours over four months. This is what the architecture looks like, and — just as much — where it still doesn't work.
The documents nobody wants to write, and the repo nobody builds
Three frustrations, in the order they mattered
Every project pays the same tax. It needs a plan, requirements, a technical design, a build spec, a deployment checklist — and roughly 80% of each of those is derivable from the one before it. They are expensive to write and they go stale the moment they're written. An LLM can obviously draft them. What nothing was doing was chaining them: each document generated with the full context of what came before, and a human gate between each so a bad early call doesn't silently propagate to the end.
That was the first frustration. The second was sharper. "AI wrote a spec" is not "AI built the thing." The Build stage in the first version produced a lovely BUILD.md that described, in prose, a manifest of files to create — and then created nothing. The gap between the spec and the repository was the entire point of the project, and it was empty.
The third was economic. A developer already paying for a Claude Code or Gemini CLI subscription should not also pay per-token API costs for the same model to write their BRD. The intelligence is already installed on the machine. It just isn't reachable from inside a containerised web app.
The document was never the deliverable. A repository you can run is the deliverable. Everything upstream of that is scaffolding for getting there — and paying twice for the same model to build the scaffolding is a bug, not a pricing tier.
The obvious design that broke in production
Streaming a ten-minute job down an HTTP connection
The first instinct was the obvious one: stream generation straight over Server-Sent Events to the browser, so the user watches the document appear token by token. It demos beautifully. It also breaks the moment the work gets real. The Design, Build, and Ship stages take five to ten minutes each. Any navigation, any tab going to sleep, any flaky connection killed the run and lost the work with it.
Architecture decision · background jobs over streaming
The connection is gone long before the work is done.
POST …/generate now returns 202 with a jobId and a pollUrl immediately. A detached job-executor.ts runs the generation off to one side and persists partial output to Mongo every three seconds. The client polls every two seconds. Generation survives navigation, disconnection, and — up to its time budget — a slow provider. The HTTP request that started it is irrelevant by the time the model produces its first token.
The second wrong turn was quieter and more expensive. Each stage originally called each provider's REST API by hand. That meant four hand-rolled SSE parsers, four different error-code mappings, four token-usage shapes — roughly 350 lines per provider, 60% of it boilerplate. Adding one feature meant re-implementing the wire protocol four times.
Architecture decision · ADR-006 · docs/SDK_MIGRATION.md
A provider abstraction, so a stage never sees a wire protocol.
An AIProvider interface, a BaseProvider that owns timeouts, validation, cost, and error normalisation, a self-registering registry, and a router with execution modes and a fallback chain. A stage processor builds a prompt, asks the router for text, validates the shape, and returns. Swapping Anthropic for a local CLI changes zero lines in brd.ts. (Honest note: ADR-006 is approved and the abstraction is live, but the adapters underneath still use hand-rolled fetch — the migration to official SDKs is designed, not done.)
There were more of them, each one actually taken and then reversed. Dropping the CLI bridges to go SDK-only would have killed the whole zero-marginal-cost differentiator, so both paths stayed first-class. Spawning the local claude binary from inside the Linux container was impossible — it's a host binary, and mounting the host toolchain in is a sandbox violation waiting to happen — so a host-side bridge server was born (Section 07). And reaching for Redis to run the job queue would have added a whole service to a stack with a hard 2 GB RAM ceiling, so atomic findOneAndUpdate claims in Mongo do the job instead. Every one of those is a place where the heavier, more obvious option was the wrong size for the problem.
The reframing that turned prose into an executable plan
The document is not the deliverable — the manifest is
The unlock was a single constraint: BUILD.md is no longer allowed to be prose. It must contain a machine-parseable File Generation Manifest — a table of files to create, each with the specialist generator responsible for it. The moment that table exists, the Build stage stops being a description and becomes an executable plan.
# From an approved BUILD.md — a File Generation Manifest row | path | generator | purpose | |-----------------------------|----------------------|-------------------------| | src/app/page.tsx | frontend-specialist | habit list + add form | | src/app/api/habits/route.ts | api-specialist | CRUD over habits | # build-codegen.ts walks each row and generates that file with its own model call → src/app/page.tsx (one model call, manifest context) → src/app/api/habits/route.ts (one model call, manifest context)
build-manifest-parser.ts reads the table; build-codegen.ts walks it, one model call per file; repo-bundle.ts fills in the scaffolding the manifest didn't ask for.
Three files do the work. build-manifest-parser.ts reads the manifest. build-codegen.ts walks it and generates each file with its own model call. And repo-bundle.ts — a pure function — fills in everything the manifest didn't bother to specify: package.json, Dockerfile, tsconfig.json, next.config.mjs, a README, a .gitignore, an .env.example — and detects the stack while it's at it. The zip that downloads is a repository you can npm install && npm run dev.
Once the Build stage was required to contain a parseable manifest, it stopped being prose and became an executable plan. — the insight the codegen path hangs from
The second reframing is just as load-bearing: provider identity is a routing decision, not a code path. Every stage asks the router for text. The router picks by execution mode, task type, stage-appropriate model tier, and cost ceiling, and falls through a chain on retryable failures while surfacing non-retryable ones — auth, model-not-found — immediately. A stage processor contains no provider knowledge whatsoever. That single seam is why the same processors serve the interactive path, the one-shot autorun, and the headless trigger without modification.
Two clean seams, and a dashed line that explains a whole server
Eight stages, one router, two AI paths
The shape is a layered stack with two seams that carry the whole design. Stage processors know nothing about providers. The router knows nothing about HTTP. Everything else hangs off those two facts.
| Component | Responsibility | Bound |
|---|---|---|
| State machine | Eight stages with approve/reject gates; the next stage unlocks only when the previous one is approved | 8 stages |
| Job executor | Runs generation detached, persists partial output every 3s, checks cancellation every 10s | 540s outer |
| AI router | Selects provider and model by mode, task type, tier, and cost ceiling; falls through a chain on retryable errors | no HTTP |
Two more seams matter for testability. Pure cores are split from I/O shells on purpose: gateway/headless.ts (config, token check, slug, manifest shape — pure) sits apart from services/headless-new-app.ts (DB, AI, orchestration), and repo-bundle.ts is a pure function so the download route can stream its output straight into a zip while tests assert the exact file set. And every stage output is validated: malformed output triggers exactly one corrective regeneration — but only if at least 90 seconds of the job's budget remains, so a retry can never race the outer timeout.
What actually happens when you hit Generate
One idea, eight documents, a runnable repo
The everyday path is a stage generation. It is worth tracing end to end, because the interesting part is that the HTTP request is already finished long before the work is.
The autorun path collapses that into one action: a global concurrency check, then all eight processors in order — each auto-approved so the next can consume it — then generateBuildCode walks the approved manifest one file at a time, then assembleRepoBundle adds the missing scaffolding, and the run ends pointing at ?format=repo. And the headless path (Section 07) runs the very same runAutoPipeline from a token-gated endpoint with no browser and no user session at all. Three entry points, one code path, the same guarantees.
Three ways to hang, three different numbers
Three timeouts, because there are three ways to hang
A local agent CLI can think for minutes before emitting a single token. That trips every generic HTTP timeout. A demo ignores this; a thing that has actually run cannot. So the bridge carries three separate timeouts, one for each distinct failure mode.
First token 180s
How long to wait for any output at all. A CLI booting a model and reading a large context legitimately goes quiet for a while.
Silence 120s
Max gap between chunks once output has started. A stream that stalls mid-generation is dead; a stream that's merely slow is not.
Hard bound 480s
The outer ceiling on the whole call, no matter what. It sits under the job's 540s, so a provider can never outlive its own job.
Those three sit inside a custom undici Agent, because Node 20's default 300-second body timeout would kill an SSE stream that simply hasn't produced data yet. Three failure modes, three numbers, one HTTP client tuned for a workload that thinks before it speaks. That is what "we actually ran this" looks like in the diff.
The other detail in this register is subtler and cost me a real lesson. The global concurrency cap lives inside the executor, not the HTTP route. A guard wired only into the route never fires on the headless path — and the headless path is exactly the caller that can run away and spend money unattended. The route keeps a cheap early 429 AT_CAPACITY for fast feedback, but the real ceiling is enforced deeper, and countActiveRuns() is DB-backed so it holds across serverless instances where an in-memory counter would have looked correct locally and failed silently in production.
The centrepiece — one seam, two directions
The app that queues its own work, and answers when called
This is the most distinctive thing in the repo. Outside the app, on the same machine, runs a personal myAI gateway — an MCP server on localhost:3100/mcp with a dashboard on :3210. It owns three things: a task queue worked by an autonomous off-hours runner, a plan/schedule store, and a git-versioned agent memory. Every repo in my fleet speaks to it with one shared header, x-gateway-local-token. AgentFlow is the first app to speak that protocol from inside its own application code, not just from operator shell scripts — and it speaks it in both directions.
Outbound — the app queues its own work
When a human approves the BUILD stage, the approve route can hand the parsed manifest to the autonomous queue instead of generating inline:
POST http://localhost:3100/mcp
x-gateway-local-token: <shared local token>
{"jsonrpc":"2.0","method":"tools/call","id":1,
"params":{"name":"tasks_create","arguments":{
"repo":"agentFlow",
"title":"Codegen: build approved manifest for \"Habit Tracker\" (14 files)",
"priority":"P2",
"source":"agentflow-build",
"assignedAgent":"frontend-specialist",
"notes":"[build-codegen] files=14 generators=frontend×9, api×5"
}}}
The payload builder buildCodegenJobPayload is a pure function — it infers the specialist from the dominant generator column, so the exact shape the runner receives is asserted in tests whether or not a gateway is up.
Three design constraints carry the whole thing. It never throws into the request path — every call is AbortController-timeout-guarded at 5s, and every failure (disabled, unreachable, HTTP error, JSON-RPC error) resolves to a structured { ok: false, skipped }. Queueing is best-effort; it must not be able to break stage approval. It is opt-in and dark by default, enabled only when GATEWAY_MCP_URL is set — so on Vercel, where there is no local gateway, it is a no-op by construction rather than by configuration discipline. And it mirrors the shell helper's contract exactly — same tool name, same arguments, same header — so a running app and AI/scripts/schedule_task.sh land in the same queue. One queue, two producers.
Inbound — the gateway drives the app
The mirror image. The command myai new-app "<idea>" needs to run the whole idea→repo pipeline with no browser and no user session. So the app exposes one token-gated endpoint that authenticates with the same x-gateway-local-token — not a user JWT; resolves or lazily creates a dedicated service account, headless@agentflow.local, so machine-generated projects never pollute a human's list; creates the project and its eight stages, runs runAutoPipeline end to end with auto-approval, runs codegen, assembles the bundle; and returns a self-describing build manifest — projectId, slug, detected stack, stagesCompleted, filesGenerated, every repo-relative path, and both download URLs — so the gateway can register the generated repo without ever querying AgentFlow again.
Security · a later fix, worth admitting
A publicly-known default token can never open the pipeline.
The trigger refuses to run in production with the shared in-repo default token. If NODE_ENV=production and GATEWAY_LOCAL_TOKEN is unset or still equals the dev default, the config forces enabled = false and the route 404s. The first version simply trusted the flag; hardening it to fail closed came later. That is the honest order it happened in.
Right-sizing a local control plane
Every heavier option was the wrong size
The gateway shape only makes sense once you name what it deliberately isn't. It is not a public webhook API — that would mean a new auth surface, key management, and a rate-limit story for an endpoint whose entire job is to spend money on tokens. Loopback plus a shared token, dark in production, is the right size for a local control plane. It is not direct DB writes from the gateway — two writers against one schema is a migration disaster, so the app stays the only thing that understands its own models. And it is not a second queue inside the app — the fleet already has a queue, worked by a runner that knows how to branch, verify, and open a PR; adding another would just mean two schedulers competing for the same machine's tokens.
The same instinct runs one level down, into the infrastructure. Redis for the job queue would have added a service to a stack with a hard 2 GB ceiling, so atomic findOneAndUpdate claims give single-consumer semantics with no broker. A real message bus for events would have done the same, so the event bus is in-memory with wildcard patterns (stage.*, orchestration.*) feeding a notification service and an SSE manager. None of that is a compromise forced by laziness. It is a stack that fits inside its own constraints on purpose.
The gateway routes, stores, and schedules and holds no business logic — exactly the role a good message topic plays. The interesting decisions were all about what to leave out.
Five ceilings before a single token is spent
Cost control is a first-class architectural concern
Most projects in this space have nothing to say here, which is exactly why it matters. An autonomous system that spends money needs cost ceilings before it needs features, and AgentFlow stacks five independent ones.
- Tier routing. Cheap stages get cheap models.
ideais summarisation;trd,design, andbuildare hard reasoning. The tier is derived from each model's declared capability metadata — price rank within its own provider, capability flags, context window — not a hand-maintained ID list that rots every time a provider ships a new model. The hardcoded map is now only an override for models whose real strength differs from their price. - Per-stage token and dollar ceilings. A pre-flight hard dollar gate refuses any call whose projected cost exceeds the stage's ceiling. It began as warn-only; making it enforce was a deliberate later decision.
- Credit pre-flight. Before every generation, the user's plan and month-to-date usage are checked; over budget rejects with
402, and below 10% remaining it suggests a cheaper model. - Concurrency ceilings. Per-project single-flight on autorun, plus a global cap enforced inside
runAutoPipeline— not only in the route, because the headless path is the real runaway risk — and DB-backed so it survives multiple serverless instances. - Batch mode. The Anthropic Message Batches client submits whole stage sets at 50% off with 24-hour turnaround, with an estimator that shows standard-versus-batch savings per stage before you commit.
A meaningful share of one month's engineering was thrift work of exactly this kind — PR-only CI, deploy gating, local-CI equivalents, batch discounts. In agentic development, cost is not a footnote. It is load-bearing architecture.
Receipts, and the vulnerability that got through anyway
427 commits, 551 tests, and one dependency that slipped past
Four months, one person, after hours, across two Macs. The receipts are real and verifiable in-repo.
What actually works, verified rather than claimed: the eight-stage pipeline runs end to end with real providers (confirmed on a BYO DeepSeek key, plus a key-free deterministic mock that regression-tests the golden path in CI at zero cost). Approved manifest → real source files → runnable-repo zip with synthesised scaffolding and stack detection. Eight AI adapters behind one interface with five execution modes and fallback chains. Multi-agent orchestration with three dispatch strategies and three merge strategies. Background-job durability that survives navigation and disconnection. The five-layer cost stack. Encrypted BYO keys, account lockout, tiered rate limits, CSP/HSTS, zip-slip guards, a PII-redacting logger, cascading account deletion. Both gateway directions. It is a lot, and it holds together.
Lessons learnt · 2026-06-30_cycle4-day1-audit-gate-bypass.md
The gate was correct. Its delivery mechanism was the hole.
Quality was supposed to be enforced by a pre-push git hook. But the autonomous runner routinely pushes with --no-verify — the pre-push next build OOMs at the 2 GB cap (exit 137) — and a fresh runner workspace has no hooks installed at all. A HIGH-severity markdown-it ReDoS advisory reached the test branch twice through that hole.
The fix moved the gate out of the hook into a hardened audit-gate.sh with a committed self-test that seeds a real advisory and asserts the gate exits 1. "Added a gate and verified it once" was treated as done, when what mattered was whether the gate fires in the path the autonomous loop actually takes. Automation that isn't in the real execution path is decoration.
And the part most posts leave out — what is not solved, stated plainly because it's what makes the rest trustworthy. Context is still truncated, not retrieved: 3,000 chars per prior stage, 12,000 total, so by the Design stage five prior documents share a couple of thousand characters each and later stages can contradict earlier ones. A full RAG design exists (1,302 lines) and is not implemented — it is the top architectural debt. Monetisation is a stub: free and pro both grant 500k tokens, identical, with no Stripe and no tier gating; it is BYO-key only. There is no email verification or password reset in production because no email provider is wired. Production /api/health is degraded (503) pending owner-set Vercel env vars. Sentry is scaffolded but inert without a DSN. Concurrency guards are soft check-then-act, not transactional. And the codegen quality bar is "it runs," not "it's good" — one file per model call, no cross-file type-checking loop yet.
The most honest asterisk of all: the autonomous runner has been stalled for this repo since late June. Cycles 1–3 genuinely ran unattended and shipped to production via PRs #58–#75 — that happened, in the past tense. But Cycle 4's first four days were built by hand, directly, because the loop that was supposed to build them wasn't running, and nobody noticed for a while because the dashboard cheerfully showed tasks changing status while no code actually moved.
A harness, not a wish
The leverage was the scaffolding, not the model
One person producing 427 commits, 83 PRs, ~64k lines of source, 551 tests, and a real production deployment in four months is not a hobby-project pace. It is a small-team pace. But it did not come from asking an AI to write an app. It came from building a harness around the work.
That harness has a few distinct pieces. There is a routing layer for the work itself — 64 specialist agent definitions and 136 skill playbooks, with explicit rules for what runs in parallel (frontend and UI alongside devops and security) and what is strictly sequential (schema → services → fetch logic). There is durable memory instead of re-explaining — a hot STATE.md, a handoff doc, month-bucketed archives, and a git-versioned "brain" whose session atoms let the next session boot on a diff of a few hundred tokens rather than re-reading everything, across machines and even a phone. There is mechanical enforcement instead of good intentions — 52 hooks that detect machine switches, enforce the RAM ceiling, nag on stale checkpoints, and guard token budgets; the framework doesn't ask the agent to remember, it interrupts. And there is a written shipping protocol — test → CI → preview → PR → review → merge → prod, never a direct push to main, with keyword commands that expand into checklists the agent must execute and report step by step.
The honest ledger of what it cost is just as instructive. The framework-sync mechanism — shared files distributed across a fleet via Dropbox plus a sync commit — behaved exactly like the distributed-systems problem it secretly was, clobbering local fixes eighteen times; the commit log literally counts them, "(12th clobber)". A vulnerable dependency reached the integration branch twice, as above. Reviews found real bugs repeatedly — Copilot follow-up commits appear after nearly every early PR, because the AI-written code was good, not correct-by-construction. And credits were a genuine constraint on ambition: CI runs, preview deploys, and tokens are all metered, and thrift work ate a real slice of the calendar.
The leverage didn't come from the model being clever. It came from building the scaffolding that lets a clever model work unattended without being trusted.
Retrieval, verification, and a pipeline you can call
The interesting product is a machine-drivable pipeline
The near-term backlog already has designs written against it. Retrieval over truncation — replace the 12,000-char clip with selective full-fidelity retrieval over per-stage embeddings, the single change most likely to keep an eight-stage pipeline coherent. Generate-then-verify — the next jump in codegen quality isn't a bigger model, it's a loop: generate the file, type-check the repo, feed the errors back, repair. It's the same shape as the validate-and-retry that already works, one level up. The provider SDK migration behind ADR-006. Transactional concurrency guards to replace the soft ones. And, gated on owner actions I simply haven't done yet, real billing and tier enforcement.
But the thesis underneath all of it is bigger than any one feature. The interesting product was never "AI writes your BRD." It's a machine-drivable delivery pipeline — a stage graph where every artefact is addressable, every transition is gated, every run is costed and bounded, and the whole thing can be invoked by a human in a browser, by a script on the same machine, or by an autonomous runner at 3am, with the same code path and the same guarantees. Of those three, the first two work today and the third is half-working. The headless trigger returning a registerable build manifest is the smallest honest proof that "build me an app" can be a call rather than a conversation.
And there's a loop in it that I find genuinely interesting: the same off-hours runner, task queue, and ten-day plan that AgentFlow can emit codegen jobs into are also how AgentFlow is maintained. When it's running, the app that generates software is improved by the same kind of loop it's built to sell. When it's stalled, that story wears an asterisk — and the fix for that is the same lesson the whole project keeps teaching: the loop needs its own observability, because a dashboard that shows state changing without proving code moved is worse than no dashboard at all.
The best delivery pipeline is one a human can gate and a machine can call — and the honest version tells you exactly where it can't yet do either.