Selected AI Work

Multi-agent system · LangGraph + deep agents · Human-in-the-loop · MCP

Batta Engineering: A Multi-Agent AI Software Team That Ships Real Pull Requests

Personal AI product · Built solo, Oct 2026 · Phases L0–L3 complete (backend loop, Swift loop, web console, editors) · L4 next: Railway and the Batta bridge

A small AI software team that works on my Batta assistant like real colleagues would. The Lead plans and reviews, the Engineer writes the code, the Tester runs the checks and a cheap Code searcher finds things for the others. I approve the plan, they do the work in a sandbox, and I approve the merge. Everything they do, think and spend shows up live in a web console.

Role
Solo builder · AI engineer · Architecture to web console
Year
2026
Focus
Multi-agent system
Stack
10 technologies
  • LangGraph
  • Deep agents (LangChain)
  • Multi-agent orchestration
  • Gemini · Claude · GPT · Ollama
  • MCP server
  • FastAPI
  • PostgreSQL
  • Docker sandbox
  • Clean architecture
  • HTML/CSS/JS console
Full repositoryWeb console (client UI)Agents (deep agents)Workflow (LangGraph)API (FastAPI)MCP server (editors)Architecture doc

  • 4

    Specialised agents: Lead, Engineer, Tester, Code searcher

  • 4

    Pull requests merged on the real Batta app

  • $0.52

    Average model cost per task (763 calls, 19 tasks)

  • 3

    Human gates: plan, merge, extra budget

  • 4

    LLM providers per agent: Gemini, Claude, GPT, Ollama

  • 299

    Automated tests, incl. an architecture test

Web console (client UI)

Screens from the running console on the Mac mini, with the team's real tasks and spend · tap to enlarge

Overview: pipeline and live activity
Task thread: plan → PR → merge
Agents and models
Providers and keys
Team memory (approved lessons)
Spend, budgets and stop switch

The problem

Coding agents are impressive in a demo and risky in a real repository. They wander, rewrite things nobody asked for, run up a bill, or claim tests pass when they don't. A business that wants an AI team needs the opposite: predictable steps, a human in charge of the decisions that matter, a hard ceiling on cost, a record of everything, and the freedom to swap models as prices and quality change. Batta Engineering is my answer to that, built and used on a real product: my Batta voice assistant (Swift apps plus a FastAPI backend).

Overview

A task ("add a test for the Gmail helper") can come from the web console, the eng CLI, or an editor through MCP: VS Code, Xcode or Claude Code. Next comes Batta itself, by voice. The task becomes a thread, like a team chat, where every agent message, tool call, approval card and cost appears live.

A worker on a Mac mini picks the task up and runs a fixed LangGraph workflow: prepare a git worktree → the Lead plans → I approve the plan → the Engineer implements → the checks run → simulator screenshots for UI changes → the Lead reviews the diff → a pull request opens → I approve the merge → it squash-merges after green CI. Failed checks, review changes or a rejected merge send it back for a fix round, up to three times.

The agents are LangChain deep agents (create_deep_agent), each with a narrow role, its own model, the project's own rules (its CLAUDE.md), and lessons learned on earlier tasks. They read and write files only inside the task's worktree, and run commands only through an allowlisted Docker sandbox with no network. Swift and Xcode commands go to a host runner that holds a lock so they run one at a time.

The decisions that matter are made in code, never by the model: what runs next, whether a call fits the budget, which commands are allowed, and who approves. Approvals are one-time tokens created only by a human tap. The agents never push, merge or approve anything.

Any LLM per agent. Gemini, Claude, GPT and local Ollama models plug in through one factory, so each role has a ladder (normal model → stronger top tier, which needs a tap) and a fallback. A model trial runs the same task with two models side by side and compares tests, review and diff size before you switch. API keys are encrypted in the browser for the Mac mini and stored only in its Keychain.

Every model call is checked before it runs (task budget, monthly cap, stop switch) and recorded after, including prompt-cache hits at their real price. When a task runs out of budget it pauses and asks for more instead of failing or overspending. In the first month the team made 763 model calls on 19 tasks for $9.83, about $0.52 per task.

Architecture

System architecture

Four ways in, one API that owns every decision, and a worker on the Mac mini that does the work. The worker never decides anything on its own: it asks eng-api before every step, every approval and every model call. Each layer links to its code.

  • Channels
  • Service
  • AI agents
  • Data & infra
  • External API

  1. Clients (the UIs)

    console/static · cli · mcp_server · Batta (L4)
    Web console code

    • Web console

      Six screens: Overview, Tasks and chat, Agents and models, Providers and keys, Team memory, Spend and budget. Plain HTML, CSS and JS with no build step.

    • eng CLI

      Create and follow tasks, approve, change models, see spend, pull the stop switch.

    • MCP server

      eng-mcp gives VS Code, Xcode and Claude Code six tools, such as create_task, read_thread and waiting_for_you. Editor tokens can't approve or spend.

    • Batta (next)

      An engineering_team tool in the voice assistant: "ask the team to fix the weather card", approve with a tap.

    HTTPS JSON · session cookie + CSRF header · editor or worker tokens

  2. eng-api

    interfaces/api · FastAPI · Docker (Railway in L4)
    API code

    • Tasks & TeamBus

      A2A-shaped task states and threads of typed messages, parts and artifacts

    • Approvals

      One-time tap tokens (hash stored, 10-minute expiry) for plan, merge, budget and lessons

    • Model registry

      Providers, models, prices and role ladders, saved as immutable versions with rollback

    • Budget policy

      Before every call: allow, pause the task, or stop everything

    Postgres queue: claim with SKIP LOCKED · authorise every model call

  3. Worker and agents

    interfaces/worker · infrastructure/workflow · infrastructure/agents
    Agent code

    • Task slots

      Two tasks run at once, each with its own API, model and checkpoint clients

    • LangGraph workflow

      A fixed graph with interrupt() at each human gate and Postgres checkpoints, so it resumes after a restart

    • Deep agents

      Lead, Engineer and Tester, plus a Code searcher subagent they delegate searches to

    • Host jobs

      Model checks and key tests that need the Mac's Keychain, run on a second thread

    commands as argument lists, never a shell

  4. Sandbox & tools

    infrastructure/sandbox · infrastructure/workspace
    Sandbox code

    • Docker sandbox

      Allowlist (uv, pytest, ruff…), no network except installing from a lockfile, mounted on the worktree only

    • Host runner

      swift test, xcodebuild and simulator screenshots, behind a host lock

    • Git & PRs

      A worktree per task, push and gh pr create in plain code; merge only after green checks

  5. Models & data

    init_chat_model · PostgreSQL 17 · macOS Keychain

    • LLMs

      Gemini 3.5 Flash-Lite / 3.7 Flash / 3.1 Pro, Claude, GPT, and local Gemma through Ollama

    • PostgreSQL

      Tasks, messages, approvals, settings versions, spend, lessons and LangGraph checkpoints

    • Keychain

      API keys and the worker's sealing key. The database only ever sees a key's last 4 characters.

Architecture diagrams

Four diagrams drawn from the code and the repository's ARCHITECTURE.md: the system, the workflow as a state machine, the agent team with its guard rails, and the clean architecture. Tap any diagram to zoom.

  1. Fig. 01 · Architecture diagram

    System architecture

    The key split is between deciding and doing. eng-api owns tasks, approvals, model settings and money. The worker does the work on the Mac mini, where the Keychain, Xcode and the sandbox live, and checks in with eng-api before every step.

    • Every client is just a view on the same JSON API, so the console, the CLI, an editor and later a voice assistant all see the same thread
    • The Postgres queue (SKIP LOCKED) hands each task to exactly one worker slot. No Redis needed.
    • Secrets never leave the Mac mini; eng-api can move to Railway without ever holding an API key

  2. Fig. 02 · LangGraph state machine

    The fixed workflow with human gates

    The workflow is a graph in code, not a decision for the model. Agents do the creative work inside a step; the graph decides the order, the retries and when to stop. This is the same idea as a CrewAI Flow wrapped around crews.

    • Asking for an approval and waiting for it are separate nodes, because LangGraph re-runs an interrupted node on resume and a card must be posted only once
    • Three fix rounds at most, each carrying the exact failure or review comment
    • Running out of budget pauses for a tap and continues from the same step with a handoff of what's already done

  3. Fig. 03 · Multi-agent design

    The agent team and its guard rails

    Four narrow roles instead of one do-everything agent. Each has its own model from the registry, so the cheap Code searcher runs on Flash-Lite while the Lead and Engineer use Flash and can move up to Pro with a tap.

    • Delegation: the Lead and Engineer hand wide searches to the Code searcher subagent instead of reading the whole repo
    • Guard rails in code: budget callback, worktree-confined files, allowlisted commands, step limits
    • Memory that can't be poisoned: only lessons a human approved are ever read back, and every read is recorded

  4. Fig. 04 · Layer diagram

    Clean architecture

    The rules (task states, budgets, approvals, model settings) live in pure Python with no framework. LangGraph, deepagents, FastAPI and SQLAlchemy are adapters at the edge, so any of them can be replaced without touching the rules.

    • tests/unit/test_architecture.py fails the build if a layer imports an outer one or a framework
    • Use cases take ports in their constructor and are tested with in-memory fakes, with no LLM, database or Docker
    • Typed errors carry a user-facing message; the API maps them to HTTP codes

CrewAI ↔ this build

The same building blocks as a CrewAI crew

I build multi-agent systems with CrewAI (my BrokeMe research runs a six-agent CrewAI pipeline) and with LangGraph. The concepts carry over one to one. For this project I chose LangGraph with deep agents because a task can wait two days for a human tap, and LangGraph's Postgres checkpoints and interrupt() make that wait durable.

  1. 01

    Agent: role, goal, backstory

    AgentProfile + role prompt

    Each role has a profile (like an A2A agent card) and a prompt with its goal, its exact answer format and what it must never do.

  2. 02

    Task with expected output

    SUMMARY: / REVIEW: APPROVE

    Every step asks for a strict format that code parses, the same idea as a CrewAI Task's expected_output or output_pydantic.

  3. 03

    Crew process

    Fixed LangGraph graph

    A sequential process with a manager: the Lead plans and reviews, specialists execute, and code fixes the order.

  4. 04

    Flow with routers

    Conditional edges

    Like @router in a CrewAI Flow: verify ok → preview, failed → fix; review approve → PR, changes → fix.

  5. 05

    human_input

    interrupt() + tap tokens

    Human review is a hard gate in the graph, not a polite request in a prompt.

  6. 06

    Delegation

    Code searcher subagent

    Like allow_delegation to a cheap specialist: wide searches go to a Flash-Lite subagent and come back as paths and line numbers.

  7. 07

    Tools

    sandbox_tool · files

    Narrow tools with clear descriptions, run inside a sandbox the model can't escape.

  8. 08

    LLM per agent

    Model registry

    Each role picks its own provider and model, like setting llm= per CrewAI agent, but versioned and switchable from a console.

  9. 09

    Memory

    Checkpoints + lessons

    Short-term state in checkpoints, long-term lessons per project and role, approved by a human.

  10. 10

    max_iter / max_rpm

    Step limits + budgets

    Bounded runs and a money budget enforced before every call, with a stop switch for everything.

How a task runs

One task, from request to merged PR

A real run from the console: "Add a test for the Gmail helper". It cost $0.21 of a $0.50 budget and ended as a merged pull request on the Batta repo.

  1. 01

    Request

    console · CLI · MCP

    The task is saved with a size (small $0.50, normal $2, large $5) and becomes the first message of its thread.

  2. 02

    Claim

    eng-worker slot

    A worker slot claims it and creates a git worktree on eng/task-<id> from origin/main.

  3. 03

    Plan

    Lead

    Reads only what it needs (with the Code searcher) and posts a plan of about five steps.

  4. 04

    Approve the plan

    Human tap

    An approval card appears in the console. Rejecting sends the reason back to the Lead.

  5. 05

    Implement

    Engineer

    Edits files and runs focused tests in the sandbox, then the full suite once.

  6. 06

    Verify & review

    Tester · Lead

    The checks that cover the changed paths run; the Lead reviews the diff against the plan.

  7. 07

    Pull request

    Plain code

    The branch is pushed and a PR opened. The merge card lists every changed file.

  8. 08

    Approve the merge

    Human tap

    Squash-merged only after the checks are green; the branch and worktree are cleaned up.

Tools & tech

AI & agents

Each agent has one job, and its model is a setting, not code.

LangGraph
The fixed workflow as a StateGraph, with conditional edges, interrupt() at the human gates and Postgres checkpoints keyed by task.
deepagents (LangChain)
create_deep_agent for each role, a FilesystemBackend confined to the worktree, and a Code searcher subagent.
LangChain chat models
init_chat_model for Gemini, Claude (Anthropic), GPT (OpenAI) and Ollama, with per-provider settings and cache-aware cost tracking.
Prompt engineering
Role prompts with strict answer formats, the project's own rules, approved lessons, a tool-call allowance and a handoff for continued runs.
Model trials
The same task run with two models for one role; a report compares tests, review and diff size before you switch.
MCP
An MCP server (stdio) so editors and coding assistants can hand work to the team and follow it.

Backend & infrastructure

Python 3.12 · FastAPI · Pydantic v2
52 API routes, typed errors mapped to HTTP codes, Pydantic only at the interface edge.
PostgreSQL · SQLAlchemy 2 · Alembic
Queue, threads, approvals, versioned settings, spend and lessons, with 14 migrations.
Docker (OrbStack)
Postgres, eng-api and the command sandbox. The worker runs on the macOS host for Xcode and the Keychain.
cryptography · macOS Keychain
Keys sealed in the browser with RSA-OAEP for the worker; stored in the Keychain only after a real test call works.
GitHub · gh CLI · Actions
Worktrees, PRs, waiting for checks and squash-merging. The team built the Batta repo's own CI workflow.
uv · ruff · pytest
299 tests, including one that enforces the clean-architecture dependency rule.

Client UIs

Web console
Plain HTML, CSS and JavaScript served by eng-api, no build step. Live threads by polling, sign-in by a one-time link, an HttpOnly session cookie and a CSRF header.
eng CLI
Everything the console does, from the terminal.
Editors
VS Code, Xcode and Claude Code through the MCP server, with "Work on it yourself" to open a task's worktree in VS Code or Claude Code.

Modules & repos

Results

Spend by agent (first month)

$9.83 in total, 763 model calls on 19 tasks. The Tester's runs on the local Gemma model cost nothing.

  • Engineer · 541 calls$7.96
  • Lead · 216 calls$1.87
  • Tester · 6 calls$0.00

Codebase

Lines of code, counted from the repository.

  • Python (domain → interfaces)10.4K
  • Python tests4.9K
  • Web console (HTML/CSS/JS)2.9K

Key features

  • Create a task from the console, the CLI, VS Code, Xcode or Claude Code, and follow it live as a team chat
  • Approve or reject the plan and the merge with one tap; a rejection's reason goes straight back to the agent
  • Pick any model for any agent (Gemini, Claude, GPT or a free local model), switch everyone with one preset, and roll back any change
  • Try a new model safely: a model trial runs the same task twice and compares the results
  • See every dollar: spend per day, agent, model and task, budgets per task size, a monthly cap and a stop switch
  • Team memory: lessons the team proposes after a problem, saved only with your approval, and visible wherever they were used
  • Simulator screenshots in the thread for UI changes to the Swift apps
  • "Work on it yourself": open any task's worktree in VS Code or Claude Code and take over

What this means for a client project

  • I can design and build a multi-agent system around your process, in CrewAI or LangGraph: agents with narrow roles, tools they can safely use, and a workflow your team can trust
  • Humans stay in charge: approvals are hard gates in code, and agents never take an irreversible step on their own
  • Cost is controlled before it is spent: per-task budgets, a monthly cap, cache-aware cost tracking and a stop switch
  • No lock-in to one AI vendor: each agent's model is a setting, with side-by-side trials before switching
  • Production habits from day one: clean architecture, typed errors, 299 tests, Docker, migrations and an audit trail of every step
  • A UI your team will actually use: a console for approvals, live progress, models and spend, plus CLI and MCP access for developers

Safety & security

  • Text agents read (code, issues, tool output) is treated as data, never instructions
  • Commands run as argument lists in a no-network Docker sandbox, from an allowlist, on the task's worktree only
  • No secrets in code, config or the database: keys live in the Keychain and are only ever shown as their last 4 characters
  • Console sign-in by a one-time link, an HttpOnly SameSite=Strict session and a CSRF header; editor tokens can't approve, cancel or spend
  • Agents never push, merge or approve; merges wait for green checks and never happen on a failing PR

Engineering highlights

  • 01Designed and built a working multi-agent software team alone, from the domain model to the web console, and used it to ship real pull requests on a real product
  • 02A durable LangGraph workflow with human-in-the-loop gates, fix rounds and budget pauses that survives restarts and multi-day waits
  • 03Provider-agnostic agents: Gemini, Claude, GPT and local models per role, with versioned settings, rollback and model trials
  • 04Cost engineering: every call authorised and metered, cache hits charged at their real price (the first version over-counted long runs about 5×), smaller plans and focused tests to cut spend
  • 05Security by design: sandboxed commands, worktree-confined files, browser-sealed API keys and one-time approval tokens
  • 06Clean architecture enforced by a test, and 299 automated tests

Status & next steps

  • L0–L3 complete: setup, the backend loop with approvals and PRs, the Swift loop with simulator screenshots, the web console and the MCP server
  • Next (L4): eng-api on Railway, a bridge from Batta and an engineering_team voice tool, a Team screen in the app and push notifications for approvals
  • Later: customer issues from my KASI app reach Batta, which notifies me and hands the fix to the team (multi-project support)