Runbox

I wrote the spec before I wrote a line of code, and the first section was a table: capability on the left, the roles asking for it on the right. Agent orchestration. Sandboxed execution. Real-time streaming. Usage metering. Multi-tenant isolation. Large-scale React with genuinely hard client state.

That is an unusual way to start a project and I would not recommend it for a product. But this was never a product. It was evidence, and the honest question for evidence is what it proves. So I picked the smallest single system that could prove all of it at once, and I wrote down what I was deliberately not building — no plugin ecosystem, no real billing, no autoscaling — because knowing where to stop is the part people actually test you on.

The runner is Go. Its entire job is container lifecycle, a bounded worker pool, per-run timeouts and cancellation that propagates to a child process. That is what context, goroutines and channels exist for, and doing it in Python would be a fight with the GIL for no benefit.

The control plane and the agent are Python — FastAPI for the API surface, and the agent lives where the model SDKs are. The dashboard is TypeScript because it is a live-updating UI over server-sent events.

The obvious objection is that one Go service could do all of it, and that is true. My answer is that the agent loop belongs in Python, and once it is a separate process anyway, splitting the runner out is what makes cancellation and isolation clean rather than tangled through one binary. I would rather defend that than pretend the choice was obvious.

Streaming a trace to a browser is easy. Streaming it without gaps is not.

Every event is written to Postgres and published to Redis. A client connecting mid-run needs everything already durable plus everything about to arrive, exactly once, in order. Subscribe after replaying and you drop every event produced while the query ran. Replay after going live and they arrive out of order.

So: subscribe first and buffer, then replay from Postgres, then drain the buffer discarding anything the replay already covered, then go live. Deduplication is by a sequence number that is unique per run and monotonic, which turns “already covered” from a guess into a comparison. That ordering is four lines of comment above about forty lines of code, and it is the only part of the system I would call subtle.

Everything passed. Tests green, CI green on six of seven jobs, the whole thing running happily on my machine. Then I deployed it, and three bugs surfaced in an afternoon that nothing else would ever have caught.

The first was latency. Every streamed token is a trace event, and I wrote each one to the database synchronously. Against a local Postgres that is sub-millisecond and invisible. Against a managed database in another region it measured 115 milliseconds, so a four-hundred-token run spent roughly forty-six seconds doing nothing but network round trips. Batching them into one insert every fifty milliseconds took the same work from twenty-three seconds to under one. That was not a tuning oversight — the design had an assumption in it that only localhost could hide.

The second was a flag. The agent’s Dockerfile passed pip a boolean option with a value, which is a usage error, so the sandbox image had never successfully built anywhere — not locally, where I had no Docker, and not in CI, where it was the one red job among seven. A line that looked deliberate, changed nothing, and broke the build.

The third was four digits. The per-run socket directory was mode 0700, and the agent runs as a different user inside its container, so it could not traverse in to reach the proxy. Every run failed with a permission error one step before it reached the model. The isolation was one level up all along; the directory just needed to be traversable.

None of those are interesting bugs. That is the point. The interesting thing is that a system can be fully tested, fully green, and still have three separate reasons it cannot work, all of which need a real deployment to find.

Every run gets a container with no network at all, a read-only root filesystem, every capability dropped, a non-root user, and memory, CPU and process limits. The agent reaches the model through a unix socket the runner controls, so the provider key is attached upstream and the container running untrusted code never sees it.

That is a hardened sandbox. It is not a hostile-tenant boundary, and the README says so in those words. A real guarantee against an adversary wants microVMs — Firecracker, or gVisor — which is the natural next step and is deliberately out of scope.

I think stating that plainly is worth more than the feature would be. Overclaiming isolation is the one thing that would actively hurt me in a conversation about this project, and knowing where the edge of your own claim sits is most of what makes the claim worth anything.

It runs on your key, not mine. The provider key arrives in a header, lives in Redis for the second between starting a run and a worker claiming it, and is deleted on read — never written to the database, never handed into the sandbox. That started as a way to make an open demo affordable and turned out to be the better design anyway.

The dashboard, the API, the databases and the runner are four separate deployments, because one of them needs a Docker socket and no platform-as-a-service will give you one. That constraint shaped more of the infrastructure than any preference I had.

It is open source, it is running, and the parts I would build differently next time are written down in the README under the heading of what breaks first.