Individual · Backend platform prototype

AgentOps

A multi-tenant control plane for configuring, running, and observing AI-agent workflows. I built durable run state, append-only events, a Redis-backed live update path, usage and cost tracking, approvals, evaluations, and an external trace API.

Scope
Individual
Status
Prototype
Stack
Python · FastAPI · PostgreSQL · Redis · WebSockets

01 Context

The problem

Agent workflows need more than a model call. Operators need a durable record of what ran, which configuration was used, what tools were called, how an interrupted run resumes, and how clients recover missed updates—without weakening tenant boundaries.

My role

I designed and implemented the platform individually, from tenant-scoped APIs and persisted execution through event delivery, evaluation flows, and defensive tool execution.

  • Designed tenant, project, authentication, and API-key boundaries.
  • Built idempotent persisted runs, append-only events, and the outbox relay.
  • Added live WebSocket catch-up, usage/cost records, approvals, evaluations, and trace ingestion.

02 Architecture

System at a glance

AgentOps system flowRequests enter the API and durable PostgreSQL run state. A worker consumes persisted work. Committed outbox events flow through Redis and authenticated WebSockets to reconnecting clients.APItenant boundaryPostgreSQLruns · events · stateWorkerpersisted workOutboxcommit boundaryRedis relaylive updatesWS clientscatch up · resume
Requests enter the API and durable PostgreSQL run state. A worker consumes persisted work. Committed outbox events flow through Redis and authenticated WebSockets to reconnecting clients.

03 Decisions & trade-offs

Durable state before orchestration theatre

Runs are persisted with idempotency at the project boundary. Each step and state transition becomes an ordered event, so the worker can resume database-backed work and the API can reconstruct a run without relying on process memory. This is also why I do not describe the current execution path as Temporal-backed: Temporal appears in planning and local compose material, but the public worker is database-backed.

Versioned configuration publication makes a completed run explainable later. A run points to the configuration that actually executed instead of a mutable draft that may have changed since.

One event history, two delivery paths

The database is the durable source of truth. Append-only run events and an outbox capture committed changes; a relay publishes them through Redis for low-latency delivery. Authenticated WebSocket clients can receive the live path, then reconnect with sequence information and catch up from persisted history when they miss messages.

This separates delivery speed from correctness: Redis can accelerate updates, but a transient relay or connection failure does not erase the run record.

Trust boundaries are part of the feature

Tenant and project scope are carried through authentication, API keys, and persisted records. External tool execution validates inputs and outputs, applies timeout and retry rules, normalizes responses, and includes safeguards against server-side request forgery. Usage, pricing snapshots, redaction, approvals, evaluation results, child runs, and handoffs remain inspectable as part of the trace rather than disappearing into logs.

The current trade-off is intentional: the platform demonstrates the control-plane model and durable data flow, but it does not claim production operations, comprehensive coverage, or a migration to a workflow engine that the public runtime does not yet use.

04 Evidence

Inspect the work

Current status & limitations

  • Execution currently uses a database-backed worker; Temporal is not the public runtime.
  • The project is a backend prototype with no claimed deployment, customers, or scale benchmark.
  • Test coverage is narrow, and one migration had a formatting discrepancy when verified.