Agent Substrate

1 · What it is

An AI agent spends most of its time waiting: for a user’s next message, a tool’s result or a model’s reply.

Agent Substrate runs on Kubernetes, which starts and manages software in pods, the smallest units Kubernetes can create and manage. If every agent gets its own pod, that pod’s memory and processor stay reserved while the agent waits. Running each agent in a sandbox, an isolated environment, makes it safer but does nothing about this waste.

The project’s answer is to separate the agent from the pod. (The project calls each managed agent an actor.) An agent is always in one of two states:

• Suspended: saved as a snapshot of its memory and files in snapshot storage. It holds no worker pod.
• Resumed: restored onto any free worker pod, one of a pool started in advance, and ready to handle requests. The project says resuming takes under 500 milliseconds. Its roadmap targets 100 milliseconds for 95% of resumes.

The surprising part is what Kubernetes sees: only the fixed pool of long-running worker pods. Agents move in and out of those pods without ever waiting for the Kubernetes scheduler, the part of Kubernetes that decides which machine runs each new pod.

How agents are kept apart

The design isolates every agent, although the project’s own threat model says security hardening has barely begun.

• Sandbox: Each agent runs either under gVisor, which isolates it at the kernel level, or in a lightweight virtual machine such as Kata Containers.
• Inbound traffic: Requests never reach an agent directly; they pass through an authenticated entry point in Agent Substrate.
• Secrets: An experimental gateway adds API keys to outgoing HTTPS calls, so the agent never holds them.

Main components

• ateapi (control plane): It decides which worker pod each agent runs on and tracks each agent’s lifecycle.
• atelet (node helper): It runs on each machine and moves snapshots between worker pods and snapshot storage.
• atenet (router): It reads the target agent from the request header, a labelled field sent with the request, and resumes that agent before forwarding the request. It is built on the Envoy proxy. If every worker pod is taken, it holds the request briefly instead of failing at once.
• PostgreSQL (state store): It holds agent records instead of etcd, the key-value store for Kubernetes’ own data, because those records change too often.

Who builds and uses it

Google built Agent Substrate with its Kubernetes Engine team. Contributors also come from NVIDIA, Solo.io and Microsoft. The Cloud Native Computing Foundation (CNCF) has voted to accept it as a Sandbox project, pending a signed contribution agreement. Sandbox is the name of a CNCF project level, not an agent sandbox.

Agent Executor, from Google, builds on Agent Substrate and adds durable execution, so long tasks can resume after outages or interruptions. kagent, a Kubernetes-native agent framework, also runs its agents on Agent Substrate.

What to check before production

Check four things before relying on it in production:

• Exposure: Keep it internal. The project expects Agent Substrate to sit behind your own service. Its threat model recommends firewalls to block outside traffic.
• Capacity: Size the worker pool for your busiest minute. When every worker pod is taken, a held request fails after a short, bounded wait.
• Operations: Keep the control plane available, secure and fast.
• Debugging: Plan per-agent logs and traces before the first incident. Agents move between worker pods all the time.