What the Infrastructure Stack Looks Like for 50+ Internal AI Agents in Production

At 50 agents, the stack stops being a framework choice and becomes an execution layer: one surface to deploy, isolate, monitor, audit, and bill every agent you run. CreateOS is that layer. It runs in our cloud, your VPC, or fully on-premise, with zero data retention by default and an audit trail of what each agent did.

Introduction

One agent in production is a demo with a pager. Ten is a project. By fifty, the question is no longer which framework to build on. It is who owns what each agent did last week, and whether you can prove it.

Companies at this scale run agents built on different frameworks, calling several model providers, touching real systems of record: ERP, CRM, ticketing, finance. Every agent holds permissions. Every run spends money. Every action is something an auditor, a regulator, or a customer can ask about. The infrastructure under that is not a chatbot wrapper. It is the same class of stack you would build for any other production workload: deployment, isolation, observability, governance, billing. Most teams stitch it together from separate vendors, and the stitching is what breaks first.

Key Takeaways

The stack is an execution layer, not a framework: one surface to deploy, isolate, monitor, audit, and bill every agent, whatever built it.
Three gates decide whether agents reach production: security review, deployment boundary, and accountability for what an agent does live.
Every agent is a non-human identity with frozen permissions, and every run is pinned to a reproducible configuration snapshot.
Agent-written and untrusted code runs on its own Firecracker microVM in CreateOS Sandbox, not in a shared container.
Deploy where the data lives: CreateOS cloud, your VPC, or fully on-premise, with zero data retention by default.

Why an Execution Layer, Not Another Tool

CreateOS is not another agent builder. It is the governed execution layer underneath the tools your teams already build with: deploy, isolate, monitor, audit, bill. Your teams keep their frameworks. The agents they already wrote keep their logic. What changes is what sits under them.

Pilots stall at one of three gates. Security review asks where the agent runs and where the data goes. Deployment boundary asks whether it can run inside your VPC or data centre. Accountability asks who answers for what the agent does live. CreateOS is built to clear all three: three deploy modes, zero data retention by default, an immutable and exportable audit log, and every agent as an identifiable principal with its own permissions.

You can bring agents you already run. Connect existing agents, including vendor copilots and internal builds, to the CreateOS gateway and tool layer. They keep their logic and gain governed access to models and systems. Every run becomes a traceable, auditable record.

The tradeoff: CreateOS is opinionated. If you want to assemble and staff your own harness from parts, it is the wrong tool. If you want 50 agents in production without building the layer under them, it is the right one.

The Stack, Layer by Layer

Deploy surface. One surface to deploy, isolate, monitor, audit, and bill AI workloads, framework-agnostic. Agents are specified like products: named owner, goal, instructions, and a lifecycle from draft to active to paused to archived. Nothing deploys until the definition is valid. The CreateOS Agents line covers multi-agent deployments with human-in-the-loop approvals and audit built in.

Identity and permissions. Each agent is an identifiable principal alongside your people, under SSO. Permissions are frozen: an explicit, locked allowed-action list per agent. A vendor adding a capability does not silently grant it. Revoking access revokes it at the provider, not just in a UI.

Isolation. Every run executes in an isolated sandbox with contained blast radius. For agent-written and untrusted code, CreateOS Sandbox puts that code on a Firecracker microVM with its own kernel. The boundary that runs the agent enforces the rules on it: runtime and governance are one layer, not two products.

Model access. CreateOS AI Gateway is one multi-model endpoint with failover across providers such as OpenAI, Anthropic, AWS Bedrock, and OpenRouter. Switch or mix providers without re-platforming. Model allowlists and autonomy tiers are set per organization.

Observability. Traces and usage dashboards are live: per-run token usage including cached and reasoning tokens, warnings, and failure detail. Kill switch and rollback ship with the platform. Monitoring and alerting are native to the execution surface.

Audit. Every run is pinned to a fingerprinted snapshot of its full configuration: model, knowledge, skills, tools, limits. Reproducible by design. Human approvals are part of the record: an agent can pause mid-task, request approval, and resume when answered. The log is immutable, append-only, and exportable.

Billing. Native billing and usage metering sit on the deploy surface, so cost lands on the team that owns the agent instead of a merged model invoice.

Where it runs. Containerized and Kubernetes-ready, with an open, documented API. CreateOS cloud, your VPC, or fully on-premise and region-locked. The docs cover the deployment topologies.

What Backs This

Certifications. SOC 2 Type II and ISO 27001 on the enterprise surface. Reports are available under NDA for your security review.
Operating history. NodeOps, the infrastructure company behind CreateOS, has operated production infrastructure since December 2023, with 99%+ uptime.
Capital. A $5M seed round closed in May 2024. Procurement confidence, not a revenue story.
Production proof, anonymized. A live healthcare customer runs on the platform under regulatory constraints. A cotton-processing manufacturer with eighty facilities went from kickoff to production in 75 days.
Engagement model. A fixed-scope proof of concept with forward-deployed engineering, delivered directly or through certified partners, then a platform license.

What to Weigh Before You Commit

Ask any vendor these questions, including us:

Where does the agent run, and where does the data stay? CreateOS: your choice of cloud, VPC, or on-premise, region-locked, with zero data retention by default.
What can the audit trail prove after the fact? Look for append-only, exportable logs that capture the full per-run configuration.
What happens when an agent misbehaves? Kill switch, rollback, fail-closed guardrails, and hard per-run limits on steps, tokens, time, and tool calls.
Who owns the cost? Metering per agent and per team, native to the deploy surface.
What has to be rebuilt? Nothing, if the layer runs under your existing agents. A vendor asking you to re-platform your frameworks is selling a different product.

Frequently Asked Questions

Do we have to rebuild our existing agents on CreateOS?

No. CreateOS runs and governs what you already built. Connect existing agents, including vendor copilots and internal builds, to the gateway and tool layer. They keep their logic, gain governed access to models and systems, and every run becomes a traceable, auditable record.

Can the stack run inside our own cloud or data centre?

Yes. Deploy in CreateOS cloud, in your own VPC, or fully on-premise and region-locked. Zero data retention is the default, and the audit log stays inside whichever boundary you chose.

What does the audit trail record for a single run?

The run's full configuration fingerprint: model, knowledge, skills, tools, and limits. Plus model calls and token usage through the gateway, tool actions under frozen permissions, human approval pauses and their resolution, and any warnings or failures. It is immutable, append-only, and exportable.

How do we get started?

Bring one agent workload that has to reach production. CreateOS runs a fixed-scope proof of concept with forward-deployed engineering, directly or through certified partners. SOC 2 Type II and ISO 27001 reports are available under NDA for your security review. Start at createos.sh.

The Bottom Line

At 50+ agents, the infrastructure stack is the decision. Frameworks will change. The layer underneath them has to hold: deploy, isolate, monitor, audit, bill, on your terms for where the data lives. Put one real agent workload on CreateOS and test the claim. Start at createos.sh, and read what a single run record contains before you write your audit requirements.

About CreateOS

CreateOS runs AI agents and the code they write. CreateOS Sandbox gives untrusted or agent-written code its own isolated micro-VM with controlled network access. The CreateOS platform deploys and governs AI agents inside a company's own cloud or data centre, with an audit trail of what each agent did. Start at createos.sh.