Three of the world's most powerful technology companies just showed their hands on the same question within 72 hours: where will AI agents actually run? On October 21, NVIDIA launched the OpenShell Reference Architecture at GTC Berlin. On October 22, Google announced Agent Compute Engine at AGNTCon in San Jose. On October 23, AWS and UC Berkeley open-sourced Infinity, a Rust runtime that fits 75,000 concurrent agents into 8 GB of memory. The timing is not coincidental. The message is unanimous: the factory floor for AI agents is being built, and the companies building it want to own the foundations.
Why this matters now
Until this week, the agent conversation was dominated by models: which one reasons best, which one is cheapest, which one hallucinates least. Those questions still matter. But they've been overtaken by a more fundamental one — where do agents live when they're running, waiting, pausing, and resuming across hours or days of work?
The answer used to be "wherever your code runs." A container. A Lambda function. A long-running server. But agents are different from traditional workloads. They burst into activity for seconds, wait for minutes or hours on tool calls and human approvals, then burst again. They execute AI-generated code that may be adversarial. They need persistent state across sessions that can last days. Standard compute primitives weren't built for this lifecycle, and the cracks are showing as enterprises try to move from pilots to production.
A Google Cloud survey of 1,402 global IT leaders found that 83% of organisations require infrastructure upgrades to support production-grade autonomous systems. Mordor Intelligence estimates the enterprise agent infrastructure market will grow from US$0.55 billion in 2026 to US$4.45 billion by 2031 — a 52% compound annual growth rate. The three announcements this week are the opening moves in a race to capture that market.
What each company built — and how they differ
The three approaches share a common premise: agents are untrusted workloads that need isolation, governance, and efficient lifecycle management. But they diverge sharply on where those guarantees come from.
Google: managed microVMs with checkpoint and resume
Agent Compute Engine gives every agent a hardware-isolated microVM with its own guest kernel, root access, and persistent Hyperdisk workspace. It was built to power Gemini internally and is now being opened to customers. The standout feature is checkpoint-and-resume: when an agent finishes a tool call and enters a waiting state, its microVM is suspended. When work arrives, it resumes from a snapshot — no rebuilding execution environments, no lost context. Google's Titanium IPU provides silicon-rooted security, and long-lived credentials stay outside the sandbox, injected by Agent Gateway only after policy checks.
The design philosophy is explicit: "Assume an attacker controls everything inside the sandbox, and make sure they find nothing there to use," Google's engineers wrote. Agent Compute Engine is targeted for customer preview in the coming months.
NVIDIA: governance baked into the silicon
NVIDIA's OpenShell Reference Architecture takes a full-stack approach that extends from models down to hardware. Agents think on Vera Rubin GPUs, act on Vera CPUs purpose-built for agentic workloads, and are governed by BlueField-4 DPUs running NVIDIA Sentry — an out-of-band monitoring system that can quarantine a rogue agent in milliseconds.
"AI capability and safety must advance together," said Jensen Huang, founder and CEO of NVIDIA. "As enterprises prepare to deploy billions of AI agents, safety and control must be built into the computing architecture." The partner list is a who's who of enterprise technology: Anthropic, AWS, Cisco, CrowdStrike, Google Cloud, IBM, Meta, Microsoft, OpenAI, Palo Alto Networks, SAP, ServiceNow, and Siemens are all working with the architecture.
OpenShell itself is open source. Its policy prover uses formal verification to mathematically check whether a proposed policy change would grant risky new access — a level of rigour uncommon in agent governance. The approach is distinctly NVIDIA: sell the silicon that makes the governance possible, give away the software.
AWS: lightweight, open-source, serverless-native
Infinity takes the opposite approach to Google's managed microVMs. Instead of giving each agent a whole virtual machine, it treats agents like asynchronous coroutines. When an agent calls a tool, Infinity saves its conversation state, ends the execution slice, and resumes from saved state when the result arrives. Between turns, an agent is pure data — no task, no stack, no open connection — at roughly 100 KB per idle agent.
The 75,000-agents-in-8-GB headline comes with an important asterisk, as alfw.ai noted: the benchmark ran on a server-class AMD EPYC processor with simulated model calls and in-memory stores, not on an actual Raspberry Pi. Real agents accumulate larger conversation histories, and production workloads will fit fewer. But the architectural contribution is genuine — treating tool calls as suspension points eliminates the cost of idle compute.
Three philosophies, one signal
Read the three announcements side by side and a pattern emerges. Google is betting that agents need their own computers — lightweight, fast, but fully isolated. NVIDIA is betting that governance needs to live below the software stack, in silicon that agents cannot compromise. AWS is betting that agents should cost nothing when they're not thinking, and the runtime should be invisible.
These aren't mutually exclusive. An enterprise might run agent code on Infinity's lightweight runtime, inside a Google-style microVM for isolation, governed by NVIDIA's silicon-level enforcement. The real competition isn't between runtimes — it's between ecosystems. Each announcement is a bid to define the layer that enterprises depend on most, and therefore the one hardest to leave.
This is the pattern we've seen before with ServiceNow's AI Passport for cryptographic agent governance and IBM's multi-agent control plane. The infrastructure layer for agents is consolidating fast, and the choices enterprises make now will shape their architecture for years.
What this means if you run a business
If you're a business owner who's been focused on which AI model to use, this week is a signal to widen the lens. The model matters. But increasingly, the infrastructure layer — where your agents run, how they're secured, and what happens when they wait — matters just as much.
That statistic, from a MarketsandMarkets analysis of the multi-agent orchestration market, explains why three of the world's largest companies are investing so heavily. The bottleneck to agent adoption isn't the models. It's everything around the models: security, persistence, credential management, lifecycle efficiency, and audit trails.
For Australian businesses in particular — where AI adoption already lags behind comparable economies — this creates both risk and opportunity. The risk is that agent infrastructure becomes yet another layer of complexity that slows adoption. The opportunity is that purpose-built runtimes and governance tools dramatically lower the bar for deploying agents safely. When the infrastructure is managed, you don't need a platform team to figure out agent isolation from scratch.
What to watch
Three things will determine whether this week's announcements translate to real business impact. First, interoperability: Forkast's analysis of AGNTCon noted that while individual governance stacks are shipping, the connective tissue between them remains "a series of drafts and blueprints." Agents that work seamlessly across Google, AWS, and NVIDIA environments are still theoretical.
Second, pricing. Agent Compute Engine's checkpoint-and-resume model could eliminate idle compute costs, but Google hasn't published pricing. The economics of running agents at scale — where every tool call is an inference round-trip and every waiting period is potential waste — will be the deciding factor for most businesses.
Third, the Five Eyes security guidance on agentic AI has made it clear that governments expect agent deployments to meet serious security standards. Infrastructure that bakes in governance — rather than bolting it on — will have a structural advantage as regulation arrives.
The factory floor for AI agents is being built. The question for every business is not whether to eventually stand on it — but how long you can afford to wait.
Sources
- Introducing Agent Compute Engine and expanding agent governance — Google Cloud Blog
- Agent Compute Engine deep dive — Google Cloud Blog
- NVIDIA Launches OpenShell Reference Architecture for Safe AI Agents — NVIDIA Newsroom
- How NVIDIA OpenShell Reference Architecture Secures and Governs AI Agent Workloads at Scale — NVIDIA Technical Blog
- Introducing Infinity: a runtime for extreme agent concurrency — AWS Open Source Blog
- GTC Berlin 2026: Live Updates From Jensen Huang's Keynote — NVIDIA Blog
- Google targets AI agents' idle-time costs with new compute service — InfoWorld
- AWS Infinity: 75,000 Agents, but No Pi in the Test — alfw.ai
- The AGNTCon Ledger: What Shipped This Week and What Is Still a Slide — Forkast
- Enterprise Agent Infrastructure Market Report — Mordor Intelligence
- Multi-Agent Orchestration Platform Market — MarketsandMarkets
- 2026 State of infrastructure in the agentic AI era — Google Cloud
