AI & Machine Learning · Cloud · · 7 min read

The Neocloud Problem Isn't GPUs. It's Everything Around Them.

I’ve been skeptical of the long-term neocloud opportunity for a while. I’ve compared them to Rackspace. Not because I question the demand for GPUs or the ability of companies like CoreWeave to operate them efficiently.

It’s because enterprise AI applications need much more than GPUs.

Microsoft Ignite 2026 reinforced that argument. The keynote was packed with announcements, several of them general availability, but what caught my attention wasn’t any single product. It was how the pieces fit together.

Foundry, the MAI models, HorizonDB, Cobalt 200, and the broader Azure application platform tell a story Microsoft didn’t explicitly call out.

Hyperscalers already have the infrastructure enterprise agentic AI needs. Neoclouds are trying to build their way toward it.

Agentic AI changes the infrastructure problem

I’ve spent a lot of time in my lab exploring the economics of inference. Two DGX Sparks, Apple silicon, CPU inference, cloud GPUs, different models, and different serving frameworks.

One question keeps coming up: how much infrastructure do I actually need to accomplish a useful business task?

My Clinic lab takes that question beyond tokens per second. It’s a simulated medical scheduling application with 1,000 synthetic patients. I’ve run hundreds of thousands of conversations, testing how different models handle scheduling, payment, consent, and ambiguous instructions.

The lesson isn’t that GPUs are unnecessary. It’s that inference is only one part of the application.

Consider what an agent does when scheduling an appointment. It checks eligibility, queries availability, interprets patient preferences, enforces business rules, and updates the scheduling system. Some of those operations need model intelligence. Others need deterministic code.

Now scale that from one scheduling application to thousands of enterprise workflows running concurrently.

Every agent is potentially generating API calls, database transactions, identity checks, tool executions, and requests to other agents.

Agentic AI doesn’t just increase demand for inference. It creates enormous demand for application infrastructure.

And that infrastructure is precisely what hyperscalers have spent decades building.

Microsoft already has the application platform

Satya Nadella spent part of his keynote on primitives that have nothing to do with GPUs: micro VMs that boot in under a second as isolated sandboxes, and an agent grid that controls what each agent can reach.

Foundry provides access to models, evaluation, training, optimization, and the same managed agent harness Microsoft uses for Copilot.

Microsoft IQ and Fabric connect agents to enterprise context. HorizonDB, a managed PostgreSQL service that went GA at the keynote, addresses transactional data. Entra and Agent 365 bring identity, policy, and management into the picture.

And Azure already provides the compute, networking, storage, and application services these agents need.

Microsoft doesn’t have to build an enterprise cloud around its AI platform. It already has one.

The GitHub example from the keynote illustrates the point. Nadella described a test that pointed 5,000 agents at a single repository and sustained 7,200 writes per second, after rearchitecting core Git infrastructure by decoupling storage and compute.

That’s not a GPU benchmark. That’s the kind of application scaling problem agentic AI introduces.

Cobalt 200, now generally available, fits into that same story.

I’m not suggesting Cobalt replaces GPUs for inference. My CPU inference experiments have demonstrated how difficult that proposition can be.

Cobalt matters because agents execute tools, run code, interact with databases, and invoke application services. Those operations need general-purpose compute at scale.

Microsoft is optimizing the infrastructure surrounding inference, not just the infrastructure performing inference.

Good enough intelligence changes the economics

Mustafa Suleyman claimed MAI Code 2 delivers 98% of GPT-6.1 Sol’s performance at 40% of the cost on everyday enterprise coding benchmarks.

I haven’t validated those claims, and Suleyman himself said the models aren’t yet tuned for long-horizon agentic work. But the approach is compelling.

Not every enterprise task requires a frontier model.

My Clinic experiments have reinforced the importance of separating model interpretation from deterministic business decisions. I don’t necessarily need the most intelligent model available to determine what a patient is asking. I need a model that performs that bounded task reliably, at acceptable latency and cost.

The Alaska Airlines demo showed the same pattern. Its maintenance agent started on a frontier model, then moved to a fine-tuned MAI Code 2 Flash that carried its instructions as weights instead of prompt.

An enterprise can assemble that workflow using existing Azure services without first figuring out where to rent GPUs, how to serve the model, or how to integrate another infrastructure provider.

This doesn’t eliminate GPUs from the architecture. It eliminates GPU infrastructure management as a prerequisite for building the application.

That’s a significant distinction.

Forge versus Foundry isn’t the real competition

CoreWeave’s Forge is interesting because it recognizes that providing GPU capacity isn’t enough.

Forge moves CoreWeave into fine-tuning, inference, evaluation, observability, and continuous optimization.

In October I wrote that nothing in Forge looked hard for AWS, Microsoft, or Google to copy. When Microsoft walked through Foundry’s improvement loop with Alaska Airlines, I immediately saw the overlap.

But a feature comparison between Forge and Foundry misses the bigger architectural question.

CoreWeave is building an AI application platform on top of specialized GPU infrastructure. Microsoft is adding AI capabilities to an existing hyperscale application platform.

Those are very different starting points.

For a Microsoft shop, enterprise applications already run in Azure. Their data lives in databases and storage services. Their identities and policies are integrated with enterprise systems. Their networks, monitoring, and operational processes are established.

An agent that needs to interact with those systems has a natural place to run.

Why introduce another cloud boundary unless there’s a compelling reason?

A lower price per token might be that reason for some workloads. Dedicated GPU inference and frontier training remain attractive opportunities for specialized infrastructure providers.

But an enterprise agentic application isn’t simply an inference workload.

It’s an application that happens to use inference.

The metric isn’t cost per token

This is where I think much of the neocloud conversation misses the point.

We’re accustomed to comparing AI infrastructure using tokens per second, time to first token, GPU utilization, and cost per million tokens.

Those are useful infrastructure measurements. I use them in my labs.

But the enterprise doesn’t ultimately care how cheaply an agent generates tokens. It cares whether the agent completes the business process correctly and what that outcome costs.

The economics include the model, but also tool execution, orchestration, database transactions, retries, human intervention, and the cost of operating the entire application.

Moving inference to a neocloud might save money on GPU compute while adding complexity elsewhere.

That doesn’t make external inference a bad architectural choice. It means the savings need to be evaluated against the complete workflow.

So I’m going to measure it. The next round of my Clinic lab prices the cost of a correctly completed scheduling task instead of the cost of a token. That means inference, the round trips to wherever the model runs, tool calls, retries, and the human review the failures require, all at a fixed quality bar. If inference across a cloud boundary still comes out ahead on the complete workflow, I’ll say so.

And hyperscalers have a substantial advantage because so much of that workflow already runs on their platforms.

Where does that leave neoclouds?

I don’t believe neoclouds disappear. There’s a substantial market for specialized GPU infrastructure, dedicated inference, and frontier model training.

But there’s a difference between supplying compute to an AI application and becoming the primary platform on which enterprise AI applications run.

Microsoft is the clearest example of the difference. It accounted for roughly 67% of CoreWeave’s 2025 revenue, according to CoreWeave’s annual report, and it signed a multi-billion-dollar capacity agreement with Nebius. Microsoft buys neocloud GPUs and sells the platform. That’s what a supplier relationship looks like.

That distinction becomes more important as agentic AI matures.

The neocloud proposition emerged when access to GPUs was the infrastructure bottleneck. Enterprise agentic AI is introducing a different bottleneck: executing and coordinating enormous numbers of application operations against existing enterprise systems.

Ignite made clear that hyperscalers are investing across that entire problem.

CoreWeave and other neoclouds can certainly build more application services. Forge demonstrates that ambition. But they’re competing against platforms with decades of investment in enterprise application infrastructure, established customer workloads, and existing data gravity.

That’s a difficult position from which to become the next enterprise cloud.

Neoclouds are optimized for the infrastructure demands of AI models. Hyperscalers are optimized for the infrastructure demands of enterprise applications.

Agentic AI needs both.

But as models become components within increasingly complex business workflows, I expect the second advantage to matter more.

Microsoft didn’t announce the end of neoclouds at Ignite.

It demonstrated why enterprise agentic AI may never need them as its primary infrastructure platform.