Running AI at scale takes more than a powerful model. It takes huge amounts of computing power and the cost of running every prompt, coding task and AI agent keeps adding up.
Microsoft is now betting more heavily on its own silicon to handle that demand. Its Maia 200 chips are already running production workloads, and the company plans to scale the Maia family to multi-gigawatt deployments starting in 2027, with Maia 300 leading the next phase.
The goal isn’t just to build another AI chip. It’s to make running AI more efficient across the products millions of people already use.
Table of Contents
The expensive part of AI isn’t over when trained
Think about what happens when you ask an AI assistant to write code, summarize a long document or work through a task in several steps. The model has to keep generating responses, processing context and in some cases, making multiple decisions before it finishes.
Now multiply that by millions of people doing it every day.
For Microsoft, the challenge isn’t simply building powerful AI models. It’s serving them across Copilot, Teams, coding tools and other services without letting computing costs and power requirements grow out of control.
And as AI agents take on longer, more involved tasks, the amount of work happening behind a single request can grow too. Microsoft needs infrastructure that can handle that workload efficiently, not just hardware that can perform more calculations.
Microsoft Is Designing the Hardware, Too
Microsoft has already started down this path. Its Maia 200 accelerator is running production workloads, including Microsoft Copilot, and supports OpenAI models, Microsoft’s own MAI models and open-source models.
Now the company is preparing Maia 300, the third generation of its custom AI accelerator program. Production deployments are planned for 2027, with Microsoft aiming to scale the Maia family to multi-gigawatt deployments starting that year.
Why build chips in-house when companies like Nvidia already make powerful AI hardware?
Because Microsoft runs AI services at enormous scale. Designing the accelerator around those workloads gives its engineers more control over how computing, memory and data movement work together. The aim is to make the whole system more efficient, rather than simply making the chip faster.
And Microsoft has put a number on what it expects that approach to deliver.
More AI for the Same Power
Microsoft says Maia 300 is expected to deliver 40% more tokens per megawatt than leading GPU-based systems.
A token is a small unit of text an AI model processes or generates. When a model answers a question, writes code or works through a task, it generates tokens along the way. The more efficiently a data center can produce them, the more AI work it can serve with the electricity available to it.
And that electricity is a real constraint. Data centers need power for the chips themselves, but also for memory, networking and cooling. When a facility is already operating within a fixed power budget, simply adding more accelerators isn’t always an option.
That’s why Microsoft is focusing on tokens per megawatt rather than raw chip speed alone. If its estimate holds up in real deployments, the same power capacity could support more AI responses and longer-running tasks.
There is an important caveat: the 40% figure is Microsoft’s own projection, and the company hasn’t named the specific GPU systems behind the comparison in its announcement. The real result will depend on the workloads being run and how the systems perform at scale.
To understand where that efficiency is supposed to come from, we need to look beyond the accelerator itself. Microsoft has redesigned the memory, networking and rack around it, too.
System Design, Not Just a Chip
A faster AI chip doesn’t automatically make an entire AI system faster. The processor needs data from memory, and when several accelerators work together, they need to exchange information without spending too much time waiting for one another.
That becomes especially important when an AI agent is working through a long conversation or handling a task that requires several steps. The system has to keep track of growing amounts of context while moving data between different parts of the infrastructure.
Microsoft’s approach with Maia 300 is to design those pieces together.
Each chip includes eight stacks of HBM4 memory, providing 288 GB of capacity and 18 TB/s of peak bandwidth. The system also uses a chiplet design, separating compute from memory-controller and I/O functions so that different parts of the processor can be optimized for their respective jobs.
The same thinking extends beyond the chip. Each rack brings together 96 accelerators, with custom Ethernet-based networking designed to help them communicate efficiently. Liquid cooling removes much of the heat generated by the accelerators and networking hardware.
This is what hardware-software co-design means in practice: the chip, memory, networking, software and cooling are developed around the demands of the workload they need to run. Microsoft’s aim is to reduce wasted movement and waiting, allowing more of the available power to go toward useful AI computation.
Also Read: Are AI Chats Private? What the Claude Diary Case Reveals
Does This Mean Leaving Nvidia?
Not necessarily.
Microsoft is building its own accelerators to handle more of the AI workloads running across its infrastructure.
But building custom chips doesn’t mean Nvidia GPUs suddenly become unnecessary. Different workloads have different requirements, and Microsoft still operates a broader Azure infrastructure for customers with their own computing needs.
The shift is about having another option, especially for the enormous volume of AI inference Microsoft expects to run across its own products and services.
If Maia can deliver the efficiency Microsoft is targeting, the company could serve more AI requests within the same power budget while relying less on external accelerator supply for those workloads.
That doesn’t make Nvidia irrelevant but it gives Microsoft more control over one of the most expensive parts of running AI
Also Read: AI Is Getting Cheaper, But AI Bills Could Still Go Up
The Test Starts in 2027
Microsoft can design Maia around its own workloads, optimize the entire rack and project better efficiency. But none of that tells us yet how the system will perform across a full-scale rollout.
The first production deployments are planned for 2027. That’s when Microsoft’s efficiency claims will meet the realities of running AI across data centers at enormous scale.
The outcome will matter beyond Microsoft. As AI becomes part of more products and workflows, the cost of running it will help determine how much capacity companies can offer and what that capacity costs.
The race to build AI isn’t only about making models more capable. It’s also about building the infrastructure that makes using them at scale practical.
