Microsoft's Azure Maia 300 targets cloud AI inference
Unveiled at the OCP Global Summit in San Jose, the multi-chiplet Maia 300 uses 288 GB of HBM4 memory and doubles rack density to 96 accelerators, as Microsoft works to become less reliant on Nvidia.
Microsoft has announced its third-generation Maia AI chip, the Azure Maia 300, unveiled on October 12 at the OCP Global Summit in San Jose. The company says the chip is designed to become the primary AI inference system for its own products and services, from Microsoft 365 Copilot and GitHub Copilot to the models on Microsoft Foundry, with production beginning in 2027 and multi-gigawatt deployments of the Maia family to follow.
Built for inference, not raw speed
Rather than chasing peak FLOPs with more power and silicon area, Maia 300 is built to move as much useful data as possible within a fixed energy budget. It is a multi-chiplet part on TSMC's N2P process that separates the compute cores from the memory-controller and I/O dies. Each accelerator carries eight HBM4 stacks providing 288 GB of memory and 18 TB/s of bandwidth, about 2.5 times the bandwidth of Maia 200 with 33 percent more capacity. Across the rack and a new 12,288-accelerator superpod, Microsoft says the design boosts compute FLOPs by 36 percent and roughly doubles HBM bandwidth per watt, while new narrow MXFP4 and sub-3-bit formats cut the bytes moved per token.
Why it matters
Microsoft projects up to 40 percent more tokens per megawatt than leading GPU-based systems, and up to seven times the rack-scale token throughput of Maia 200 in a full 96-accelerator configuration, double the prior generation's density. Those gains matter most for the long-context, reasoning-heavy and agentic workloads that dominate modern AI, and the company framed the design as a direct answer to Nvidia's dominance: in-house Maia silicon reduces Microsoft's reliance on third-party GPUs, which Evercore ISI estimates will make up about 10 percent of Microsoft's AI chip spending by the first quarter of 2027.
Inference efficiency is quietly becoming the real currency of AI, where power is the tightest constraint on growth. As hyperscalers like Microsoft ship bespoke inference silicon, the cost of running Copilot and other AI services should keep falling, giving you more AI capability for less. For cloud buyers and developers, it also means a more competitive, less single-vendor compute market by 2027.
The Maia 300 follows the Maia 200 and 100 generations already deployed across the US, Europe and Australia. Microsoft says performance figures are company estimates, and the final production schedule was not disclosed beyond a 2027 start, as executives note the pace of change in the chip market is still fast.
Sources: Microsoft Azure Blog - Deep dive into Azure Maia 300 architecture; The Verge - Microsoft's new AI chip makes it less reliant on Nvidia's GPUs; CNBC - Microsoft will mainly use its Maia chips for AI inference.
Corrections: Report an error at corrections@wireshaman.com.