Microsoft has announced Azure Maia 300, the third generation of its custom AI accelerator, designed to process AI workloads more efficiently. The company plans to begin scaling its Maia infrastructure to multi-gigawatt deployments in 2027, with the system expected to become a primary inference platform for Microsoft products and services.
Improved Energy Efficiency and Performance
According to Microsoft, Maia 300 is expected to deliver 40% more tokens per megawatt than leading GPU-based systems. Tokens are units used by AI models to process input and generate responses.
Microsoft also says a full rack configuration containing 96 accelerators can deliver up to seven times the rack-scale token throughput of its predecessor, Maia 200. This figure refers to the performance of the complete rack system rather than an individual chip.
The accelerator is designed for demanding AI workloads, including long-context reasoning, latency-sensitive conversations and multi-step tasks performed by AI agents.
How Maia 300 Works
Microsoft is developing Maia 300 as an integrated system that combines compute, memory, networking and software rather than treating the chip as an isolated component.
Each accelerator features eight HBM4 memory stacks, providing 288 GB of memory capacity and 18 TB/s of peak memory bandwidth, according to Microsoft. These specifications are intended to support large AI models and memory-intensive inference workloads.
The design uses a multi-chiplet architecture that separates compute, memory-controller and input/output functions. Its compute chiplets are built using TSMC’s N2P process technology, allowing different parts of the system to be optimized for their specific roles.
At rack scale, 96 accelerators operate within a power envelope of approximately 167 kilowatts. This dense configuration is designed to improve the amount of AI computing capacity that can be deployed within the power and cooling limits of a data centre.
Expanding Microsoft’s AI Infrastructure
Maia 300 builds on Microsoft’s existing Maia programme. The company says Maia 200 is already running production workloads, including Microsoft Copilot, with further applications across its AI services.
Developing its own accelerators gives Microsoft greater control over how its infrastructure is optimized for its models and products. It also aims to improve efficiency as demand for AI-powered services continues to grow.
However, Microsoft has not announced that Maia 300 will completely replace GPUs from suppliers such as Nvidia across all workloads. The announcement focuses on expanding the use of its custom chips for AI inference — the process through which trained models generate responses to user requests.
Microsoft plans to begin production deployments of Maia 300 in 2027. The broader goal is to expand AI computing capacity while improving energy efficiency and supporting a growing range of AI applications.
