Who is winning AI inference outside Nvidia?
In our data center market deck, you will find everything you need to understand the market
SUMMARY
Broadcom is winning AI inference outside Nvidia today, because it captures the custom-chip buildout across several of the biggest AI buyers rather than betting on a single accelerator platform.
The non-Nvidia inference market is already large enough to stand on its own. AWS has landed more than 1.4 million Trainium2 chips, Meta has deployed hundreds of thousands of MTIAs, Microsoft is serving OpenAI and MAI models on Maia 200, and Google is already on its eighth TPU generation.
The market is fragmenting by layer rather than converging on one Nvidia replacement. Broadcom leads custom-chip economics, AWS leads proprietary inference sold broadly to outside customers, Google leads in hyperscaler custom-silicon maturity, AMD leads the merchant-GPU challenge, and Cerebras leads the independent specialist category.
Inference is a more vulnerable part of Nvidia's stack than training because repetitive production workloads reward specialization. Once the same model is serving enormous traffic, even a modest reduction in cost, power or latency can justify the engineering effort required to move away from a general-purpose GPU.
The biggest competitive threat to Nvidia is therefore not one rival chip company. It is the fact that Nvidia's largest customers are increasingly becoming chip designers themselves, often with Broadcom helping underneath.
AWS has done the best job turning proprietary silicon into a normal cloud product. Bedrock, SageMaker and EC2 let Amazon hide Trainium under managed services, which makes the chip easier to adopt because customers can buy inference rather than rewrite their stack around a new accelerator.
Google still has the deeper custom-AI-chip history, but AWS has the stronger commercialization story. TPU has roughly a decade of maturity behind it, while Trainium has become a major third-party platform with huge customer commitments and a direct route into managed inference.
AMD matters because it gives large buyers a general-purpose GPU alternative without requiring them to design an ASIC. Its main weakness remains software, but that weakness is less damaging for stable inference workloads where companies can justify optimizing around the same model family for years.
Cerebras is the most differentiated independent challenger because it is competing on latency rather than trying to imitate Nvidia. The open question is execution: its backlog and performance are already striking, but it still has to convert very large commitments into deployed capacity and sustained multi-billion-dollar revenue.
CUDA remains a huge moat when developers interact directly with accelerators, but managed inference services make the underlying chip less visible. If disaggregated inference takes off and different hardware handles prompt processing and token generation, the inference market could fragment even further rather than produce one clear Nvidia successor.
This market map, featured in our data center market deck, highlights top companies and startups in the data center market
Is there finally a real AI inference market outside Nvidia?
Yes. AI inference outside Nvidia is already a real, large market, and the clearest proof today is that Google, Amazon, Meta, Microsoft and OpenAI are putting proprietary accelerators into production at a scale that would have sounded unrealistic a few years ago.
The change is easiest to see inside the hyperscalers. Amazon has landed more than 1.4 million Trainium2 chips and says most inference on Amazon Bedrock runs on Trainium. Meta says it has deployed hundreds of thousands of MTIA accelerators. Microsoft now runs OpenAI and its own MAI models on Maia 200. Google has reached its eighth TPU generation, with TPU 8i designed specifically around inference and reinforcement learning.
Broadcom gives us another way to measure the shift because it supplies custom accelerator technology to several of these companies. In its latest reported quarter, Broadcom generated $16.7 billion from AI semiconductors, up 221% year over year. Management said XPUs represented 73% of that revenue, which implies roughly $12.2 billion from custom accelerators in a single quarter. XPU shipments were also more than 3.5 times higher than a year earlier.
That is already too much volume to describe non-Nvidia inference as an experiment. Nvidia remains the biggest AI-compute supplier by a wide margin, but a second market has formed around custom chips and alternative GPUs.
| Evidence of scale | What we see now |
|---|---|
| AWS | 1.4M Trainium2 chips landed; most Bedrock inference runs on Trainium |
| Meta | Hundreds of thousands of MTIA chips deployed |
| Microsoft | Maia 200 serving OpenAI and MAI models |
| Broadcom | $16.7B quarterly AI semiconductor revenue; 73% from XPUs |
| Eighth TPU generation now split between training and inference designs |
What does “winning AI inference” actually mean?
For AI inference, we think “winning” should mean running a lot of real workloads at attractive economics, then convincing customers or internal teams to keep expanding those deployments.
The companies outside Nvidia are playing very different games. Broadcom can make billions from inference without operating a Broadcom cloud. Amazon wins when customers use Trainium through AWS. Google can save billions by serving its own workloads on TPU even when no external customer sees the chip. AMD wants companies to buy a broadly usable alternative GPU. Cerebras is selling extreme inference speed.
A benchmark by itself tells us very little about who is winning. A chip generating 1,000 tokens per second for a handful of demonstrations matters less than a somewhat slower accelerator quietly serving trillions of production tokens every week.
We therefore give the most weight to actual deployment, revenue or committed spending, repeat adoption and cost advantages large enough to change buying behavior.
| What we measure | Current leader outside Nvidia |
|---|---|
| Custom-chip revenue | Broadcom |
| Hyperscaler inference sold to customers | AWS Trainium |
| Mature custom-silicon deployment | Google TPU |
| Merchant GPU alternative | AMD Instinct |
| Internal recommendation inference | Meta MTIA |
| Ultra-fast frontier-model inference | Cerebras |
If you want more recent data on this point, please see our latest data center market report.
As this chart shows, and as featured in our data center market deck, search interest in data centers has increased significantly
Is Nvidia actually losing its grip on AI inference?
Yes, slowly. Nvidia still dominates general AI acceleration, but inference is now the part of the stack where the largest customers have the strongest reason to reduce their dependence on Nvidia.
The economics favor specialization. Training workloads keep changing as researchers experiment with architectures, numerical formats and parallelization strategies. That flexibility makes Nvidia's broad CUDA ecosystem extremely valuable. Once a model reaches production, the operator may generate the same type of inference billions or trillions of times. At that point, shaving even 20% from infrastructure cost can justify a lot of engineering.
Amazon is already explicit about this. Andy Jassy says Trainium2 offers about 30% better price-performance than comparable GPUs, while Trainium3 improves another 30% to 40% over Trainium2. Amazon expects its own silicon to save tens of billions of dollars in annual capital expenditure at scale and add several hundred basis points to AWS operating margins compared with relying on outside chips for inference.
Google, Meta and Microsoft are chasing the same prize in different ways.
Nvidia can therefore keep growing very quickly while losing some share of inference. Those two things can happen together. What has changed is that the largest AI buyers now have credible alternatives for workloads they understand well.
Is Broadcom the hidden winner of AI inference?
Yes. Broadcom is currently the strongest overall winner outside Nvidia because it gets paid when several of Nvidia's largest customers build their own accelerators.
The latest numbers are unusually strong. Broadcom's AI semiconductor revenue rose from $8.4 billion in its first fiscal quarter to $10.8 billion in the second and $16.7 billion in the third. Management expects $21.7 billion in the fourth quarter, which would bring annual AI semiconductor revenue to roughly $58 billion.
Custom accelerators are driving most of that business. Broadcom said XPUs represented 73% of AI revenue in the latest quarter, alongside XPU shipment growth of more than 3.5 times year over year. That works out to about $12.2 billion of quarterly XPU revenue.
The customer mix strengthens the case. Broadcom is producing Google TPUs, working with Meta across multiple generations of MTIA, supplying Anthropic-related TPU capacity and shipping OpenAI's first custom accelerator, Jalapeño. Broadcom says production shipments of Meta's inference-focused MTIA are also beginning.
OpenAI adds a much bigger future leg. Broadcom and OpenAI previously announced plans for 10 gigawatts of OpenAI-designed accelerators through the end of the decade. Jalapeño has now moved into shipments rather than remaining a roadmap announcement.
Broadcom is winning because it does not have to persuade Google, Meta or OpenAI to use a standardized Broadcom chip. It helps each customer build the chip that makes sense for its own models.
| Broadcom program | Current position |
|---|---|
| Shipping Ironwood and next-generation TPU 8i | |
| Meta | Multi-generation MTIA partnership; production shipments starting |
| OpenAI | Jalapeño shipping; 10GW long-term program |
| Anthropic | Large TPU infrastructure deployments |
| Latest XPU scale | About $12.2B implied quarterly revenue |
If you want more recent data on this point, please see our latest data center market report.
This chart, featured in our data center market deck, illustrates yearly venture capital funding for data center startups
Is AWS Trainium the strongest Nvidia alternative customers can actually use?
Right now, yes. AWS Trainium has the strongest case among proprietary accelerators that outside customers can consume at very large scale without designing their own hardware.
Amazon has already landed 1.4 million Trainium2 chips, and Trainium2 has largely sold out. Trainium3 began production workloads this year and is nearly fully subscribed. Even Trainium4 capacity, still well ahead of broad availability, has already been reserved by customers.
Inference usage is the more important number for this article. Amazon says most inference on Bedrock runs on Trainium. Bedrock now has hundreds of thousands of customers, and Amazon says customer spending in its latest quarter exceeded spending in all previous Bedrock quarters combined.
The commitments have become enormous too. Amazon has disclosed more than $225 billion of Trainium revenue commitments. Anthropic alone has committed to spend more than $100 billion over ten years across AWS technologies and can secure as much as five gigawatts of capacity. OpenAI has separately committed to roughly two gigawatts of Trainium capacity beginning to ramp next year.
Those figures include training as well as inference, so we should not pretend that $225 billion represents an inference backlog. They do show that Trainium has crossed the hardest hurdle for a proprietary cloud chip: major external customers are willing to commit tens of billions of dollars to it.
The original separation between Trainium for training and Inferentia for inference has also become less meaningful. Trainium is increasingly AWS's general AI accelerator, and Bedrock gives Amazon a natural way to put it underneath workloads without asking users to think about the chip.
Is Google TPU still ahead of AWS Trainium?
Google TPU is still ahead in maturity, while AWS Trainium now looks stronger as a commercial cloud platform.
Google has been building TPUs for roughly a decade and has reached its eighth generation. The latest roadmap makes the strategy explicit: Google split the family into TPU 8t for training and TPU 8i for inference and reinforcement learning. TPU 8i triples on-chip SRAM versus the previous generation, raises HBM capacity to 288GB and is designed around the memory-heavy behavior of large mixture-of-experts models.
Google says TPU 8i improves inference performance per dollar by as much as 80% over Ironwood at low-latency targets. The seventh-generation Ironwood is already generally available, while TPU 8i is moving into deployments.
External demand is also much stronger than the old perception of TPU as a chip built mainly for Google. Anthropic has become a major TPU customer, and Broadcom says it delivered Ironwood in high volume to both Google and Anthropic during its latest quarter.
Amazon, however, has done a better job of turning proprietary silicon into something that feels like a normal cloud product. Trainium sits underneath Bedrock, SageMaker and EC2, and AWS can spread the hardware across a much broader customer base.
If we had to choose today, Google gets the edge for custom-AI-chip maturity. AWS gets the edge for proving that proprietary accelerators can become a huge third-party cloud business.
This chart, featured in our data center market deck, shows how Equinix is capturing share in data centers
Can AMD become the default non-Nvidia AI GPU?
AMD has become the only serious candidate. For companies that want a general-purpose GPU without building their own ASIC, AMD Instinct is now a credible alternative at hyperscale.
The recent commitments are much larger than the MI300-era wins. Meta agreed to deploy up to six gigawatts of AMD Instinct infrastructure across multiple generations. The first gigawatt is expected to use a custom MI450-based accelerator. Anthropic has separately committed to as much as two gigawatts of MI450-series systems, with its first gigawatt expected to begin deployment next year.
Microsoft is bringing AMD Helios racks into Azure for frontier inference. OpenAI, Oracle and several specialist cloud providers are also working with AMD's newer platforms.
AMD's latest data-center revenue reached $6.7 billion, up 107% year over year. That number includes EPYC CPUs, so we cannot treat it as Instinct GPU revenue, but the direction is clear: AMD's AI accelerator business has moved beyond isolated second-source orders.
ROCm remains the biggest constraint. Nvidia's CUDA stack still gives developers broader libraries, tooling and accumulated engineering knowledge. AMD has been closing that gap through better PyTorch support, vLLM, optimized kernels and the new ROCm.ai developer environment.
For stable inference workloads, the software disadvantage hurts less than it does in fast-moving research. If a company expects to generate trillions of tokens from the same family of models, spending engineering time to optimize AMD becomes easier to justify.
AMD does not need every engineer to prefer ROCm over CUDA. It needs the world's largest compute buyers to believe they can move the next gigawatt of infrastructure if Nvidia's economics become unattractive. Meta, Anthropic and Microsoft now provide real evidence that they can.
If you want more recent data on this point, please see our latest data center market report.
Is Meta MTIA already replacing Nvidia inside Meta?
Yes, for a large slice of Meta's internal inference workloads. MTIA has already reached production scale in recommendations and advertising, although generative-AI inference is the next, harder test.
Meta says hundreds of thousands of MTIA accelerators are already deployed. These chips support ranking, recommendations and advertising workloads that run continuously across Facebook and Instagram, making them ideal candidates for custom silicon because Meta understands the models and traffic patterns exceptionally well.
Meta is accelerating the roadmap rather than slowing it. The company plans four new MTIA generations over roughly two years and recently expanded its Broadcom partnership into a multi-gigawatt program. The first phase alone exceeds one gigawatt.
Generative AI is becoming a bigger part of that roadmap. Upcoming MTIA designs are being built with much more memory and bandwidth so they can handle workloads closer to large-model inference rather than mainly recommendation models.
Meta is also buying up to six gigawatts of AMD infrastructure and remains a huge Nvidia customer. The mix shows the economics pretty clearly: custom silicon does not have to replace every GPU to matter. Moving repetitive internal workloads onto MTIA already reduces the number of expensive merchant accelerators Meta would otherwise need.
This chart, featured in our data center market deck, illustrates yearly funding for data center startups
Has Microsoft Maia become a serious Nvidia alternative?
Yes. Maia 200 has crossed from an interesting Microsoft chip project into production inference, although its deployment history remains much shorter than Google's TPU program.
Microsoft designed Maia 200 around token generation. It has 216GB of HBM3e, 7TB/s of memory bandwidth and native FP4 and FP8 compute. Microsoft says the system delivers about 30% better performance per dollar than the latest-generation hardware previously running in its fleet.
More importantly, Microsoft disclosed in its latest earnings call that Maia 200 is now supporting both OpenAI and Microsoft's MAI models. When Microsoft's own models run on Maia 200, the company says it sees roughly 40% better performance per watt.
That is where an internal accelerator becomes valuable. Microsoft can co-design models, inference software, networking and silicon, then put repetitive Copilot and Azure workloads onto the cheapest suitable hardware.
Maia still has less proof of scale than TPU or Trainium. Microsoft has not disclosed anything comparable with Amazon's millions of landed AI chips or Meta's hundreds of thousands of MTIAs.
We would therefore place Maia behind Google and Amazon today, but it has clearly entered the serious group.
Is Cerebras the strongest independent AI inference challenger?
Yes. Cerebras is currently the most convincing independent inference specialist because its extreme speed advantage is finally being matched by meaningful revenue, major customers and a very large contracted backlog.
The easiest proof is visible in the product. OpenAI uses Cerebras for GPT-5.6 Sol at roughly 750 output tokens per second. Cerebras also co-launched Codex-Spark at more than 1,000 tokens per second for latency-sensitive coding. Those speeds change how interactive reasoning, coding agents and other sequential workloads feel to the user.
The commercial numbers have caught up surprisingly quickly. Cerebras reported $127.7 million of core cloud and services revenue in its latest quarter, up 287% year over year. Total core revenue reached about $210 million, more than double the previous year.
Its remaining performance obligations reached $25.4 billion. The largest piece comes from OpenAI, which committed to buy 750 megawatts of Cerebras inference capacity under a deal valued at more than $20 billion. OpenAI also has an option for another 1.25 gigawatts.
That backlog should not be confused with delivered revenue. Cerebras is still producing hundreds of millions of dollars per quarter while its commitments stretch over several years. Execution is now the bigger question: the company has more than 600 megawatts live or contracted for delivery and plans to increase manufacturing capacity more than tenfold.
Customers beyond OpenAI are also appearing in workloads where latency really matters, including Cognition, Lovable, CrowdStrike, Figma, Block, AlphaSense and GSK.
Groq deserves mention here because it helped prove that specialized inference hardware could make LLMs dramatically faster. Its position has since become harder to classify as “outside Nvidia” after Nvidia licensed Groq technology and hired key technical personnel. Cerebras now has the cleaner claim to being the leading independent inference architecture.
If you want more recent data on this point, please see our latest data center market report.
This chart, featured in our data center market deck, compares the main business model options for hyperscale data center operators
Are custom AI chips already beating GPUs for inference?
For the biggest repeat workloads, increasingly yes. Across the full AI market, GPUs still win because they can handle far more types of work.
The dividing line is scale. A startup running a few million model calls has little reason to design a semiconductor. Google Search, Meta's ranking systems, Amazon Bedrock and Microsoft's Copilot products can generate enough repeated inference to spread chip-design costs across enormous usage.
Custom accelerators can then remove flexibility the workload does not need and spend silicon area, memory bandwidth and power on the parts it uses constantly.
The latest roadmaps make that specialization much more explicit. Google now has TPU 8i specifically for inference. OpenAI's Jalapeño is designed around LLM inference. Meta is expanding MTIA toward generative AI. Microsoft's Maia 200 was built around token generation.
Broadcom's revenue shows how quickly this is becoming an industry rather than a handful of internal projects. Its XPU shipments grew more than 3.5 times year over year in the latest quarter. As seen above, custom accelerators accounted for 73% of Broadcom's $16.7 billion AI semiconductor revenue.
General-purpose GPUs will remain essential wherever model architectures change quickly or customers need maximum flexibility. The giant repetitive workloads are where custom silicon has become hardest for Nvidia to defend.
Does CUDA still protect Nvidia from these inference chips?
Yes. CUDA remains Nvidia's biggest defense, but inference services are gradually making the underlying hardware less visible to customers.
A company choosing servers directly gets an enormous amount with Nvidia: CUDA libraries, debugging tools, communication software, optimized kernels, model-framework support and engineers who already know how to use all of it.
Every serious alternative has therefore found a different way around the software problem.
AMD is making ROCm easier for CUDA-oriented developers to adopt. Google supports PyTorch, JAX and Keras on TPU, with XLA handling much of the hardware-specific translation. Amazon can hide Trainium underneath Bedrock and managed AWS services. Microsoft controls both Maia and many of the models or products running on it. Meta controls its entire internal stack. Cerebras exposes familiar APIs so customers do not need to understand wafer-scale hardware.
The last three approaches are especially important for inference. Someone calling a model API usually cares about price, latency, reliability and output quality. The exact processor underneath can disappear from view.
That weakens Nvidia's software advantage at the point where inference becomes a managed service. CUDA remains a huge moat for developers building directly on accelerators, but it matters much less to a Bedrock customer whose tokens happen to come from Trainium.
This chart, featured in our data center market deck, shows the revenue mix across customer segments in the data center market
Could disaggregated inference change who wins?
Yes. Splitting prompt processing from token generation could give specialized chips a much bigger role because those two stages want different kinds of hardware.
Large-model inference has a compute-heavy prefill stage, when the system processes the prompt, followed by a decode stage that generates tokens sequentially and is much more sensitive to memory movement and latency.
Cerebras and AMD are already building around that difference. Their planned disaggregated system uses AMD infrastructure for high-throughput work and Cerebras for ultra-fast generation. Cerebras says the architecture can raise throughput by as much as five times while preserving its low-latency decode.
AWS is pursuing a similar setup. Amazon and Cerebras plan to bring disaggregated inference to Bedrock, combining AWS infrastructure with Cerebras's fast generation. The first production deployment is expected next year.
If those systems work economically at scale, comparing one accelerator with another becomes less useful. Operators could use a Trainium or AMD system where throughput matters most and Cerebras where milliseconds between generated tokens matter most.
That would favor a more fragmented inference market. Different chips can win different parts of the same request.
So who is winning AI inference outside Nvidia right now?
Broadcom is winning overall today. AWS Trainium is the strongest proprietary inference platform sold broadly to outside customers, Google TPU remains the most mature hyperscaler custom-chip program, AMD leads the open merchant-GPU challenge, and Cerebras is the strongest independent specialist.
Broadcom gets our number-one position because the revenue and customer breadth are now on another level. Its latest quarter produced $16.7 billion of AI semiconductor revenue, roughly $12.2 billion of which can be attributed to XPUs using the company's disclosed 73% mix. Those accelerators span Google, Meta, OpenAI, Anthropic-related infrastructure and other major AI customers. Broadcom expects quarterly AI semiconductor revenue to rise again to $21.7 billion.
AWS has built the strongest customer-facing alternative. Most Bedrock inference already runs on Trainium, 1.4 million Trainium2 chips have landed, Trainium3 is nearly fully subscribed and Amazon has more than $225 billion of Trainium commitments.
Google deserves almost equal weight from a technology perspective. It has spent a decade refining TPU, is already shipping seventh-generation Ironwood at scale and is moving toward a dedicated TPU 8i architecture with an advertised 80% inference price-performance improvement over the previous generation.
AMD has finally become a credible alternative for companies that want GPUs rather than a proprietary hyperscaler chip. Agreements covering up to six gigawatts at Meta and two gigawatts at Anthropic are far more meaningful than benchmark wins.
Cerebras is much smaller financially, but its position has changed fast. Core cloud revenue almost quadrupled year over year, OpenAI has committed to 750 megawatts under a deal worth more than $20 billion, and remaining performance obligations have reached $25.4 billion. We still need to see Cerebras turn that backlog into thousands of deployed systems and billions of annual revenue, but today it is the independent inference company with the clearest combination of differentiated performance and real demand.
The bigger change across all of these companies is the rise of custom silicon. Nvidia still owns the broadest and most useful general AI-compute platform. Yet inference is fragmenting much faster than training because Google, Amazon, Meta, Microsoft and OpenAI now run enough repetitive AI workloads to justify chips designed around their own economics.
For now, the strongest attack on Nvidia is coming from that collection of TPUs, Trainiums, MTIAs, Maias, AMD GPUs and specialized inference systems. Broadcom sits underneath more of that shift than anyone else, which is why we would call Broadcom the clearest winner outside Nvidia today.
If you want more recent data on this point, please see our latest data center market report.
This chart, featured in our data center market deck, shows how hyperscale AI-ready campus technology has evolved over time
OUR METHODOLOGY
This analysis asks who is winning AI inference outside Nvidia by comparing several very different routes to non-Nvidia inference: hyperscaler custom silicon, merchant GPUs, custom-chip design programs and specialized inference architectures.
We did not use a single benchmark or market-share estimate as the deciding metric. We weighted production deployment, real inference usage, realized revenue, external customer adoption, committed capacity, cost and price-performance, software accessibility, and evidence that deployments are continuing to expand.
Production evidence carried more weight than theoretical performance. Chips already serving meaningful workloads, generating revenue or being expanded by customers were treated as stronger evidence than peak specifications, isolated demonstrations or roadmap announcements.
We also separated delivered business from future business. Revenue already recognized, hardware already deployed and inference already running were treated differently from reservations, remaining performance obligations, signed capacity commitments and multi-year infrastructure programs.
Where a company disclosed AI accelerator revenue, capacity or commitments that covered both training and inference, we used those figures as evidence of overall platform scale rather than treating them as pure inference numbers. Internal hyperscaler deployment also counted as real adoption, but third-party customer commitments received additional weight because they provide a different form of validation.
Company-reported price-performance claims were treated as evidence of why a platform is being adopted, not as independent proof that one architecture is universally superior. We also avoided forcing Broadcom, AWS, Google, AMD and Cerebras into a false apples-to-apples comparison because they capture value at different layers of the inference stack.
Fresh primary evidence received the most weight. The core source set included Broadcom's Q3 FY2026 financial results, Amazon's discussion of Trainium and AWS custom silicon, Google Cloud's TPU 8i and AI-infrastructure roadmap, Meta's update on MTIA and custom silicon, and Microsoft's FY2026 Q4 earnings call covering Maia 200 deployment.
We also used AMD's Meta infrastructure agreement, AMD's Anthropic agreement, and Cerebras's Q2 2026 results on cloud revenue, capacity and remaining performance obligations. The final ranking is a structured editorial judgment based on the combined evidence rather than a mechanical score.
In our data center market deck, we identify pain points entrepreneurs should prioritize
Related blog posts
