Which business models can win in AI inference chips?

Why is AI inference becoming the bigger chip business right now?

AI inference is becoming the recurring money engine of AI infrastructure because every model interaction consumes compute again and again.

Training creates huge hardware orders, but they arrive in waves. Inference follows the model into production: every ChatGPT query, Copilot session, recommendation, generated video and autonomous-agent action requires more computation. Once a product reaches hundreds of millions of users, that repeated workload can become larger economically than the original training run.

The latest infrastructure choices show where the industry expects the spending to go. Microsoft built Maia 200 specifically for inference and says it delivers 30% better performance per dollar than the latest-generation hardware previously used in its fleet. Google is pushing its TPU roadmap deeper into inference, while AWS has spent multiple chip generations reducing inference costs with Inferentia and Trainium.

Meta offers an even clearer example. The company says it has already deployed hundreds of thousands of its internally designed MTIA chips. The first generations concentrated on recommendation workloads, but MTIA 300, 400, 450 and 500 are broadening into training, generative AI and GenAI inference.

Specialists are following the same demand. Cerebras's core cloud and services revenue reached $127.7 million in its latest reported quarter, up 287% from a year earlier. Groq now says more than six million developers use its platform across 13 data centers.

AI inference has therefore become much more than a second market after training. It is where AI infrastructure starts generating repeated usage revenue.

What does it actually mean to win in AI inference chips?

Winning in AI inference chips means capturing a large and durable share of the money spent running AI models, whether that revenue comes from chips, custom silicon, complete systems or compute services.

This distinction matters because Nvidia, Broadcom, Google and Cerebras can all build strong inference businesses while selling completely different things.

Nvidia sells broadly programmable hardware wrapped in networking and software. Broadcom helps hyperscalers build custom accelerators. Google designs TPUs and then rents the resulting computing capacity through Google Cloud. Cerebras increasingly sells access to infrastructure built around its own chips.

Their margins, capital needs and customer risks look very different.

Business model	What the customer pays for	Main advantage	Main risk
Merchant accelerators	GPUs or accelerator systems	Huge addressable market	Price and platform competition
Full compute platform	Chips, networking, systems and software	High switching costs	Enormous R&D requirements
Custom ASICs	Customer-specific AI silicon	Large committed programs	Customer concentration
Hyperscaler-owned chips	Cloud services or internal compute	Lower internal cost	Only works at enormous scale
Inference cloud	Tokens or dedicated capacity	Recurring usage revenue	Capital and utilization risk
Dedicated inference systems	Complete racks and appliances	Better workload-specific economics	Narrower workload coverage
IP licensing	Accelerator or CPU architecture	Much lower capital needs	Captures less end value
Edge inference	Embedded AI chips	Power and latency advantages	Fragmented market

The biggest mistake would be to rank these models using chip benchmarks alone. A technically slower architecture can still produce the better business if customers find it easier to deploy, if utilization is higher or if the supplier captures more of the infrastructure stack.

Can Nvidia keep winning AI inference with expensive GPUs?

Nvidia can keep winning a huge part of AI inference because customers currently pay for the whole Nvidia platform, not just the GPU's raw speed.

The latest numbers leave little ambiguity. Nvidia generated $89 billion of Data Center revenue in its latest reported quarter, up 117% year over year. Data Center accounted for more than 92% of total revenue, while companywide GAAP gross margin reached 75%. A 75% gross margin sets a high bar; we show where the margins really are in our report on AI chips.

Those economics give Nvidia room to invest in CUDA, networking, libraries, interconnects, CPUs, rack design and developer tooling at a scale that smaller inference-chip companies cannot easily match.

That software advantage changes the purchase decision. Suppose a specialist delivers 30% cheaper inference on one model. The customer still has to migrate workloads, validate models, retrain engineers, rebuild tooling and accept the risk that the next model generation behaves differently.

Nvidia has also kept expanding the amount of each AI deployment it can sell. Current rack-scale platforms combine GPUs, CPUs, NVLink, networking, switches and software. The company gets paid across far more of the cluster than a conventional chip vendor.

For challengers, beating one Nvidia GPU on tokens per second is the easy part. The commercial hurdle is making the savings large enough that customers are willing to change the rest of their infrastructure.

Does AMD need to beat Nvidia for its AI inference business to work?

AMD can build a very large AI inference business without beating Nvidia because hyperscalers have strong reasons to support a credible second GPU platform.

AMD's latest Data Center revenue reached $6.7 billion, more than double the level a year earlier. The segment produced about $2.1 billion of operating income, although those numbers include EPYC CPUs alongside Instinct accelerators.

The opportunity comes partly from customer bargaining power. Microsoft, Meta, Oracle and large AI clouds do not want one supplier controlling virtually every important accelerator purchase. A usable AMD alternative helps them diversify supply and negotiate harder.

AMD is also moving beyond individual accelerators. Its Helios rack-scale architecture combines Instinct GPUs, EPYC CPUs and networking in a full system, bringing its commercial approach closer to Nvidia's.

Software remains the hard part. ROCm has improved substantially, but a second hardware supplier only becomes a genuine second platform when large workloads can move onto it without creating a painful engineering bill.

AMD therefore has a straightforward route to success: become good enough that every large AI buyer wants AMD in its procurement plan. In a compute market this large, second place can still be worth tens of billions of dollars.

Are custom AI chips becoming the strongest alternative to Nvidia?

Custom AI chips currently look like the strongest alternative business model because the hyperscalers can trade some flexibility for much better economics on workloads they already run at enormous scale.

Broadcom gives us the clearest proof. Its latest quarterly AI semiconductor revenue reached $16.7 billion, up 221% year over year and 54% in one quarter. Broadcom expects the figure to rise again to $21.7 billion in the following quarter.

That trajectory is more useful than one large contract announcement. Broadcom reported $4.4 billion of AI semiconductor revenue in the comparable quarter a year earlier, then $10.8 billion two quarters ago, $16.7 billion most recently and now expects $21.7 billion. Custom AI silicon has moved from a promising hyperscaler project into one of the largest semiconductor businesses in the world.

The model suits inference particularly well. Once a company repeatedly runs the same recommendation systems, internal AI services or model families, it can optimize memory, precision, networking and data movement around those workloads.

Meta now says hundreds of thousands of MTIA chips have already gone into production, and its roadmap is moving from recommendation inference into broader generative-AI workloads. That makes the threat much more concrete than the earlier wave of prototype hyperscaler chips.

Custom silicon has one obvious limit: very few customers generate enough demand to justify a leading-edge design program. That is why Broadcom's model works so well. It can spread semiconductor expertise across several giant customers without needing to own the cloud itself.

Why are Google, Amazon, Microsoft and Meta building their own inference chips?

Google, Amazon, Microsoft and Meta build AI inference chips because shaving even 20% or 30% from compute costs becomes enormous when the same workload runs billions of times.

Microsoft's Maia 200 is a good current example. Microsoft says the chip delivers 30% better performance per dollar than the latest-generation hardware previously in its fleet, and it is already supporting OpenAI and Microsoft's own models.

Google takes the model one step further by exposing its proprietary TPUs directly to outside customers. An Ironwood TPU currently costs $12 per chip-hour on demand in one U.S. region. A three-year commitment brings that down to $5.40, which is 55% lower. Prices that drop 55% with a three-year commitment raise the question of who pays, and how much, which we study in our report covering AI chips.

Meta has a different incentive. Recommendation and advertising models sit inside products used by billions of people, so infrastructure savings flow back into the economics of Facebook, Instagram and Meta's ad business. Its latest MTIA update shows four newer chip generations being deployed or prepared as workload coverage expands.

These companies do not need to earn a traditional semiconductor margin on every chip. Saving money inside Azure, Google Cloud, AWS or Meta's own services can provide the return.

That makes vertically integrated silicon one of the hardest models for independent chip companies to compete with. The hyperscaler controls the workload, software, data center and distribution at the same time.

Can Nvidia keep growing while custom AI chips take market share?

Nvidia can keep growing quickly even while custom AI chips gain share because overall AI compute spending is still expanding much faster than the substitution between architectures.

Look at the latest reported growth rates. Nvidia Data Center revenue rose 117% year over year. AMD Data Center revenue rose 107%. Broadcom's AI semiconductor revenue rose 221%.

Those three businesses grew rapidly at the same time.

The more interesting pressure comes later. Every recommendation model moved permanently onto MTIA, every internal workload shifted to a TPU and every hyperscaler inference fleet built around a Broadcom-designed accelerator removes future demand that might otherwise have gone to a merchant GPU.

Inference is particularly exposed because mature workloads become easier to optimize. A hyperscaler may still want flexible Nvidia systems for new models while moving repetitive production traffic onto cheaper internal silicon.

For now, the market is large enough for both models to grow. Over time, custom accelerators should take a bigger bite out of the predictable inference jobs where flexibility carries less value.

Can Cerebras or Groq make more money by selling inference instead of chips?

Cerebras and Groq have a better chance of building large businesses by selling AI inference capacity than by competing with Nvidia rack by rack in merchant hardware.

Cerebras's recent numbers show how quickly the mix can change. Its core cloud and other services revenue reached $127.7 million in the latest quarter, up 287% year over year and already accounting for about 61% of core revenue.

Groq has moved even more decisively toward cloud infrastructure. The company now operates 13 data centers, says its platform serves more than six million developers and processes trillions of tokens every week. It raised $650 million to expand its inference cloud and then announced another $350 million financing round, bringing those recent raises to $1 billion.

This model lets the supplier monetize one accelerator over and over. Instead of recognizing one sale when the hardware ships, the company charges as customers consume inference.

It also changes what customers need to believe. A company may hesitate to buy a proprietary accelerator fleet, train people around it and accept hardware obsolescence. Renting fast inference through an API or dedicated cloud removes much of that commitment.

The economics only work when the infrastructure stays busy. That leads directly to the biggest risk in this model: expensive silicon sitting underused in expensive data centers.

Are long-term AI inference contracts actually better than pay-as-you-go tokens?

Long-term inference-capacity contracts are currently much more valuable to specialized chip companies because they make utilization predictable before billions of dollars are committed to infrastructure.

Cerebras gives us a useful example. Alongside on-demand cloud consumption, the company signs dedicated-capacity arrangements that can require customers to pay for reserved compute even when they do not fully consume it.

That transforms the risk profile. A provider building hundreds of megawatts of infrastructure cannot safely assume developer traffic will arrive exactly when the servers do. Capacity running at 35% utilization produces very different economics from the same equipment running at 80%.

Committed contracts help line up chip manufacturing, data-center leases, power and financing with expected revenue.

The downside is concentration. Cerebras disclosed that two customers represented 76% of its accounts receivable at the end of its latest reported quarter. Its relationship with G42 and MBZUAI has also represented a very large share of historical revenue.

Broadcom faces a softer version of the same problem. Its five largest end customers accounted for roughly 45% of revenue in the first half of its fiscal year.

Large committed customers are therefore extremely valuable, but the best inference business eventually needs several of them. One giant contract can solve utilization while creating a different problem if too much of the company depends on one buyer.

Can specialized inference chips really beat general-purpose GPUs?

Specialized AI inference chips can beat GPUs commercially when the workload is repetitive enough that customers are willing to give up flexibility for lower latency, lower power or more tokens per dollar.

The technical opportunity comes from the way inference behaves. Token generation can become heavily constrained by memory movement and bandwidth, leaving expensive compute units waiting for data. A processor designed around that bottleneck can use its silicon very differently from a general-purpose GPU.

Cerebras attacks the problem with wafer-scale processors and unusually large on-chip memory. Groq built its LPU architecture around predictable, low-latency execution. Etched has taken an even narrower approach with Sohu, designing specifically around transformer inference. With three very different chip designs competing, we try to tell who is ahead and who is falling behind in our AI chips market report.

There is real value in that specialization, but model architectures change faster than semiconductor design cycles. Meta made this point explicitly when explaining its latest MTIA roadmap: the workload a company expects when it begins designing a chip may already have changed by the time the silicon reaches production.

Mixture-of-experts models, longer contexts, multimodality, new numerical formats and different decoding methods can all move the bottleneck.

The strongest specialist will therefore be the one whose advantage survives several model generations. A chip designed perfectly for one narrow snapshot of transformer inference can become obsolete surprisingly fast.

Is selling complete AI racks better than selling inference chips one by one?

Selling complete AI inference systems increasingly looks stronger than selling standalone accelerators because customers care about the performance and cost of the whole rack.

Nvidia has already pushed the market in this direction. Its current platforms combine accelerators, CPUs, high-speed interconnects, networking and software. AMD is following with Helios. Cerebras's wafer-scale design has always tied the processor closely to the surrounding system.

The reason is practical. A chip can look exceptional in isolation and still create a poor cluster if it needs too many networking components, wastes memory bandwidth or consumes too much power around the accelerator.

Complete systems also let the supplier capture more revenue per deployment. The sale can include compute, memory, networking, rack integration and software instead of handing most of that value to other vendors.

Etched is following this path with complete inference systems around Sohu rather than presenting itself purely as another accelerator vendor.

The trade-off is operational complexity. Designing a chip is already difficult; shipping racks brings manufacturing, supply-chain, cooling and deployment problems on top.

For companies that can handle it, though, rack-level economics are where customers increasingly make the real comparison.

Is software now more important than having the fastest inference chip?

Software can decide whether a faster AI inference chip becomes a real business, because customers care much more about running their models reliably than winning a benchmark.

Every alternative platform needs compilers, kernels, model support, quantization tools, orchestration, monitoring and integration with the frameworks developers already use.

AWS has Neuron around its custom chips. Microsoft is building Maia support around familiar software including PyTorch and Triton. Google has spent years making TPU workloads easier to run through its cloud stack. AMD continues to invest heavily in ROCm.

These efforts all attack the same problem: reducing the cost of moving away from CUDA.

Nvidia's revenue makes the importance of that barrier obvious. Plenty of accelerators have produced excellent benchmark results over the past decade. Almost none translated those wins into an ecosystem approaching Nvidia's scale.

A startup that delivers 15% or 20% cheaper inference while creating a painful migration probably has little leverage. A system that makes an important production workload several times faster or materially cheaper has enough room to justify the engineering work.

This is why specialized hardware needs a big advantage. Small benchmark wins get swallowed by software friction.

Can edge inference create big chip winners without competing with Nvidia?

Edge AI inference can produce major chip companies because robots, cars, cameras and industrial machines have a completely different set of constraints from cloud data centers.

A factory robot may need an answer in milliseconds even when its internet connection is unreliable. A camera may generate too much data to send everything to the cloud. A vehicle has strict limits around power consumption, heat and latency.

Those requirements create room for inference architectures that would never compete for a giant frontier-model cluster. Chips that never compete for a giant cluster still have room to grow, and we look at where the biggest opportunities are in our report on AI chips.

Qualcomm is one obvious candidate because it already knows how to sell low-power computing into phones, vehicles and embedded devices. Companies such as SiMa.ai are taking a more specialized route into industrial and physical-AI workloads.

The business moves differently from hyperscale silicon. Nvidia or Broadcom can generate enormous revenue from a handful of infrastructure customers, while edge chip companies often need many OEM design wins. Qualification cycles are slower, but a chip designed into a car or industrial product may stay there for years.

Cloud inference and edge inference therefore deserve separate judgments. A company can build a strong edge business without ever becoming a serious alternative to Nvidia inside an AI data center.

Will cheaper AI inference eventually destroy chip margins?

Cheaper AI inference will hurt undifferentiated chip suppliers first, while companies with a structural cost advantage can benefit as lower prices encourage much heavier usage.

The key question is how quickly consumption rises when tokens get cheaper.

AI products already use more inference than simple chatbots did. Reasoning models can generate long internal reasoning traces. Coding agents make repeated model calls while reading files, testing code and correcting mistakes. Multimodal tools process images, audio and video. Agentic workflows may invoke models dozens of times to finish one task.

This makes falling unit prices compatible with rising infrastructure revenue.

Google's current TPU pricing gives us a useful example of how wide the economic range can be. Ironwood costs $12 per chip-hour on demand in one U.S. region and $5.40 under a three-year commitment. The same underlying hardware can therefore be priced 55% lower when utilization becomes predictable.

The danger comes when competitors reduce costs at the same speed. A startup whose whole pitch is 20% cheaper tokens can lose that advantage after one Nvidia, AMD, TPU or Trainium generation.

Durable inference businesses need a reason their economics keep improving faster than the market: architecture, software, utilization, customer commitments or vertical integration.

Is the inference-cloud model too capital-intensive to win?

Inference clouds can become huge businesses, but the model is unforgiving because the provider has to finance chips, data centers and power before it knows exactly how much inference customers will consume.

Cerebras shows both sides. Its cloud business is growing rapidly, and it now reports 600 MW of data-center capacity under contract. The company also has $25.4 billion of remaining performance obligations. Capacity under contract does not guarantee profit, so we look at who makes money and who burns cash in our AI chips business models report.

Yet the accounting economics need careful reading. Cerebras reported a 20% GAAP gross margin for cloud and other services in its latest quarter. Its adjusted "core" cloud margin was much better because the company excludes items including customer-warrant amortization and certain IPO-related costs.

That gap matters when comparing an inference cloud with Nvidia. Nvidia can sell the accelerator and leave much of the building, financing and utilization risk with its customer. Cerebras or another neocloud earns recurring revenue but carries far more infrastructure exposure.

Groq's recent capital raising points in the same direction. The company raised $1 billion across two announced rounds in a matter of months while expanding its global inference footprint and targeting hundreds of megawatts of capacity.

Capital access is starting to become part of the competitive moat. A company may have excellent inference technology and still fail to capture demand if it cannot finance enough machines, power and data-center space.

Model	Who finances the infrastructure?	Utilization risk	Revenue profile
Nvidia merchant platform	Mostly customer	Low for Nvidia	Hardware/system sales
Broadcom custom silicon	Mostly hyperscaler customer	Low for Broadcom	Large chip programs
Google/AWS internal silicon	Hyperscaler itself	Internal	Cloud and product economics
Cerebras cloud	Cerebras and partners	High	Recurring capacity/usage
Groq cloud	Groq and partners	High	Recurring inference usage
IP licensing	Customer/licensee	Very low	Royalties/licensing
Edge chip sales	OEM/customer	Low	Chip sales over device lifecycle

The inference-cloud model can absolutely win, but we should demand more evidence from it than top-line growth. Utilization, financing cost and gross margin tell us whether those revenues are actually attractive.

Which AI inference-chip business models look strongest today?

The strongest AI inference-chip models today are Nvidia's full-stack platform, hyperscaler custom silicon and vertically integrated cloud silicon; specialized inference clouds have bigger execution risk but could still produce major winners.

Business model	Evidence today	Defensibility	Main weakness	Our judgment
Nvidia-style full platform	$89B quarterly Data Center revenue; 75% company gross margin	Very high	Customer push toward alternatives	Strongest overall
Custom hyperscaler ASICs	Broadcom AI semiconductor revenue at $16.7B and still accelerating	High	Few giant customers	Clear winner
Hyperscaler-owned silicon	TPU, Trainium, Maia and MTIA expanding	Very high inside each platform	Requires enormous scale	Clear winner
Merchant GPU challenger	AMD Data Center revenue at $6.7B	High if software catches up	CUDA advantage	Viable at large scale
Proprietary inference cloud	Cerebras and Groq growing rapidly	Potentially high	Capital and utilization	High upside, higher risk
Complete inference systems	Several specialists moving to rack scale	Medium to high	Operational complexity	Promising
IP licensing	Customers can build around licensed architectures	Architecture-dependent	Smaller share of spending	Attractive niche
Edge inference	Large embedded and physical-AI opportunity	High in specific verticals	Fragmented demand	Separate major market

The ranking comes down to control.

Nvidia controls a mature software and hardware ecosystem. Broadcom sits inside difficult multi-generation custom-chip programs. Google, Amazon, Microsoft and Meta control their own workloads and distribution. Cerebras and Groq are trying to combine proprietary infrastructure with recurring inference revenue.

A standalone accelerator startup has much less protection. Better benchmark numbers give it an opening, but the company still has to defeat established software, procurement relationships, hyperscaler silicon and increasingly complete rack-scale platforms.

Which business models can actually win in AI inference chips?

The AI inference-chip businesses most likely to win are the ones that control more than the processor: Nvidia's full platform, custom hyperscaler silicon, vertically integrated cloud chips and a smaller group of differentiated inference clouds.

Nvidia currently has the strongest model. Its latest Data Center revenue and 75% gross margin show how much value a company can capture when hardware, networking, software and developer adoption reinforce one another.

Custom silicon is the most credible competing model. Broadcom's AI semiconductor revenue has gone from $4.4 billion in the comparable quarter a year earlier to $16.7 billion now, with $21.7 billion expected next. That is already too large to treat custom accelerators as a future threat.

Hyperscaler-owned chips should also keep spreading. Google, Amazon, Microsoft and Meta can justify proprietary silicon through lower cloud and product costs even when nobody outside the company ever buys the chip.

For startups, inference-as-a-service looks more convincing than trying to become another merchant accelerator supplier. Cerebras and Groq can hide hardware differences behind cloud access and collect recurring usage revenue. The price is much heavier capital requirements, so utilization and contract quality matter enormously.

IP licensing and edge inference can work too, but they address narrower versions of the opportunity.

The hardest model to defend is a standalone inference accelerator whose main advantage is a modest benchmark lead. That company faces Nvidia's software ecosystem, AMD's scale, hyperscaler ASICs, Broadcom-designed custom chips and cloud platforms that can bundle proprietary silicon directly into compute services.

Our conclusion is fairly sharp: the AI inference market can support several winners, but very few of them will make their money by simply designing a faster chip and selling it. The durable businesses are attaching silicon to software, cloud distribution, custom customer relationships, recurring capacity, complete systems or a tightly defined edge market.

A fast chip gets a company into the race. Control over the rest of the inference stack is what gives it a business worth keeping.