Which AI inference chip startups can challenge Nvidia?

Is any AI inference chip startup actually close to Nvidia today?

No AI inference chip startup is close to Nvidia's overall scale today, but a small group can now take production inference workloads that would otherwise have run on Nvidia GPUs.

The size gap is still almost absurd. Nvidia's latest reported quarter produced $89 billion of Data Center revenue, up 117% from a year earlier. One quarter of Nvidia's datacenter business is therefore worth vastly more than the annual revenue of every private inference-chip challenger combined.

Yet the competitive question has changed. A few years ago, most Nvidia alternatives had prototype silicon, benchmark slides and promises about performance per watt. These days, we can point to commercial racks, production cloud services and customers actually paying for alternative inference hardware. Prototypes and benchmark slides have given way to racks and paying customers; we show who leads on shipments and who only promises in our report on who's ahead in AI chips.

SambaNova has JPMorganChase using its RDUs for on-premises inference and OVHcloud using them for hosted AI endpoints. FuriosaAI has moved RNGD into mass production and Samsung SDS now sells RNGD-powered cloud instances. Rebellions is shipping RebelRack systems and has a Japanese deployment planned to scale toward 100 racks. Positron has already delivered roughly 50 Atlas racks and is developing its larger Asimov processor after raising $875 million at a $5 billion valuation.

Cerebras shows how far a specialist architecture can eventually go. The company is public now, so we should no longer call it a startup, but its latest quarter included about $210 million in core revenue, with cloud revenue nearly quadrupling year over year. It also has agreements with OpenAI and AWS.

Nobody is close to replacing Nvidia as the default AI computing platform. Several companies are already credible alternatives for particular inference jobs.

Company	Where it stands now	Best evidence	Main problem
Nvidia	Dominant AI computing platform	$89B quarterly Data Center revenue	High power and cost leave openings
Cerebras	Public specialist benchmark	~$210M quarterly core revenue; major OpenAI and AWS deals	No longer a startup
SambaNova	Leading private full-stack contender	JPMorganChase, OVHcloud, sovereign deployments	Tiny next to Nvidia
Positron	Fast-rising datacenter challenger	~50 Atlas racks shipped; $875M new financing	Next-generation ASIC still has to prove itself
Etched	Very high-upside newcomer	First Jane Street rack; >$1B customer demand	Production history is extremely short
d-Matrix	Production inference accelerator	Corsair now shipping in volume	Often complements Nvidia rather than replacing it
FuriosaAI	Real low-power production alternative	4,000 RNGD chips received; Samsung SDS cloud service	Smaller software ecosystem
Rebellions	Strong sovereign-AI contender	RebelRack shipping; Japanese expansion	International footprint is still early

What would it actually mean for an AI chip startup to challenge Nvidia?

An AI inference startup can challenge Nvidia without replacing CUDA or taking Nvidia's market-share crown; winning a meaningful slice of expensive production inference would already count.

That distinction keeps the rest of the analysis realistic. Training a frontier model and serving that model to millions of users have very different hardware requirements. Inference itself also covers several very different problems. An interactive coding agent cares intensely about how quickly each token arrives. A batch-processing service may care much more about total tokens per dollar. An enterprise datacenter may care about fitting new compute inside an existing 15-kilowatt rack. A sovereign cloud may care about avoiding dependence on one American supplier.

Groq built around predictable low-latency token generation. SambaNova is pushing fast large-model inference through reconfigurable dataflow hardware. d-Matrix concentrates heavily on the decode stage. FuriosaAI emphasizes what can run efficiently in ordinary air-cooled datacenters. Positron is attacking the memory bottleneck. Etched takes the most extreme route by building silicon specifically around transformer workloads.

A meaningful challenge would therefore look like customers routinely saying, "We use Nvidia for this workload, but this other chip makes more sense for that one."

Why are AI inference chips a bigger threat to Nvidia than AI training chips?

AI inference gives startups a better opening because repetitive production workloads reward specialization much more than frontier-model training does.

Training still favors an extremely flexible platform with mature software, networking and huge clusters. Nvidia has spent years building exactly that. Once the model exists, however, serving it efficiently becomes a different engineering problem.

Token generation is often limited by moving model weights and cached data through the system. Simply adding more theoretical arithmetic does not solve every bottleneck. That creates room for architectures with unusually large on-chip memory, different dataflow designs, deterministic execution or compute located closer to memory.

Agentic AI makes those inefficiencies more expensive. A chatbot might invoke a model once for one user request. A coding agent can call the model over and over while reading files, planning, writing code, checking results and trying again. Small latency or cost differences then multiply across a long chain of model calls.

SambaNova has demonstrated systems where GPUs handle prefill while its RDUs handle decode. d-Matrix is pursuing a similar disaggregated approach. Cerebras is working with AMD on another version of the same idea.

If this architecture becomes common, startups can take meaningful accelerator spending without displacing the entire Nvidia server.

How hard is Nvidia's AI inference moat to break?

Nvidia's biggest advantage today is everything surrounding the GPU, which makes a technically faster inference chip much less threatening than benchmark charts suggest.

CUDA remains part of that advantage, but the moat now stretches much further. Nvidia has TensorRT-LLM for model optimization, Dynamo for distributed inference, NIM for deployment, Triton for serving, NVLink for scale-up communication and Spectrum-X or InfiniBand for networking. Nvidia's moat spans a long list of tools, so we measure how far the challengers are behind it in our AI chips market report.

Customers buying Blackwell infrastructure are therefore buying a working system rather than assembling a chip, compiler, networking fabric and production serving stack themselves.

Nvidia also keeps improving hardware that customers already understand. According to the company's recent financial results, Blackwell now represents the vast majority of Nvidia's revenue. Nvidia says its upcoming Rubin platform can reduce inference token costs by as much as tenfold compared with Blackwell in some configurations.

That pace creates an unpleasant problem for startups. Beating H100 no longer tells us much. By the time new accelerator silicon reaches mass production, Nvidia customers may be installing Blackwell Ultra or preparing for Rubin.

A challenger therefore needs a large advantage in a workload customers genuinely care about. A 10% benchmark win will not pull serious infrastructure away from Nvidia. A several-fold improvement in latency, rack density or total cost might.

Nvidia advantage	Why startups struggle with it
CUDA	Developers already know the platform
TensorRT-LLM	Popular models get heavily optimized
Dynamo	Nvidia can optimize inference across whole clusters
NVLink	GPUs communicate extremely quickly inside rack-scale systems
Spectrum-X / InfiniBand	Nvidia also controls much of the networking stack
NIM and Triton	Deployment tooling is already mature
Cloud availability	Every major cloud sells Nvidia capacity
Huge installed base	Customers already own Nvidia infrastructure
Fast product cadence	Startups are competing against the next Nvidia chip, not the last one

Has Cerebras already proved that Nvidia GPUs can lose AI inference workloads?

Yes, Cerebras has already proved that specialized AI inference hardware can win major workloads from GPUs, although Cerebras itself is now public rather than a startup.

Its strongest advantage is latency. Cerebras regularly sits near the top of independent inference-speed measurements because its wafer-scale design keeps enormous amounts of compute and memory bandwidth close together.

Its latest systems can serve GPT-5.6 Sol at around 750 tokens per second, according to the company's latest financial filing. Other supported open models have reached far higher output rates. Once an AI agent makes many sequential calls, cutting a generation step from several seconds to around one second materially changes what the product feels like.

The commercial numbers now support the technical story. Cerebras recently reported roughly $210 million of core quarterly revenue, more than double the previous year's level, while its cloud business nearly quadrupled. Its customer list now includes OpenAI, Cognition, Lovable, Block, Figma, AlphaSense and GSK. It has also contracted around 600 MW of datacenter capacity.

Cerebras has answered one important question already: serious customers will put important generative-AI workloads on something other than Nvidia GPUs.

Is SambaNova currently the strongest private Nvidia challenger?

SambaNova currently has the best combination of mature silicon, large-model performance, enterprise customers and enough money to keep scaling.

The company recently completed the first close of a $1 billion financing at an $11 billion post-money valuation. That valuation by itself tells us very little about competitiveness. The better evidence comes from who is actually installing the hardware.

JPMorganChase selected SambaNova RDUs for fast on-premises inference. OVHcloud chose the platform for premium AI endpoints in Europe. SambaNova also powers sovereign-AI services and has been expanding partnerships in Japan and other markets.

The architecture is useful because large language models constantly move huge quantities of data. SambaNova's systems give models high memory capacity and bandwidth without requiring the same GPU cluster structure. Its newer SN50 generation has continued to perform strongly on independent inference testing, particularly during decode.

SambaNova is also willing to live beside Nvidia. Its work with Intel has included a disaggregated inference design where GPUs handle prefill and SambaNova RDUs handle decode.

We still lack a clean view of SambaNova's revenue, and that matters. But among private companies, it currently has the strongest mix of real deployments, technical maturity and financing.

Is Positron becoming a serious Nvidia competitor faster than expected?

Positron has suddenly become one of the more credible Nvidia inference challengers because it has moved from an interesting memory idea to shipped systems, major customers and a $5 billion valuation.

Earlier in the year, Positron raised $230 million at a valuation just above $1 billion. More recently, the Wall Street Journal reported another $875 million round at $5 billion. Two rounds a few months apart is a fast climb; we cover who is getting funded, and by whom, in our report covering AI chips.

That fivefold valuation jump would be easy to dismiss if nothing operational had changed. But Positron has already shipped roughly 50 Atlas server racks, including systems for Oracle, and has customers that include Jump Trading and Parasail.

Its basic bet is easy to understand. Large-model inference spends a huge amount of time moving data. Positron therefore designs around memory capacity and bandwidth rather than trying to win a contest in headline FLOPS. Its coming Asimov processor is expected to offer up to 2.3 terabytes of memory per chip, an unusually large amount aimed at inference workloads where keeping models close to compute can dramatically change economics.

Positron also tries to avoid some of the supply-chain bottlenecks affecting leading AI accelerators by reducing dependence on HBM and advanced CoWoS packaging.

Atlas proves that Positron can build and deliver hardware. Asimov now has to prove that the same idea works against Blackwell and Rubin at much larger scale.

Is Etched really worth taking seriously as an Nvidia challenger already?

Etched deserves to be taken very seriously, but its $21 billion valuation is far ahead of the amount of production evidence we have today.

Etched makes the most aggressive architectural bet in the group. Sohu is designed around transformers rather than serving as a broadly programmable accelerator. Removing hardware that transformer inference does not need can free enormous amounts of silicon and power for the operations that matter.

The company now has stronger evidence than a normal pre-production semiconductor startup. Etched says it has more than $1 billion of customer demand, and Jane Street tested the hardware before leading a $700 million financing at a $21 billion valuation. Etched has also shipped its first rack to Jane Street.

Shipping a first rack is a big technical milestone, but Nvidia operates at an entirely different industrial scale. Etched still needs to prove reliability across large fleets, software stability, supply-chain execution and customer support.

Its second risk comes directly from the specialization that makes Sohu attractive. Mixture-of-experts models, multimodal workloads, new attention mechanisms and new inference techniques can shift where compute is needed. A GPU can adapt through software more easily than hardwired silicon can.

Etched has huge upside, but today its valuation reflects much more than the company has yet proved in production.

Can d-Matrix take business from Nvidia without replacing Nvidia GPUs?

Yes. d-Matrix may have found one of the easiest ways to take inference spending from Nvidia because its accelerator can sit beside GPUs and handle the part of inference where GPUs are less efficient.

Its Corsair platform uses digital in-memory compute, bringing arithmetic closer to the data instead of repeatedly moving model information back and forth through the chip.

Corsair recently entered full production, and d-Matrix says volume shipments are beginning for hyperscalers, neoclouds and frontier AI labs. As Corsair ships to hyperscalers and neoclouds, we watch which technologies are gaining traction in our AI chips report.

The more interesting work is happening in heterogeneous inference. d-Matrix has shown systems where GPUs handle one part of a model request while Corsair takes the generation workload. In specific disaggregated configurations, the company has reported more than a tenfold response-time improvement.

d-Matrix has also acquired GigaIO's datacenter business and Wallaroo.ai while embracing Nvidia's NVLink Fusion ecosystem.

That approach lowers the switching barrier considerably: customers can keep their Nvidia infrastructure while moving decode onto another accelerator.

Are FuriosaAI and Rebellions now real Nvidia alternatives?

FuriosaAI and Rebellions are real AI inference alternatives today, particularly where electricity, existing datacenter infrastructure and sovereign control matter more than having the broadest possible software ecosystem.

FuriosaAI has made the clearer jump into commercial production. The company received an initial batch of 4,000 RNGD accelerators from its manufacturing partners and now sells the chip as a PCIe card and inside turnkey servers.

RNGD operates at about 180 watts. That lets Furiosa pack substantially more inference compute into ordinary air-cooled datacenters than high-end GPU systems that demand much denser power and cooling. Samsung SDS has already turned the hardware into a commercial NPU-as-a-Service product.

Rebellions has meanwhile raised $400 million in pre-IPO financing at around a $2.34 billion valuation, taking total funding to roughly $850 million. RebelRack and RebelPOD package Rebellions chips into deployable infrastructure rather than leaving customers to integrate accelerators themselves.

Its international footprint has also started to become more concrete. A recent Japanese partnership calls for deployment in a Tokyo datacenter with room to scale beyond 100 RebelRack systems.

Neither company has anything close to Nvidia's worldwide footprint. Their opportunity comes from customers whose power constraints, sovereignty requirements or supplier preferences make the Nvidia default less attractive.

Company	What is real today	Where it has an edge	What remains weak
FuriosaAI	RNGD mass production; Samsung SDS NPU cloud	180W design; existing air-cooled datacenters	Smaller global ecosystem
Rebellions	Shipping RebelRack and RebelPOD	Sovereign AI; Korean ecosystem	International scale
SambaNova	Enterprise and sovereign deployments	Large-model inference	Revenue visibility
Positron	Dozens of racks already delivered	Memory-heavy inference	Asimov still needs large-scale proof
d-Matrix	Corsair in volume production	Decode and heterogeneous inference	Fewer named large deployments
Etched	First production rack	Extreme transformer specialization	Very early manufacturing ramp
Groq	Large global inference cloud	Very low latency	Now deeply tied to Nvidia

Does Groq still count as an independent threat to Nvidia?

Groq still gives customers an alternative inference service, but it is becoming much harder to describe Groq itself as an independent strategic threat to Nvidia.

Groq built one of the strongest early cases for specialized inference. Its Language Processing Unit uses deterministic execution and large amounts of fast on-chip memory to produce tokens with unusually predictable latency.

GroqCloud also achieved real scale. The company now says it serves more than six million developers across 13 datacenters and processes trillions of tokens each week.

Then the relationship with Nvidia changed dramatically.

Groq licensed inference technology to Nvidia on a non-exclusive basis, while founder Jonathan Ross, president Sunny Madra and other members of the team moved to Nvidia. Groq remained an independent company under a new CEO.

Lately, the two companies have moved even closer. Groq became an Nvidia Cloud Partner, clearing the way for its datacenters to run Nvidia accelerated computing alongside Groq infrastructure. Nvidia also planned to participate in Groq's latest $350 million financing, which valued the company at $3.5 billion.

Groq's operating footprint remains large enough to matter. Its deepening Nvidia relationship, however, makes it much less convincing as an independent challenger.

Are specialized inference chips actually faster and cheaper than Nvidia GPUs?

Some specialized AI inference chips are dramatically faster for interactive token generation and can be cheaper in specific workloads, but neither advantage holds automatically across an entire datacenter.

Cerebras gives the clearest speed example. Its systems can produce hundreds or, on some open models, well over a thousand tokens per second for a single stream. SambaNova and Groq also regularly appear near the top of independent output-speed rankings.

For agents, that difference can be substantial. Suppose an agent needs ten sequential generation steps of 1,500 tokens each. At 50 tokens per second, generation alone can consume roughly five minutes. At 500 tokens per second, the same output takes about 30 seconds before we account for tool use or other overhead. Agents chaining many calls have changed what speed is worth, one of the shifts we follow over the past year in our AI chips market report.

Cost is more complicated. FuriosaAI's 180-watt RNGD can fit inside racks that cannot support the power density of modern Nvidia systems. Positron attacks the memory bill. d-Matrix tries to reduce expensive data movement. Etched removes general-purpose circuitry in favor of transformer-specific hardware.

Those architectural advantages can disappear if the accelerator sits idle. A flexible Nvidia GPU can run many models and many kinds of jobs, helping cloud providers keep utilization high.

Nvidia is also pushing inference costs down quickly. The company says Rubin can cut token costs by as much as tenfold relative to Blackwell for some inference configurations.

So the strongest startup opportunities are where the gain is unmistakable: much lower latency, dramatically higher rack density, lower power or a large saving on one specific stage of inference.

Can AI inference startups keep up when AI models change so quickly?

Model changes remain one of the biggest risks for specialized AI chips, and the most rigid architectures carry the most risk.

Recent models have already complicated inference. Mixture-of-experts architectures activate different parts of a network for different tokens. Long context creates huge key-value caches. Multimodal systems mix text, vision, audio and sometimes video. New decoding techniques can alter how much work happens during each generation step.

Every change can move the hardware bottleneck.

Nvidia benefits from programmability here. When researchers invent a new model structure, developers can usually start making it work on GPUs through software before anyone could design, fabricate and deploy a new ASIC.

The better startups understand this problem. SambaNova uses a reconfigurable architecture rather than fully fixed-function silicon. Tenstorrent combines programmable compute with an open software stack. FuriosaAI has been releasing new SDK versions rapidly as it adds model support, batching, Kubernetes integration and production tooling. d-Matrix bought Wallaroo.ai partly to strengthen its software layer.

Etched has the biggest exposure because Sohu deliberately sacrifices flexibility for transformer efficiency. That choice could be brilliant if frontier workloads remain close to the assumptions built into the chip. A meaningful architectural break would hurt Etched more than a programmable competitor.

Can AI chip startups scale hardware without forcing customers off CUDA?

Only a few AI inference startups have shown that they can manufacture at meaningful volume, and the easiest adoption path currently keeps Nvidia infrastructure in place rather than asking customers to abandon CUDA.

FuriosaAI receiving 4,000 RNGD accelerators is meaningful because thousands of production chips are very different from engineering samples. d-Matrix moving Corsair into full production clears another important hurdle. Positron's roughly 50 delivered Atlas racks show that it can build complete systems rather than just chips.

Etched is earlier. The Jane Street rack proves the product exists and works well enough for a demanding customer to install it, while the company now has to go from that first deployment toward the much larger infrastructure it talks about publicly.

Manufacturing is only half the problem. Direct hardware customers also need monitoring, drivers, orchestration, model support, debugging tools and engineers who understand the system. Nvidia is far ahead on those parts of production infrastructure.

That is why heterogeneous inference keeps appearing. SambaNova works with GPUs for prefill and RDUs for decode. d-Matrix connects its accelerators to Nvidia infrastructure. Cerebras is developing disaggregated systems with AMD.

Hosted APIs make adoption easier still. A developer calling a model endpoint may not care whether the response came from an Nvidia GPU, a Groq LPU, a Cerebras wafer-scale engine or a SambaNova RDU.

The most realistic route for startups is to enter through one workload while leaving CUDA and Nvidia hardware in place around it. That dramatically lowers both the manufacturing and switching barrier.

Which AI inference chip startups can actually challenge Nvidia now?

SambaNova, Positron, Etched and d-Matrix are the private AI inference chip companies we would take most seriously right now, although none comes close to threatening Nvidia across the whole AI accelerator market.

SambaNova deserves the top spot today. It combines production hardware, strong large-model inference, enterprise customers, sovereign deployments and enough financing to keep building. We can see evidence of adoption across several unrelated customers rather than one spectacular benchmark.

Positron moves into the first tier after its recent financing and hardware deliveries. Roughly 50 shipped Atlas racks give it more commercial substance than many startups with larger valuations, while its upcoming Asimov architecture attacks a genuine problem in inference: moving enormous models through memory. The uncertainty is whether the next-generation chip lives up to the enthusiasm already priced into a $5 billion valuation.

Etched has the largest upside. More than $1 billion of demand, a Jane Street deployment and a $21 billion valuation put it in a completely different category from a typical semiconductor startup. Its production ramp and architectural rigidity also make it the riskiest company near the top.

d-Matrix has a less glamorous but very practical route into the market. If GPUs keep prefill while purpose-built hardware increasingly handles decode, d-Matrix can capture valuable inference spending without asking enterprises to abandon infrastructure they already own.

FuriosaAI and Rebellions follow. Both have crossed into real commercial hardware and can become important in sovereign and energy-constrained datacenters. Their current footprints are much smaller than the first group.

Groq remains a large inference provider, yet its deepening relationship with Nvidia now makes it a poor candidate for the title of independent Nvidia disruptor.

Tenstorrent is harder to rank. Its architecture and open-stack ambitions are interesting, and Blackhole systems are shipping, but we still have less public evidence of large recurring inference deployments than we do for the companies above.

At the edge, SiMa.ai is becoming a meaningful competitor to Nvidia Jetson rather than Nvidia's main datacenter business. The company just raised $150 million at a $1.45 billion valuation and is targeting robotics, drones, automotive systems and other physical-AI devices.

The ranking is pretty clear. Nvidia currently has no startup equivalent capable of challenging the whole platform. What Nvidia does face is a growing group of specialists attacking the places where a universal GPU architecture becomes expensive: ultra-fast generation, decode, memory-heavy inference, low-power datacenters, sovereign infrastructure and physical AI.

Cerebras has already shown that one of those wedges can become a large commercial business. The private companies most likely to repeat that today are SambaNova, Positron, Etched and d-Matrix.

Rank	Company	Our view today	Why it could take Nvidia workloads	Biggest unresolved issue
1	SambaNova	Strongest private challenger	Production systems, enterprise buyers, sovereign deployments, fast large-model inference	Revenue scale remains unclear
2	Positron	Fastest-rising challenger	Shipped racks, strong customers, memory-first architecture, huge new funding	Asimov still needs production proof
3	Etched	Highest upside	Extreme specialization, >$1B demand, Jane Street deployment	Very early production ramp
4	d-Matrix	Best decode specialist	Volume production and easy fit beside GPUs	May remain one component of a Nvidia-heavy stack
5	FuriosaAI	Strong efficiency play	Mass-produced 180W hardware and Samsung SDS deployment	Smaller ecosystem and global footprint
6	Rebellions	Credible sovereign-AI challenger	Shipping rack systems and expanding outside Korea	Still early internationally
7	Tenstorrent	Technically ambitious contender	Shipping Blackhole hardware and open architecture	Less visible commercial inference scale
8	Groq	Major provider, weaker independent threat	Six million developers and large global cloud	Nvidia relationship keeps getting deeper
9	SiMa.ai	Serious edge challenger	Production physical-AI hardware and strong new financing	Competes mainly outside central datacenter inference