Do AI data centers need fewer GPUs and more memory?
In our AI infrastructure market deck, you will find everything you need to understand the market
SUMMARY
AI data centers do need fewer GPUs for many fixed workloads and much more memory around each GPU, but total accelerator deployments are still rising because AI demand is growing even faster than efficiency.
The important shift is not from compute to memory. It is from judging systems by peak compute alone to judging how quickly they turn model data, cached context, and network traffic into useful tokens.
Inference exposes this change most clearly. Prompt processing can be compute-heavy, while token-by-token generation often waits on memory bandwidth, so the same request can hit different hardware limits a few seconds apart.
More memory capacity can remove a device boundary. When a model fits on one accelerator instead of two, the operator saves not only a GPU but also communication overhead, deployment complexity, and part of the failure surface.
Bandwidth becomes the next problem once the model fits. NVIDIA keeping Blackwell Ultra and Rubin at 288 GB while increasing bandwidth from roughly 8 TB/s to 22 TB/s is a strong sign that feeding the chip is becoming harder than simply giving it more room.
Long-context models and agents make memory pressure less visible but more persistent. The data center is no longer serving isolated prompts; it is carrying the working history of thousands of ongoing processes, often across many model calls.
Mixture-of-experts models push the hardware balance further toward memory. They compute with only part of the model for each token, yet the much larger expert pool still has to live somewhere and be reachable quickly.
Quantization, prefix caching, paged allocation, speculative decoding, and cache offloading genuinely reduce memory use. In practice, developers usually spend the saving on longer contexts, larger batches, more users, or extra reasoning rather than leaving infrastructure idle.
The likely data-center design is a hierarchy: hot weights and cache blocks in HBM, warmer data in server RAM or pooled memory, and colder reusable context on flash or remote storage. The hard engineering problem is moving each block before the accelerator needs it.
Future inference fleets may also become less uniform. Compute-heavy prefill and bandwidth-heavy decoding can justify different worker pools, schedulers, and eventually different accelerator profiles.
The practical verdict is simple: fewer accelerators per unit of useful work, more memory per accelerator, and more accelerators overall. AI racks are becoming more memory-heavy without becoming less crowded.
This market map, featured in our AI infrastructure market deck, highlights top companies and startups in the AI infrastructure market
Why are AI data centers talking about memory so much now?
AI data centers are talking about memory now because many expensive accelerators spend part of their time waiting for model data or cached context to arrive.
For years, the easiest way to describe AI infrastructure was through GPU counts and computing power. That description is now incomplete. A modern accelerator must load model weights, store temporary calculations, keep previous tokens available, and exchange data with other chips. When memory cannot supply that data quickly enough, thousands of powerful computing cores sit underused.
The latest hardware makes the shift unusually clear. NVIDIA’s Rubin GPU combines 288 GB of HBM4 with 22 TB/s of memory bandwidth. AMD’s MI350 series offers 288 GB and 8 TB/s. Google’s Ironwood TPU has 192 GB and roughly 7.4 TB/s. Three competing platforms have reached the same broad conclusion: AI chips need much larger and faster memory systems around their compute.
Software is moving in the same direction. NVIDIA Dynamo can manage an AI model’s context across GPU memory, server RAM, SSDs, and remote storage. NVIDIA has also introduced a flash-based context-memory tier for agentic workloads. These products exist because keeping useful data close to the accelerator has become a major part of inference performance.
Compute still matters enormously. The difference is that a GPU’s theoretical power tells us less than it used to unless we also know how much memory it has, how quickly that memory can be read, and how efficiently data moves through the rack.
What does “fewer GPUs and more memory” really mean for an AI data center?
For an AI data center, “fewer GPUs and more memory” is mostly true per model or per million tokens, while total GPU fleets are still growing.
The useful comparison is a fixed job: the same model, the same traffic, and the same response-time target. With more memory on each accelerator, a model may fit on one GPU instead of being split across two. With faster memory, one GPU can generate more tokens every second. With better caching, several requests can reuse previous calculations instead of repeating them.
All three improvements can reduce the hardware required for that particular workload.
The industry-wide picture looks very different. Companies are using efficiency gains to run larger models, serve more users, generate longer answers, and let AI agents make dozens of model calls for one task. A system that becomes twice as efficient rarely leads its operator to shut down half the cluster. Usually, it makes twice as many applications affordable.
That resolves most of the confusion around the title. AI data centers increasingly need fewer accelerators for a fixed amount of useful work. They continue to install more accelerators because the amount of useful work is rising even faster.
If you want more recent data on this point, please see our latest AI infrastructure market report.
As this chart shows, and as featured in our AI infrastructure market deck, search interest in AI infrastructure has risen sharply
Are AI workloads memory-bound today?
Many AI inference workloads are already memory-bound today, especially when a model generates tokens one by one.
During token generation, the accelerator repeatedly reads model weights and information from earlier tokens. The amount of calculation required for one new token can be relatively small compared with the amount of data that must be moved. A GPU may therefore have unused computing capacity while it waits for memory.
Batching changes the balance. When a server handles many requests together, one reading of the model weights can support several users. The compute cores become busier, and raw processing power starts to matter more. Low-traffic or low-latency services have fewer opportunities to build these large batches, leaving them more exposed to memory bandwidth.
The prompt-processing stage behaves differently too. Reading and understanding a long input can involve large matrix calculations that use the accelerator heavily. Generating the answer afterwards becomes more sequential and often more dependent on memory.
So an AI accelerator can hit different limits during the same request. Prompt processing may strain compute, token generation may strain bandwidth, and high user concurrency may fill the available memory capacity. Peak FLOPS hide those shifts.
For inference operators, the practical measurement is tokens produced within a latency target. Memory determines that result more often than the impressive compute number printed at the top of a chip’s specification sheet.
Is AI inference more limited by memory than AI training?
AI inference is usually more exposed to memory bandwidth, while AI training still consumes more total compute and far more temporary state.
Training processes many tokens together and performs both a forward pass and a backward pass. The system stores model weights, gradients, optimizer data, and intermediate activations. Those requirements can consume several times the memory occupied by the model itself.
Training teams already use techniques such as state sharding and activation recomputation to control this footprint. Recalculation deliberately spends extra compute to save memory. That trade makes sense because a training run often has plenty of parallel mathematical work to keep the accelerators busy.
Inference removes gradients and optimizer states, which greatly reduces the memory needed for each copy of a model. It introduces another problem: the service must return each token quickly, even when only a small number of requests are available for batching.
A chatbot generating one token after another repeatedly pulls weights and cached attention data through memory. If the memory system cannot feed the accelerator fast enough, adding more compute cores brings little improvement.
We therefore see two different memory problems. Training needs enough capacity for a large working set spread across many accelerators. Inference needs enough capacity for models and active users, plus enough bandwidth to keep every token moving.
This chart, included in our AI infrastructure market deck, shows annual VC investment in AI infrastructure startups
Do long-context AI models and agents make the memory shortage worse?
Long-context AI models and agents make memory pressure much worse because every active session carries a growing block of reusable attention data.
This block is usually called the KV cache. It stores information calculated from previous tokens so the model can continue a conversation without processing the entire history again. As the conversation becomes longer, the cache grows. More simultaneous users multiply the requirement.
A million-token context may sound like a feature of the model, yet it also becomes a serious infrastructure problem. One long session can be manageable. Hundreds of agents simultaneously reading repositories, reports, emails, tool outputs, and their own earlier steps can consume enormous amounts of cache memory.
Agents intensify the problem because one user request may trigger many separate model calls. A research agent might search, read, compare, write, check its answer, and revise it. Each step can reuse part of the earlier context, so retaining that context saves time and compute.
This is why current inference systems are becoming cache-aware. Requests can be routed to the GPU that already holds their context. Older cache blocks can move from HBM to server RAM or flash. Frequently reused blocks can remain close to the accelerator.
The memory required by agentic AI is larger than the visible conversation suggests. Data centers are storing the working history of many ongoing processes, rather than serving a succession of isolated prompts.
Does AI need more memory capacity or more memory bandwidth?
AI systems need both, but memory bandwidth becomes the harder performance problem once the model already fits.
Capacity answers a basic question: can the model, its active context, and enough user requests fit on the accelerator? When the answer is no, operators must reduce the batch, shorten the context, move data to slower memory, or spread the workload across more GPUs.
Bandwidth determines how quickly the accelerator can use the data after it fits. A GPU with a huge memory pool can still generate tokens slowly if the weights and cache cannot reach its compute cores fast enough.
The H200 showed how capacity and bandwidth can improve the same architecture. NVIDIA increased memory from 80 GB on the H100 to 141 GB on the H200 and raised bandwidth to 4.8 TB/s. In NVIDIA’s Llama 2 70B test, the H200 delivered up to 1.9 times the H100’s inference throughput, partly because the extra memory supported a much larger batch.
The move from Blackwell Ultra to Rubin points even more strongly toward bandwidth. Both reach 288 GB per GPU, while Rubin raises bandwidth from roughly 8 TB/s to 22 TB/s. NVIDIA is keeping the maximum capacity broadly stable and giving the chip far more ability to consume it.
That design choice is fairly blunt. Many leading workloads can now fit into several hundred gigabytes. Feeding those workloads quickly enough is the next challenge.
If you want more recent data on this point, please see our latest AI infrastructure market report.
This chart, included in our AI infrastructure market deck, shows why CoreWeave is winning in AI infrastructure
Are new AI chips becoming more memory-heavy?
New AI chips are becoming decisively more memory-heavy, and the shift is visible across NVIDIA, AMD, and Google rather than one vendor.
NVIDIA’s progression is especially revealing. H100 offered 80 GB. H200 increased that to 141 GB without replacing the underlying Hopper architecture. B200 moved to 180 GB, followed by 288 GB on Blackwell Ultra.
AMD has reached the same 288 GB level with MI350X. Google’s Ironwood TPU provides 192 GB per chip and 1.77 petabytes across a full 9,216-chip pod. These products differ in architecture and software, yet all devote much more silicon, power, packaging complexity, and cost to memory.
Bandwidth is rising alongside capacity. H100 delivered a little over 3 TB/s, H200 reached 4.8 TB/s, and several current-generation chips sit near 7 to 8 TB/s. Rubin pushes far beyond that cluster with 22 TB/s.
The broader pattern is hard to miss. Accelerator vendors still add more compute every generation, but they cannot sell that compute effectively without a matching expansion in memory and interconnect.
| AI accelerator | Memory per chip | Memory bandwidth | What the change shows |
|---|---|---|---|
| NVIDIA H100 | 80 GB | Over 3 TB/s | Earlier frontier-training baseline |
| NVIDIA H200 | 141 GB | 4.8 TB/s | Memory upgrade on the same core architecture |
| NVIDIA B200 | 180 GB | About 8 TB/s | Capacity more than doubles from H100 |
| NVIDIA Blackwell Ultra | 288 GB | About 8 TB/s | Larger models and reasoning workloads fit more easily |
| AMD MI350X | 288 GB | 8 TB/s | Large memory becomes a cross-vendor requirement |
| Google Ironwood TPU | 192 GB | About 7.4 TB/s | Google builds a multi-tier memory system around its TPU |
| NVIDIA Rubin | 288 GB | 22 TB/s | Bandwidth becomes the main generational jump |
Can larger-memory GPUs really reduce the number of GPUs?
Yes, larger-memory GPUs can reduce GPU counts when a model is split mainly because it does not fit in one device.
Consider a 70-billion-parameter model stored at 16 bits per parameter. The weights alone occupy roughly 140 GB before we add runtime overhead and the KV cache. An 80 GB H100 cannot hold those weights by itself, so the model must be divided or compressed. A 180 GB B200 has enough room for the weights and some operating headroom.
Moving from two GPUs to one brings more than a lower chip count. The system no longer has to exchange part of every calculation between two devices. Deployment becomes simpler, failures become easier to handle, and small workloads avoid paying the communication cost of distributed inference.
The same logic applies to larger quantized models. A 520-billion-parameter model stored at four bits requires around 260 GB for raw weights. A 288 GB accelerator brings single-device deployment into the realm of possibility, although metadata, cache requirements, and software overhead may still force compromises.
Memory does not remove the need for multiple accelerators when throughput is the goal. A service handling thousands of users may spread the model across many replicas even when one copy fits on one GPU. Training also demands more compute than a single device can provide.
Still, capacity-driven GPU purchases are real. When operators add another accelerator mainly to obtain its memory, a larger-memory chip can genuinely replace that additional device.
This chart, included in our AI infrastructure market deck, shows annual funding in AI infrastructure startups
Do quantization and better software remove the need for more memory?
Quantization and better software save huge amounts of memory, but AI developers are spending those gains on longer contexts, larger batches, and more reasoning.
Reducing model weights from 16 bits to 8 bits roughly halves their raw memory footprint. Moving toward 4-bit formats can cut it toward one-quarter. Similar techniques can compress the KV cache, allowing more users or longer conversations to fit into the same HBM.
Software also reduces wasted space. Paged cache allocation fills memory in smaller blocks. Prefix caching lets different requests reuse identical instructions or documents. Speculative decoding generates several possible tokens at once, reducing the number of slow sequential steps. Smarter schedulers group requests that can share data.
These techniques have changed what can run on each accelerator. They have also raised expectations. Once a model becomes cheaper to serve, developers increase context windows, add reasoning steps, or deploy it to more users. The freed capacity quickly finds another use.
We have seen the same pattern across computing history. Efficiency rarely leaves infrastructure permanently empty when demand is elastic. It lowers the price of an activity and encourages more of it.
| Memory-saving technique | Immediate benefit | Where developers usually spend the saving |
|---|---|---|
| 8-bit or 4-bit weights | Smaller model footprint | Larger models or cheaper replicas |
| Quantized KV cache | More context per GPU | Longer conversations and more users |
| Prefix caching | Less repeated prompt processing | Larger shared documents and instructions |
| Paged cache allocation | Less unused memory | Higher concurrency |
| Activation recomputation | Lower training-memory use | Larger batches or longer sequences |
| Speculative decoding | Fewer sequential decoding steps | Faster answers or more reasoning |
Do mixture-of-experts AI models need more memory than compute?
Mixture-of-experts models strengthen the case for more memory because they store many more parameters than they activate for each token.
DeepSeek-V3 contains 671 billion total parameters while activating about 37 billion for each token. Mixtral’s original technical report described roughly 47 billion total parameters with around 13 billion active during inference.
This architecture saves compute because every token uses only a small selection of experts. The remaining experts still occupy memory and must be available when the model’s router selects them.
A data center may keep all experts in HBM, divide them among several accelerators, or move less frequently used experts through slower memory tiers. Each choice has a cost. Keeping everything close requires enormous capacity. Spreading experts across chips increases network traffic. Offloading them can make the accelerator wait.
Mixture-of-experts models therefore change the preferred hardware balance. Compute requirements follow the active parameter count, while memory requirements remain tied more closely to the total model size.
The gap can become very large. A model may perform calculations similar to a much smaller dense model while carrying several hundred billion parameters in its expert pool. For these workloads, adding memory capacity and interconnect can improve economics more than adding another block of arithmetic power that sits idle too often.
If you want more recent data on this point, please see our latest AI infrastructure market report.
This chart, included in our AI infrastructure market deck, compares the main business model options for AI cloud infrastructure providers
Can CPU RAM, CXL, and SSDs take pressure off GPU memory?
CPU RAM, CXL memory pools, and SSDs can take real pressure off GPU memory, although hot model data still needs HBM speeds.
HBM is extremely fast and physically close to the accelerator, which also makes it expensive and limited. Server RAM offers much more capacity at lower cost. CXL can let several processors access expanded or pooled memory. SSDs provide far larger capacity again, with much higher latency.
The practical answer is a hierarchy. Active weights and the hottest KV-cache blocks stay in HBM. Useful data that can tolerate a short delay moves to server RAM. Colder context, inactive sessions, or reusable cache blocks can sit on SSDs or shared storage.
NVIDIA Dynamo already manages cache blocks across GPU memory, host memory, SSDs, and remote systems. NVIDIA’s context-memory storage platform extends this idea with flash designed specifically for large, frequently reused AI contexts. CXL 4.0 has also doubled its maximum data rate to 128 GT/s and added better support for bundled links and large memory systems.
Even with those improvements, the speed gap remains enormous. Remote memory works best when the system knows what data will be needed and moves it early. Poorly timed transfers can leave the GPU waiting and erase the savings from cheaper storage.
The opportunity is to keep scarce HBM focused on the data that needs immediate access. A well-managed hierarchy can support much larger workloads without attaching the maximum amount of premium memory to every accelerator.
Should AI data centers run prefill and decoding on different hardware?
AI data centers should increasingly separate prefill from decoding because the two stages use hardware in very different ways.
Prefill is the stage where the model processes the user’s input. A long prompt can be handled through large parallel calculations, making strong use of the accelerator’s compute cores.
Decoding begins when the model generates the response. Tokens arrive sequentially, and the system repeatedly reads weights and earlier context. Memory bandwidth and low latency become more important.
Running both stages on identical workers is convenient, though it can leave part of the hardware underused. A compute-heavy accelerator may be ideal for prefill and unnecessarily expensive for a memory-dominated decoding job. A memory-rich device with fewer compute resources may handle decoding more economically.
Current serving software already supports separating the stages. NVIDIA Dynamo can send prefill and decoding to different workers and transfer the resulting KV cache between them. Research designs using specialized prefill and decode hardware have reported simulated infrastructure-cost reductions of roughly 19% to 41%, depending on the workload.
Specialization will make the most sense for large operators with predictable traffic. Smaller services still benefit from flexible GPUs that can handle whichever stage is busy. Even there, separate software pools can improve utilization without requiring completely different chips.
Over time, the phrase “an inference GPU” may become too broad. Data centers may buy one hardware profile to understand prompts and another to generate tokens.
If you want more recent data on this point, please see our latest AI infrastructure market report.
This chart, featured in our AI infrastructure market deck, shows the share of revenue generated by each customer segment in the AI infrastructure market
Will faster AI GPUs make data centers buy fewer GPUs?
Faster AI GPUs cut the hardware needed for a fixed workload, yet data centers are still buying more GPUs because AI usage is growing faster than efficiency.
NVIDIA’s latest reported quarter illustrates the scale of that demand. Data-center revenue reached $75.2 billion, up 92% from the previous year. That growth arrived while new chips were already producing far more tokens per accelerator than earlier generations.
Several forces are absorbing the efficiency gains. Reasoning models generate many internal or visible tokens before reaching an answer. Agents call models repeatedly. Video and multimodal systems process much larger inputs than ordinary text. Lower inference prices also bring in applications that were previously too expensive.
This creates a rebound effect. Suppose a new generation reduces the GPU requirement for a workload from 1,000 units to 400. If usage then grows fivefold, the service needs 2,000 units. Engineers achieved a 60% reduction per unit of work while the fleet still doubled.
Two apparently contradictory headlines can remain true at the same time. Individual AI tasks will use fewer accelerators. The world’s AI infrastructure will contain more accelerators.
A real decline in total GPU demand would require efficiency gains to outrun growth in users, model calls, context length, and reasoning. Current spending and revenue trends show no sign that this crossover has arrived.
Is memory taking a bigger share of AI infrastructure spending?
Memory is taking a much bigger share of AI infrastructure spending, although accelerators still absorb the largest single hardware budget.
Micron expects the HBM market to grow from around $35 billion in 2025 to roughly $100 billion in 2028. The company brought that $100 billion estimate forward by two years compared with its earlier outlook. It has also said that its entire 2026 HBM supply was already covered by price and volume agreements.
This growth comes from more than the number of GPUs. Each accelerator generation carries more HBM, and the memory itself is becoming more complex. Wider interfaces, taller stacks, advanced packaging, and stricter thermal requirements raise the value of the memory attached to every chip.
Server DRAM and data-center SSDs are benefiting too. AI systems need RAM for model loading, cache offloading, retrieval databases, preprocessing, and CPU-side services. Flash is beginning to store active inference context rather than only files and model checkpoints.
Memory spending can grow faster than GPU unit shipments when both memory per accelerator and the price of high-end memory rise. That is happening now.
More memory spending does not mean data centers will redirect most of their accelerator budgets. Memory is becoming a larger component of a rapidly expanding system. Both sides of the infrastructure bill are growing.
This chart, included in our AI infrastructure market deck, shows how GPU cloud infrastructure technology has evolved over time
What should the next AI data center optimize for?
The next AI data center should optimize for useful tokens per dollar and per watt, with memory movement treated as a core design problem.
GPU count is an input, rather than the final measure of success. A rack containing more accelerators can still perform poorly if those accelerators wait for data, repeatedly rebuild the same cache, or exchange too much information across slow links.
Different workloads need different balances. Frontier training requires dense compute and extremely fast communication between accelerators. Short-context inference benefits from bandwidth and efficient batching. Long-context services require more cache capacity. Agent platforms need context reuse and storage that can follow a session across many calls.
The design of the rack has to follow the workload. Buying the same accelerator configuration for every task wastes compute, memory, power, or all three.
| AI workload | Main infrastructure pressure | Sensible data-center response |
|---|---|---|
| Frontier-model training | Compute and accelerator networking | Dense compute with fast scale-up links |
| Post-training | Activations and temporary training state | High HBM capacity and memory-efficient software |
| Short-context inference | Weight bandwidth and batching | Fast HBM with strong request scheduling |
| Long-context inference | KV-cache capacity | More HBM and multi-tier cache management |
| AI agents | Reusable context across many calls | Cache-aware routing and context storage |
| Mixture-of-experts models | Large expert pool and irregular traffic | High capacity with fast chip-to-chip links |
| High-volume decoding | Cost and energy per token | Memory-heavy inference pools or specialized hardware |
If you want more recent data on this point, please see our latest AI infrastructure market report.
Do AI data centers need fewer GPUs and more memory?
Partly: AI data centers need far more memory per accelerator and fewer GPUs for many fixed jobs, while total GPU deployments are still rising.
The claim is accurate at the workload level. Larger HBM pools let models fit on fewer devices. Faster memory increases the output of each accelerator. Better caching avoids repeated computation. Separating prefill from decoding can also prevent data centers from paying for compute that a memory-bound stage barely uses.
The claim becomes misleading when applied to the entire industry. AI usage, reasoning length, context size, agent activity, and multimodal workloads are expanding faster than hardware efficiency. NVIDIA’s current revenue trajectory confirms that operators continue to add accelerator capacity aggressively.
Future AI racks will become much more memory-heavy without becoming less GPU-dense. Every accelerator will carry more HBM, rely on larger pools of server RAM and flash, and use faster links to reach data elsewhere in the system.
The clearest verdict is that AI data centers need fewer GPUs per unit of work and more memory around every GPU. Across the whole market, they currently need more of both.
In our AI infrastructure market deck, we identify pain points entrepreneurs should prioritize
OUR METHODOLOGY
This analysis tests whether AI data centers need fewer GPUs and more memory by separating three different questions: what a fixed workload requires, how an individual operator designs its infrastructure, and what happens to accelerator demand across the market as a whole.
We studied memory capacity, memory bandwidth, model fit, inference throughput, KV-cache growth, quantization, memory tiering, mixture-of-experts architectures, prefill–decode specialization, and aggregate accelerator demand. For each dimension, we assessed whether the evidence applied mainly to training, inference, long-context serving, agentic workloads, or the broader data-center market.
We compared NVIDIA, AMD, and Google hardware so that the direction of system design did not depend on one vendor’s roadmap. Product specifications establish how much memory and bandwidth current accelerators provide, but they are not treated as substitutes for workload benchmarks, latency measurements, or real deployment constraints.
Model-size calculations are used only to test whether additional memory can remove a device boundary. A model fitting on one accelerator instead of two can reduce communication and deployment overhead, but it does not prove that one accelerator can meet the required throughput, latency, concurrency, or runtime headroom.
We also separated memory capacity from memory bandwidth. Capacity determines whether the model, KV cache, and active requests fit. Bandwidth determines how quickly the accelerator can use that data after it fits. This distinction is central to interpreting the move from high-capacity chips such as Blackwell Ultra to much higher-bandwidth designs such as Rubin.
Efficiency and demand were analyzed separately. A newer system can require fewer GPUs for the same model, traffic, and latency target while its operator still buys more GPUs because usage, context length, reasoning, multimodal processing, and agent activity are growing faster than hardware efficiency.
Key sources include NVIDIA’s Rubin architecture overview, NVIDIA’s Vera Rubin platform documentation, NVIDIA’s H200 specifications, AMD’s MI350X specifications, Google Cloud’s Ironwood TPU overview, NVIDIA Dynamo documentation, NVIDIA’s KV Block Manager documentation, and NVIDIA’s technical discussion of KV-cache bottlenecks.
Additional sources include NVIDIA’s context-memory storage platform, the DeepSeek-V3 technical report, the Mixtral technical report, the SPAD prefill–decode research paper, the HMA-Serve memory-heterogeneous serving paper, the CXL Consortium’s CXL 4.0 overview, NVIDIA’s fiscal Q1 2027 results, and Micron’s fiscal Q1 2026 earnings presentation.
The final conclusion reflects the evidence that remained consistent across these dimensions: larger and faster memory can reduce GPU requirements for many fixed workloads, but expanding models, contexts, agents, and user demand continue to increase total infrastructure requirements.
This chart, included in our AI infrastructure market deck, shows the share of revenue by region across Europe, Asia, North America, Africa, and South America in the AI infrastructure market
Related blog posts
- AI chip memory: which startup is ahead?
- Is GPU memory more important than GPU speed now?
- Is memory becoming the bottleneck for AI?
- Who is paying for OpenAI’s data centers?
