What is the biggest AI bottleneck today?
In our AI infrastructure market deck, you will find everything you need to understand the market
SUMMARY
Power delivery is the biggest AI bottleneck today. The problem is not a global shortage of electricity in the abstract, but the difficulty of delivering dependable power through grids, substations and electrical equipment to the exact places where very large computing clusters need to operate.
We are probably not building too many AI data centers overall. We are building too many speculative projects without firm power, credible completion dates or committed tenants, while genuinely power-ready sites remain scarce.
The old GPU shortage has turned into a complete-system shortage. A chip only becomes useful once memory, packaging, networking, cooling, switchgear and an energized building are ready at the same time.
Advanced packaging and high-bandwidth memory are still tight, but their supply chains are relatively concentrated and manufacturers can invest against visible orders. Grid expansion is spread across utilities, regulators, equipment suppliers, landowners and local communities, which makes it much harder to accelerate.
“Speed to power” is becoming a better competitive measure than announced megawatts or planned server counts. A financed campus full of ordered hardware is still mostly an option until the site can actually be energized.
Efficiency will not remove the bottleneck by itself. The cost and energy required for a comparable AI task are falling quickly, but cheaper intelligence creates more usage, larger models, video generation, extended reasoning and agents that call models repeatedly.
The biggest bottleneck changes when the unit of analysis changes. Reliability limits autonomous agents, messy data and unchanged workflows limit enterprise returns, and export controls can make advanced chips the binding constraint for Chinese frontier developers.
Power is different because it reaches across the stack. It constrains new model training, inference expansion, data-center construction and the geographic distribution of AI capacity at the same time.
Capital will become more selective, and some announced campuses will disappear. That correction will remove weak projects, but it will not manufacture transformers faster, shorten grid studies or guarantee community approval for new generation and transmission.
The next phase of the AI race will be organized around energized computing clusters rather than chips alone. The winners will be the companies and regions that can combine hardware, power, electrical equipment and permits on one workable timetable.
This market map, featured in our AI infrastructure market deck, highlights top companies and startups in the AI infrastructure market
What is the biggest AI bottleneck today?
Power delivery is the biggest AI bottleneck today. The precise constraint is access to dependable electricity through a grid, substation and equipment stack that can energize very large computing clusters on the timeline developers want.
Chips remain tight, reliability still limits autonomous agents and messy workflows hold back enterprise adoption. None of those constraints has the same reach, rigidity and lead time across the full AI system.
What should count as the biggest AI bottleneck today?
The biggest AI bottleneck today is the constraint that blocks the most valuable progress across the widest part of the industry.
That rules out several easy answers. A frontier laboratory cares about training capacity. A cloud provider cares about how quickly it can energize a new cluster. A company using AI to process invoices cares about data access, reliability and workflow design. Each group can honestly name a different bottleneck.
For this article, we are looking for a constraint with four qualities: it affects many players, substitutes are limited, new supply takes a long time to build, and removing it would release a large amount of delayed AI capacity. That definition points toward power delivery across the wider system, even though chips, reliability and company integration can be more painful in specific situations.
| Where progress is blocked | The immediate constraint | Our judgment today |
|---|---|---|
| Frontier model development | Chips, memory, packaging and large-scale power | Power-ready computing clusters |
| New AI data centers | Grid connections, electrical equipment and permits | Power delivery |
| Autonomous AI agents | Error rates and weak long-task reliability | Reliability |
| AI inside ordinary companies | Fragmented data and unchanged workflows | Integration and workflow redesign |
| AI access in lower-income markets | Affordable compute, dependable power and connectivity | Local infrastructure |
If you want more recent data on this point, please see our latest AI infrastructure market report.
As this chart shows, and as featured in our AI infrastructure market deck, search interest in AI infrastructure has risen sharply
Why did the answer move beyond GPUs?
The answer has moved beyond GPUs because computing hardware is arriving faster than the physical systems needed to use it.
The chip shortage was the clearest constraint during the first wave of generative AI. Cloud customers waited for accelerators, laboratories rationed training capacity and NVIDIA could sell almost everything it shipped. That pressure has hardly vanished, yet the scale of the supply response has changed the picture.
The International Energy Agency now estimates that the capacity of specialized “AI factories” more than tripled in only 18 months. Model capability kept improving at the same time. Stanford’s latest AI Index found that performance on SWE-bench Verified rose from around 60% to nearly 100% in one year. The industry is still adding both hardware and capability at extraordinary speed.
Physical infrastructure moves on a different clock. A model can be updated several times while a utility is still studying one grid connection. A new accelerator generation can reach customers before a substation, transformer or transmission upgrade is ready. These days, the slowest part of AI expansion is increasingly everything around the chip.
Are AI chips still the main bottleneck?
GPUs are still scarce, but they no longer explain the biggest constraint across AI.
NVIDIA’s latest quarterly results make both sides of the argument clear. Data-center revenue reached $75.2 billion, up 92% from a year earlier. Demand is still enormous, yet those sales also show that a huge volume of hardware is reaching customers. The industry has moved far beyond the period when a small increase in GPU shipments could unlock most delayed projects.
A modern accelerator also needs high-bandwidth memory, advanced packaging, fast networking, cooling and an energized building. Missing any one of those pieces leaves an expensive processor idle or prevents the project from being built. The useful measure is how many complete, powered systems can operate, not how many GPU dies can be fabricated.
Chips still dominate in a few cases. Export controls make advanced compute a direct constraint for Chinese laboratories. Smaller startups can face prohibitive cloud prices. Frontier labs also need the newest hardware in concentrations that few buyers can secure. Across the whole market, though, usable computing capacity has become harder to expand than chip purchases alone.
If you want more recent data on this point, please see our latest AI infrastructure market report.
This chart, included in our AI infrastructure market deck, shows annual VC investment in AI infrastructure startups
Have packaging and memory become the real chip bottlenecks?
The tightest chip-side constraint currently sits in advanced packaging and high-bandwidth memory.
TSMC’s latest earnings call gave unusually direct evidence. Chief executive C.C. Wei said the company’s packaging capacity was so tight that it was limiting customers’ growth. TSMC is responding with a capital budget of $60 billion to $64 billion and plans for 13 leading-edge and advanced-packaging facilities in Taiwan over the coming years.
High-bandwidth memory is under similar pressure. The IEA reports that an HBM shortage developed recently and could last through at least the end of 2027. Memory matters because accelerators must move enormous quantities of data quickly. More arithmetic capacity achieves little when the processor spends too much time waiting for information.
These constraints can delay complete AI systems even when enough logic chips exist. They deserve more weight than they received during the early GPU discussion, when almost every shortage was described as a shortage of NVIDIA chips.
Manufacturers have a clear route to expansion: add packaging lines, improve yields, increase memory output and invest in new plants. TSMC, SK hynix, Micron and their suppliers are already spending heavily. Grid capacity is spread across utilities, regulators, equipment suppliers and local authorities, making it harder to fix through one coordinated investment push.
Is electricity now the biggest physical AI bottleneck?
Electricity now sets the physical ceiling for AI because dependable power cannot reach new data centers fast enough.
The latest IEA data show global data-center electricity use growing 17% in one year, while consumption at AI-focused facilities jumped 50%. Its central forecast rises from 485 terawatt-hours to roughly 950 terawatt-hours by 2030. AI-focused consumption grows even faster and roughly triples over that period.
The United States faces the sharpest concentration. Lawrence Berkeley National Laboratory’s newest bottom-up estimate puts U.S. data-center use at 649 terawatt-hours in 2030, equal to 11.8% of national electricity consumption. Its uncertainty range runs from 521 to 843 terawatt-hours, so even the low case represents a major change in the power system.
Global electricity supply is large enough in the abstract. The difficulty is delivering hundreds of megawatts, continuously and with adequate backup, to a specific campus on a commercial deadline. AI developers can buy chips from abroad and move software between regions. They cannot transmit unlimited electricity through a congested local grid.
That makes “speed to power” the useful measure. A site with land, servers and financing has little value until it can be energized. That final step now decides which announced AI projects become real operating facilities.
If you want more recent data on this point, please see our latest AI infrastructure market report.
This chart, included in our AI infrastructure market deck, shows why CoreWeave is winning in AI infrastructure
Is getting a grid connection harder than generating more electricity?
In many major data-center markets, getting connected to the grid is harder than finding new generation.
Large AI campuses often request 300 to 1,000 megawatts with expected lead times of only one to three years, according to the U.S. Department of Energy. Utilities must then assess transmission capacity, substations, reliability, backup arrangements and who pays for upgrades. Those decisions involve far more than signing a power purchase agreement.
The regulatory response shows how serious the blockage has become. FERC recently ordered all six U.S. regional grid operators to justify or reform their rules for connecting data centers and other large loads. The Department of Energy’s latest national transmission study also identifies data centers as a major source of urgent new infrastructure needs.
Developers have tried to bypass the queue with power plants beside the data center. That route brings its own delays. The IEA estimates that dependable on-site gas generation may require 30% to 70% more installed capacity than the center’s actual demand, largely because the plant must survive maintenance and equipment failures. Gas-turbine supply is tight as well.
New generation, transmission, distribution equipment, permits and the data center itself must arrive in the right order. One missing approval or transformer can hold up the entire campus.
Are transformers, turbines and cooling the hidden AI choke points?
Transformers, turbines, switchgear and cooling now delay AI projects that already have chips and financing.
AI racks are becoming much harder to serve. The IEA calculates that server power density increased elevenfold between 2020 and 2025 and could rise another fourfold by 2027. One advanced rack may then draw as much peak power as 65 households while releasing heat comparable to 30 domestic gas boilers.
That density changes the building. Operators need higher-voltage distribution, direct-to-chip liquid cooling, larger pumps, heavier floors, new heat exchangers and batteries capable of smoothing rapid load swings. An older data center with available floor space may still be unsuitable for the newest hardware.
The supply chain has tightened at the same time. Global gas-turbine orders rose 70% in 2025, according to the IEA, while transformers and power electronics face rising demand from data centers, grids, factories and electrification. These are specialized products with limited manufacturing capacity and long delivery schedules.
Cooling is usually more manageable because the developer controls much of the design inside the site. Transformers, switchgear and grid equipment depend on external suppliers and utility standards. We see them as parts of the power bottleneck rather than separate candidates for the overall title.
This chart, included in our AI infrastructure market deck, shows annual funding in AI infrastructure startups
Can efficiency gains solve the AI power problem?
AI is becoming dramatically more efficient, yet total electricity demand keeps climbing faster.
The progress per task is remarkable. The IEA estimates that energy use for an individual AI task has been falling by at least an order of magnitude each year. Epoch AI has also measured inference-price declines ranging from roughly ninefold to 900-fold annually across different performance targets.
Cheaper intelligence encourages heavier use. Simple text requests are being joined by video generation, extended reasoning and agents that call models repeatedly while searching, coding, checking and correcting. The IEA estimates that some of these tasks can consume hundreds or thousands of times more energy than a basic text response.
We can see the rebound in the aggregate data. Electricity use at AI-focused data centers rose 50% even while the energy needed for a comparable task fell sharply. Major model providers also reported roughly three times as many active users and five times as much revenue over the latest measured year.
Efficiency remains one of the best tools available. It lowers costs, improves throughput per megawatt and lets smaller models handle routine work. For now, it is expanding the market faster than it is shrinking total demand.
Could weak demand or scarce capital become the bigger bottleneck?
Weak demand and scarce capital may kill weaker AI projects, but physical capacity still sets the wider limit.
The amount of money entering AI infrastructure remains extreme. The IEA estimates that capital spending by five large technology companies exceeded $400 billion in 2025 and could increase another 75% in 2026. NVIDIA’s data-center revenue nearly doubled in its latest quarter, while TSMC raised its annual capital budget because customer demand kept strengthening.
Those figures do not prove that every proposed data center makes sense. Project pipelines can contain duplicate connection requests, unrealistic completion dates and campuses without firm tenants. Berkeley Lab has warned that load forecasts may be biased upward when developers submit several possible sites for one eventual project.
We should expect selective failure rather than a uniform crash. Projects with uncertain customers, expensive financing or poor power access will be postponed. Facilities with secured energy, strong tenants and useful locations should remain valuable because demand for tokens and computing capacity is still rising quickly. Epoch AI’s latest work even finds that inference demand appears to be growing faster than global supply.
Capital is becoming more discriminating, which should reduce some overbuilding. It cannot shorten every grid study, manufacture a transformer instantly or guarantee that a local community accepts a new power project. Physical execution still sets the pace.
If you want more recent data on this point, please see our latest AI infrastructure market report.
This chart, included in our AI infrastructure market deck, compares the main business model options for AI cloud infrastructure providers
Is high-quality training data holding frontier AI back?
Training data is tightening, but it has not slowed frontier progress enough to become the biggest bottleneck.
Epoch AI estimates that the effective stock of quality-adjusted public human text is around 300 trillion tokens, with a wide uncertainty range. On previous dataset-growth trends, leading models could use most of that stock between 2026 and 2032. Open-weight models are already being trained with far more data per active parameter than the ratios common only a few years ago.
Labs have several escape routes. They can license private collections, train on images, audio and video, generate synthetic examples, use simulations and shift more effort into post-training and reinforcement learning. Each route has limits, especially when errors in synthetic data reinforce themselves, yet together they prevent a simple “the internet is finished” ceiling.
Current performance is the strongest counterevidence. Stanford’s latest AI Index found rapid gains in coding, mathematics, science and computer-use tasks. Capability has accelerated even as concerns about public-text exhaustion have grown.
Data scarcity could become decisive for a particular training method. Today, laboratories are changing the training recipe faster than the constraint is stopping them.
Is reliability the biggest bottleneck for AI agents?
Reliability now sets the technical limit for autonomous AI agents.
Stanford reports that performance on OSWorld, a benchmark for operating-system tasks, rose from about 12% to 66.3% in one year. That is a major advance, yet it still implies failure in roughly one out of three structured attempts. Real workplaces add ambiguous instructions, changing software, permissions, missing context and consequences that a benchmark does not fully capture.
METR reaches a similar conclusion from longer software tasks. Its time-horizon research measures the task difficulty at which an agent reaches a chosen success rate, and the gap between 50% and 80% reliability remains important. METR also warns that estimates above 16 hours of equivalent human work are still unreliable with its current task set.
This explains why agent demos can look much better than production systems. A company may tolerate occasional mistakes in drafting or search. It cannot tolerate the same rate when an agent changes customer records, transfers money, deploys code or makes regulated decisions.
Better evaluations, constrained tools, approvals and recovery systems can reduce the risk. They also add cost and human supervision. For autonomous agents, reliability clearly deserves the top spot; for AI infrastructure as a whole, it sits farther downstream than power.
This chart, featured in our AI infrastructure market deck, shows the share of revenue generated by each customer segment in the AI infrastructure market
What is stopping companies from getting real value from AI?
Most companies are blocked by messy internal data and workflows that were never redesigned for AI.
Adoption is already broad. Stanford’s latest AI Index puts organizational AI use at 88%, with generative AI present in at least one business function at 70% of organizations. Agent deployment, however, remained in the single digits across nearly every business function. Buying access has moved much faster than redesigning work.
McKinsey found the same gap from another angle. In its survey of 25 organizational practices, workflow redesign had the largest effect on whether generative AI produced a measurable impact on earnings. Only 21% of respondents said their organizations had fundamentally redesigned at least some workflows.
The reason becomes clear inside a real process. An AI system may need the current customer record, contract terms, inventory, approval limits and previous decisions. Those facts often live in separate tools with inconsistent formats and access rules. The model can write a convincing answer while still using the wrong version of the truth.
Companies that keep AI beside the workflow usually get modest time savings. Larger gains require changing who does each step, what the system can access, where humans approve decisions and how mistakes are caught. A better frontier model helps at the margin; clean data and redesigned operations determine whether the company captures the value.
Does the biggest AI bottleneck depend on who you are?
Different players hit different AI bottlenecks, while power delivery creates the widest drag across the system.
A U.S. hyperscaler can raise capital and secure chips, then wait for grid capacity. A Chinese frontier lab faces tighter access to advanced accelerators because of export controls. A hospital may already have cloud access while struggling with privacy rules, old software and unreliable data. A small business often needs neither a new model nor a private cluster; it needs one workflow worth changing.
Geography deepens the difference. Stanford counts 5,427 data centers in the United States, more than ten times the number in any other country. Many emerging markets have fewer cloud regions, less reliable electricity and higher connectivity costs. Their AI constraint starts much earlier in the infrastructure stack.
| Player | Main bottleneck | Why it dominates |
|---|---|---|
| Frontier AI laboratory | Dense, power-ready compute | Training and research require concentrated capacity |
| Hyperscaler or data-center developer | Grid connection and electrical equipment | Hardware cannot operate before the site is energized |
| Chinese frontier developer | Access to leading chips and manufacturing tools | Trade restrictions narrow available hardware |
| Large enterprise | Data integration, governance and workflow redesign | Existing models are already capable enough for many tasks |
| AI-agent product company | Reliability and supervision cost | Failure rates block autonomous use |
| Emerging-market user | Affordable compute, electricity and connectivity | Basic access remains uneven |
This chart, included in our AI infrastructure market deck, shows how GPU cloud infrastructure technology has evolved over time
Which AI bottleneck is hardest to remove quickly?
No major AI bottleneck is slower to clear than power delivery.
Software can improve within months. Inference prices can fall by an order of magnitude in a year. A company can also narrow an agent’s permissions or move a task to a smaller model. These fixes may be incomplete, but teams can test and deploy them rapidly.
Semiconductor expansion takes longer, although the chain is relatively concentrated. TSMC, memory producers and equipment suppliers can invest against visible orders. TSMC says leading-edge technology and capacity now require five to seven years to develop and ramp, yet suppliers still control most of that execution.
Power projects combine utility planning, generation, transmission, substations, equipment, land, permits, local politics and cost allocation. Progress at six stages can be erased by a delay at the seventh. The recent intervention by FERC across every U.S. regional grid operator shows that ordinary commercial negotiation has not been enough.
| Bottleneck | Typical speed of improvement | Number of parties involved | Difficulty of a fast fix |
|---|---|---|---|
| Model software and inference efficiency | Months | A few development teams | Moderate |
| Agent reliability | Months to several years | Developers and customers | High |
| Chips, memory and packaging | Several quarters to years | Concentrated manufacturers | High |
| Enterprise data and workflows | Years within each organization | Management, staff and vendors | High but local |
| Power generation and grid delivery | Several years or longer | Utilities, regulators, suppliers and communities | Very high |
If you want more recent data on this point, please see our latest AI infrastructure market report.
So what is the biggest AI bottleneck today?
Power delivery is the biggest AI bottleneck today.
The precise constraint is access to dependable electricity through a grid, substation and equipment stack that can energize very large computing clusters on the timeline developers want. Chips remain tight, particularly in packaging and HBM, yet the scale of expansion is visible in NVIDIA’s $75.2 billion quarterly data-center business and TSMC’s expanded investment plans. Model capability is also moving quickly rather than stalling.
Power has become the stronger answer because demand is concentrated, local and difficult to substitute. As we saw above, the IEA recorded a 50% jump in electricity use at AI-focused data centers in one year. Berkeley Lab’s newest U.S. estimate reaches 649 terawatt-hours by 2030. Regulators are now changing connection rules specifically because data-center requests are colliding with the slower pace of grid expansion.
The title has different answers at narrower levels. Reliability blocks autonomous agents. Workflow redesign and internal data block company returns. Packaging and memory remain the sharpest semiconductor constraints. None of those has the same reach and rigidity as power delivery across the full AI system.
For now, the industry can design models, raise money and order hardware faster than it can create energized sites. That mismatch is organizing the next phase of the AI race.
In our AI infrastructure market deck, we identify pain points entrepreneurs should prioritize
OUR METHODOLOGY
This analysis asks which constraint currently blocks the most valuable AI progress across the widest part of the industry. We compare power delivery, chips, memory and packaging, training data, agent reliability, enterprise integration, capital and local infrastructure through the same four tests: reach, substitutability, time to add supply and the amount of progress that would be unlocked if the constraint eased.
We separate system-wide bottlenecks from local ones. Reliability can be the decisive constraint for an AI-agent company, workflow redesign can dominate inside a large enterprise, and export controls can make advanced chips the binding limit for a Chinese frontier laboratory. The overall answer goes to the constraint that cuts across the largest share of the stack.
Recent evidence receives more weight than older narratives because the AI supply chain is moving quickly. The GPU shortage that dominated the first generative-AI wave remains relevant, but it is tested against newer evidence on accelerator shipments, packaging capacity, high-bandwidth memory, data-center electricity use, grid queues, equipment lead times and model performance.
We prioritize first-hand disclosures, official datasets, technical benchmarks, regulatory actions and primary industry research. Key sources include the International Energy Agency’s Energy and AI work, the Stanford AI Index, NVIDIA’s quarterly results, TSMC earnings calls and investment disclosures, and Lawrence Berkeley National Laboratory’s data-center energy research.
For grid and infrastructure constraints, we also use the U.S. Department of Energy’s National Transmission Planning Study and Federal Energy Regulatory Commission actions on large-load connections. For software and deployment constraints, we use Epoch AI research, METR’s time-horizon work, and McKinsey’s State of AI research.
No single forecast or company statement decides the conclusion. We look for convergence across independent sources and give more weight to constraints that are already delaying real projects, require several years and many outside parties to resolve, and cannot be bypassed simply by changing software, suppliers or locations.
This chart, included in our AI infrastructure market deck, shows the share of revenue by region across Europe, Asia, North America, Africa, and South America in the AI infrastructure market
Related blog posts
- Is power replacing chips as AI’s bottleneck?
- Which startups control the power behind AI today?
- Which startup is redefining how data centers are powered?
