AI infrastructure: what are the biggest unsolved problems?
Every funding round, every investor, and who is really making money from AI infrastructure, in one document. Have a look ↓
Summary
Electricity access is the biggest unsolved problem in AI infrastructure today. But securing power is only half the challenge: the industry also needs to prove that increasingly expensive data centers can generate attractive returns before their computing hardware becomes outdated.
The scale of electricity demand is already testing what grids can deliver. The International Energy Agency expects global data center consumption to increase from approximately 485 TWh in 2025 to 950 TWh by 2030, with grid constraints potentially delaying around 20% of planned capacity.
Location is becoming almost as important as computing technology. In Northern Virginia, grid connection queues can stretch to seven years, giving operators with existing power agreements an advantage that even billions of dollars in new financing cannot immediately replicate.
Chip shortages are more complicated than simply producing enough GPUs. NVIDIA, Google, AMD and Amazon consumed an estimated 90% of global high-bandwidth memory and advanced CoWoS packaging capacity in 2025, even though they represented only around 12% of advanced logic production.
NVIDIA's dominance extends well beyond the processor. Its networking systems, CUDA software ecosystem and integrated rack-scale infrastructure make switching suppliers expensive, even as Google, Amazon and AMD offer increasingly capable alternatives.
Efficiency improvements are not reducing total electricity demand. AI hardware has become dramatically more efficient, but cheaper computing encourages larger models, longer reasoning processes and more applications, absorbing much of the savings.
Inference economics are changing faster than infrastructure economics. Basic model queries have become hundreds of times cheaper, while coding agents, long-context applications and complex reasoning workloads can still consume substantial computing resources per completed task.
Networking and cooling are emerging as important constraints inside data centers. Modern systems need thousands of GPUs to communicate efficiently, while increasingly powerful racks require liquid cooling and electrical infrastructure that many existing facilities were never designed to support.
The financial numbers reveal a less comfortable picture than revenue growth alone suggests. CoreWeave generated $5.1 billion in revenue in 2025 but reported a $1.17 billion net loss and $1.23 billion in net interest expense, demonstrating how expensive it can be to build capacity ahead of demand.
Our conclusion is that electricity, financing and semiconductor supply are the three most serious constraints. Cooling, networking and software optimization have workable technical solutions, although deploying them at scale remains difficult. The larger uncertainty is whether infrastructure providers can earn consistent profits across several generations of rapidly improving hardware.
Written by the New Market Pitch teamWe track new markets so founders and investors can move faster. About us
Why is AI infrastructure struggling to keep up with demand?
AI infrastructure is growing at an incredible pace, but electricity, chip manufacturing and the cost of building data centers are already making expansion harder than many companies expected.
The amounts being invested are extraordinary. NVIDIA reported $215.9 billion in fiscal 2026 revenue, up 65% from the previous year. Alphabet disclosed $91.4 billion in capital expenditures for 2025, then raised its 2026 investment outlook to $180-190 billion in a June investor presentation. CoreWeave, one of the largest specialized AI cloud providers, increased annual revenue from $1.9 billion to $5.1 billion in 2025.
Yet spending more money doesn't necessarily mean getting more computing capacity online faster. According to the International Energy Agency's updated Energy and AI analysis, global data center electricity consumption is expected to rise from 485 terawatt-hours in 2025 to approximately 950 TWh by 2030. The agency also warns that power infrastructure and equipment shortages are preventing some projects from moving forward as quickly as planned.
Three problems stand out. AI companies need computing capacity faster than electricity grids and semiconductor manufacturers can supply it. Better chips are making AI cheaper to run, but increasingly demanding applications are consuming much of those gains. Meanwhile, infrastructure providers are committing billions of dollars to equipment whose future rental prices and useful economic lives remain difficult to predict.
Is electricity now the biggest problem for AI infrastructure, and can grids keep up?
Electricity is currently the biggest physical bottleneck for AI infrastructure, and power grids are unlikely to expand fast enough to accommodate every planned data center.
The International Energy Agency estimated that data centers consumed approximately 415 TWh globally in 2024, around 1.5% of worldwide electricity consumption. Its updated outlook expects demand to reach roughly 950 TWh in 2030. That would represent an increase of about 535 TWh in six years, exceeding the annual electricity consumption of many developed countries.
Lawrence Berkeley National Laboratory's 2024 study estimated that U.S. data centers consumed 176 TWh in 2023, compared with 58 TWh in 2014. Consumption therefore tripled over nine years. The laboratory projected that demand could reach between 325 and 580 TWh by 2028. Even the lower estimate implies an 85% increase in five years.
The problem is especially severe where data centers are concentrated. The IEA found that nearly half of U.S. data center capacity sits within five regional clusters. In Northern Virginia, one of the world's largest data center markets, grid connection queues can stretch to seven years, according to the agency's World Energy Outlook 2025.
The IEA has also warned that approximately 20% of planned global data center capacity through 2030 could face delays because of grid constraints.
Microsoft, Amazon and Google are responding through long-term power agreements, investments in energy infrastructure and more geographically distributed facilities. Some developers are also considering dedicated power generation to avoid lengthy grid connection queues.
For now, access to electricity is becoming a competitive advantage in its own right. Companies with secured power connections can often move faster than competitors with equivalent access to capital and computing equipment.
| Electricity constraint | Evidence | Why it matters |
|---|---|---|
| U.S. data center consumption in 2014 | 58 TWh | Historical baseline |
| U.S. consumption in 2023 | 176 TWh | More than tripled in nine years |
| U.S. demand projected for 2028 | 325-580 TWh | Potentially 85-230% above 2023 |
| Global consumption in 2024 | Approximately 415 TWh | About 1.5% of global electricity use |
| Global consumption in 2025 | Approximately 485 TWh | Continued expansion |
| Global demand projected for 2030 | Approximately 950 TWh | Nearly double the 2025 level |
| U.S. regional concentration | Nearly half of capacity in five clusters | Local grid congestion |
| Northern Virginia connection queues | Up to seven years | Long deployment delays |
| Planned global capacity potentially delayed | Approximately 20% | Grid constraints threaten expansion |
If you want more recent data on this point, see the latest figures in our AI Infrastructure report ↓.
We keep an up-to-date report on AI infrastructure. Have a look ↓
Sample pages from an earlier edition. The report has been updated with new data since.
Why can't chipmakers produce enough AI GPUs?
Producing enough advanced AI GPUs remains difficult because high-bandwidth memory and advanced packaging capacity are heavily constrained, even when manufacturers can produce the underlying silicon chips.
A particularly revealing analysis came from Epoch AI in March 2026. The research organization estimated that NVIDIA, Google, AMD and Amazon collectively consumed approximately 90% of global CoWoS advanced packaging capacity and high-bandwidth memory supply in 2025.
Yet those same four companies accounted for only around 12% of advanced logic die production.
The difference tells us where the shortage was most severe. Manufacturing the processor itself was only one part of the problem. AI accelerators also require specialized memory and packaging technologies that connect powerful processors to the data they need.
NVIDIA depends heavily on TSMC's advanced manufacturing and packaging capabilities. Memory suppliers SK Hynix, Samsung and Micron must expand production alongside chipmakers, creating a supply chain in which several companies need to increase capacity simultaneously.
Epoch AI's estimates also show how concentrated demand has become. NVIDIA alone accounted for roughly 69% of high-bandwidth memory consumption and 60% of CoWoS capacity in 2025.
These are estimates rather than official production allocations, but the imbalance between advanced logic and packaging is substantial enough to reveal the main constraint.
Is NVIDIA making AI infrastructure too dependent on one company?
NVIDIA has become the most influential company in AI infrastructure, and its advantage now extends far beyond GPUs into networking, software and complete computing systems.
Its fiscal 2026 financial results show just how broad that position has become. NVIDIA generated $215.9 billion in annual revenue, while data center networking revenue increased 142% compared with 59% growth in data center computing revenue.
NVIDIA is selling more of the systems that connect AI processors, including NVLink-based infrastructure, Ethernet and InfiniBand networking. Customers increasingly purchase integrated computing platforms rather than individual chips.
Google's TPUs, Amazon's Trainium processors and AMD's Instinct accelerators offer alternatives. However, switching hardware also means adapting software, optimizing models and training engineers to work with different systems.
CUDA remains particularly important because so much existing AI software has been developed and optimized around NVIDIA's ecosystem.
There is real competition in AI accelerators, especially among hyperscalers developing chips for their own workloads. But smaller companies face a tougher choice: pay for an established NVIDIA-based system or spend additional engineering resources moving to another architecture. For many, staying with NVIDIA is simply the easier decision.
If you want more recent data on this point, see the latest figures in our AI Infrastructure report ↓.
These pages are excerpts from our report. See the rest ↓
Sample pages from an earlier edition. The report has been updated with new data since.
Why is connecting AI GPUs becoming such a big problem?
AI networking is becoming a serious performance bottleneck because large models need thousands of processors to exchange information quickly, and slow communication can leave expensive GPUs waiting for data.
The rapid expansion of NVIDIA's networking business gives us a useful indication of how important this has become. In fiscal 2026, networking revenue grew 142% year over year, much faster than the company's data center compute revenue.
Much of that growth came from NVLink compute fabrics for its GB200 and GB300 systems, alongside Ethernet and InfiniBand products.
During large training runs, thousands of GPUs may need to share information repeatedly. If the network takes too long to move that information, faster chips cannot deliver their full potential.
The same problem appears in increasingly complex inference systems. When models are distributed across several servers, communication delays can increase the time users wait for responses.
NVIDIA, Broadcom and Arista are among the companies developing networking technologies for these environments, although they compete across different parts of the infrastructure.
Why do AI data centers need more electricity if chips keep getting better?
AI data centers are using more electricity because demand for computing is growing faster than improvements in chip efficiency can reduce consumption.
Stanford's 2025 AI Index reported that machine-learning hardware energy efficiency improved by approximately 40% annually over the period it examined.
Google offers a concrete example. Its Ironwood TPU architecture was designed to deliver nearly 30 times better power efficiency than the company's first Cloud TPU from 2018.
These improvements are enormous. They allow AI models to perform computations at costs and energy levels that would have been impractical with older hardware.
But total demand is moving in the opposite direction. The IEA estimates that global data center electricity consumption grew by approximately 12% annually from 2017 onward.
Google's own environmental disclosures illustrate this tension. Its 2025 report showed that data center electricity consumption increased 27% in 2024, even though the company reduced associated energy emissions by 12%.
AI infrastructure efficiency is improving rapidly, but there is currently little evidence that those improvements will reduce total electricity consumption over the next several years. Cheaper computing is giving companies more reasons to use it.
Prefer verified numbers to headlines? Our report is here ↓
Sample pages from an earlier edition. The report has been updated with new data since.
Can AI data centers handle the heat from increasingly powerful GPUs?
Cooling is becoming a major problem for AI data centers because new GPU systems pack much more computing power into individual racks, making many older facilities difficult to upgrade.
NVIDIA's Blackwell rack-scale systems illustrate how quickly hardware requirements are changing. The GB200 NVL72 integrates 72 Blackwell GPUs into a liquid-cooled rack-scale system designed for demanding AI workloads.
These systems generate enough heat that traditional air cooling becomes impractical at the highest rack densities. Operators increasingly need direct liquid cooling, specialized heat exchangers and redesigned facilities.
The challenge is particularly serious for companies converting existing data centers. Buildings originally designed for lower-density enterprise servers may need new plumbing, electrical distribution and heat-management systems before they can support modern AI equipment.
Newly constructed facilities can incorporate these systems from the beginning, although doing so adds engineering complexity and upfront expense.
There are also geographic differences. Cooling a data center in a hot climate creates different requirements from operating one in a cooler region, especially when water availability is limited.
Why is AI inference still expensive when model prices have collapsed?
AI inference is becoming much cheaper per token, but complex applications can still be expensive because they require many model calls, long reasoning processes and substantial computing resources.
Stanford's 2025 AI Index found that the cost of querying a model performing at GPT-3.5's MMLU level fell from approximately $20 per million tokens in November 2022 to just $0.07 by October 2024.
That is a reduction of more than 280 times in less than two years.
However, the comparison measures a specific level of model performance. It doesn't mean that every modern AI task has become 280 times cheaper.
Consider an AI application classifying short customer messages. It may only require a compact model and a small number of tokens. A coding agent, meanwhile, might inspect a large repository, generate code, run tests, identify problems and repeat the process several times before completing one request.
Even with inexpensive tokens, the coding agent can consume substantially more computing resources.
Long-context models introduce another challenge because processing large amounts of information requires memory and attention operations. Applications demanding immediate responses can also cost more than batch workloads that tolerate waiting.
Want to know who is really ahead? Our report is here ↓
Sample pages from an earlier edition. The report has been updated with new data since.
Why is running AI models on thousands of GPUs still so difficult?
Running AI models efficiently across large GPU clusters remains difficult because training, chatbots and AI agents all use computing resources differently, making it hard to keep expensive hardware consistently busy.
A training job might require thousands of GPUs working together for an extended period. An interactive chatbot needs spare capacity to respond quickly when requests arrive. A coding agent alternates between generating text, calling software tools and waiting for results.
Each workload benefits from different scheduling and resource-management strategies.
Software such as vLLM helps by batching inference requests, managing GPU memory and improving throughput. Quantization and caching can further reduce computing requirements.
Yet maximizing GPU utilization can increase response times if too many requests are grouped together. Reserving capacity for faster responses creates the opposite problem: some hardware remains idle.
MLCommons reflected this shift in its April 2026 MLPerf Inference v6.0 release, which introduced or updated five of eleven data center tests, including more demanding reasoning workloads.
These benchmarks provide useful comparisons, but real-world performance still depends heavily on traffic patterns and application requirements.
| AI workload | Main difficulty | Useful performance measure |
|---|---|---|
| Large-model training | Coordinating thousands of GPUs | Time needed to complete training |
| Interactive chatbots | Handling unpredictable traffic | Response latency |
| Reasoning models | Long and variable generation | Cost per successful answer |
| AI coding agents | Repeated model and tool calls | Cost per completed coding task |
| Document processing | Processing large batches cheaply | Documents processed per dollar |
| Long-context models | High memory requirements | Memory use and response time |
| Multimodal AI | Combining text, images and audio | End-to-end processing time |
| Real-time voice assistants | Maintaining fluid conversations | Delay between speech and response |
Are AI data centers becoming too expensive to finance, and can GPU clouds stay profitable?
Financing AI data centers is becoming one of the industry's biggest problems, particularly for specialized cloud providers that cannot rely on the enormous cash flows generated by Microsoft, Amazon and Google.
CoreWeave's financial results show why.
The company increased annual revenue by 168% in 2025, reaching $5.1 billion. Yet it still reported a $1.17 billion net loss and approximately $1.23 billion in net interest expense.
Interest expense alone represented roughly 24% of annual revenue. CoreWeave also moved from a $324 million operating profit in 2024 to a $46 million operating loss in 2025, despite revenue nearly tripling.
CoreWeave reported that its contracted revenue backlog increased to $66.8 billion, up from approximately $15 billion at the beginning of the year. That provides considerable visibility into future demand, although contracted revenue must still be delivered and recognized over time.
Alphabet operates under very different conditions. In its June 2026 investor presentation, the company projected $180-190 billion in capital spending for the year, almost six times its 2022 spending. It also reported that Google Cloud's first-quarter operating margin had reached 33%.
Both companies are investing heavily in AI infrastructure, but Alphabet can support those investments through an established portfolio of highly profitable businesses.
Specialized providers often need to borrow heavily or raise equity to fund expansion, creating greater exposure to interest rates, customer concentration and changes in computing prices.
Meanwhile, Stanford's AI Index shows that machine-learning hardware price-performance has historically improved rapidly, with hardware costs declining approximately 30% annually over the period studied.
That creates pressure on providers renting older GPUs. Newer hardware can deliver more useful computing per dollar, encouraging customers to migrate when contract terms allow.
The industry's financing problem will become especially important when today's equipment needs replacing. Data center buildings can remain useful for decades, while the accelerators inside them may become economically outdated much sooner.
We still lack enough long-term operating history to know which specialized AI infrastructure business models will remain consistently profitable across multiple hardware generations. The revenue growth is real. So is the financial risk.
If you want more recent data on this point, see the latest figures in our AI Infrastructure report ↓.
Is the AI infrastructure supply chain too concentrated?
AI infrastructure currently depends too heavily on a small number of companies across chip manufacturing, memory, packaging and cloud services, making supply disruptions potentially expensive for the entire industry.
The clearest example is advanced semiconductor production.
According to Epoch AI's March 2026 analysis, NVIDIA, Google, AMD and Amazon accounted for approximately 90% of high-bandwidth memory and advanced CoWoS packaging consumption in 2025.
Within that group, NVIDIA was by far the largest customer, representing an estimated 69% of HBM consumption.
Supply is also concentrated. TSMC manufactures many leading AI processors, while SK Hynix, Samsung and Micron dominate the high-bandwidth memory market.
This means competing cloud companies can still depend on the same manufacturers. A packaging capacity shortage at TSMC or manufacturing problems at a major memory supplier could affect several cloud providers simultaneously.
At the software level, NVIDIA's CUDA ecosystem creates additional dependence, while Amazon Web Services, Microsoft Azure and Google Cloud control much of the global cloud infrastructure used by businesses.
Using several cloud providers helps reduce some risks, but it cannot fully protect a customer from problems affecting shared upstream suppliers.
Are AI data center outages becoming more dangerous?
AI data center outages can already be extremely expensive, although the available evidence shows that overall data center reliability has generally improved rather than deteriorated.
Uptime Institute's 2025 Annual Outage Analysis found that outage frequency and severity had declined for four consecutive years.
Nevertheless, 54% of surveyed respondents who experienced a significant, serious or severe outage reported that their most recent incident cost more than $100,000. Roughly one in five reported losses exceeding $1 million.
Power remained the leading cause of impactful outages, while IT and networking problems accounted for 23% of incidents in 2024.
These findings are especially relevant to AI systems because modern GPU clusters rely on tightly coordinated computing, storage and networking equipment.
A hardware or networking failure during a large training run can interrupt work across thousands of accelerators. Depending on the system's fault tolerance and checkpointing arrangements, recovering from the failure may require additional computing time.
Uptime Institute's 2025 survey also found that 87% of respondents who had experienced a major outage believed better management or processes could have prevented it.
We made a report out of all this. Take a look ↓
Sample pages from an earlier edition. The report has been updated with new data since.
Can AI data centers reduce their environmental impact?
AI data centers are becoming more efficient, but reducing their total environmental footprint remains difficult because electricity consumption is rising and some cooling systems require large amounts of water.
Google's 2025 environmental report offers an interesting example. The company disclosed that its data center electricity consumption increased 27% in 2024, while associated energy emissions declined 12%.
The reduction in emissions came partly from additional clean energy projects. It shows that companies can improve the environmental impact of their electricity consumption even when the amount consumed increases.
Water is more complicated.
Google reported replenishing approximately 4.5 billion gallons of water in 2024, equivalent to 64% of its freshwater consumption. That was a substantial improvement from 18% in 2023.
However, replenishment programs do not necessarily replace water in the same location or at the same time it was consumed. Local water availability therefore remains an important consideration when choosing where to build data centers.
Cooling technology also involves trade-offs. Evaporative systems can reduce electricity requirements under certain conditions but consume water. Other cooling approaches may use less water while requiring different equipment and operating costs.
Are U.S. chip export restrictions hurting global AI infrastructure?
U.S. chip export restrictions are making global AI infrastructure harder to plan because companies cannot always sell, purchase or deploy the same advanced processors in different countries.
NVIDIA's financial disclosures provide concrete evidence of the cost.
In April 2025, U.S. export licensing requirements affecting its H20 processors contributed to a $4.5 billion charge related to inventory and purchase commitments.
Subsequent restrictions involving H200 exports also affected the company's planning and contributed to additional inventory-related charges disclosed during fiscal 2027.
These costs came from changes in export policy rather than a sudden disappearance of computing demand.
The restrictions are encouraging China to invest further in domestic AI accelerators, while creating uncertainty for suppliers that previously served customers across several major markets.
For AI infrastructure companies, this complicates decisions about equipment purchases, supply contracts and long-term capacity planning.
Different countries are also introducing rules governing where sensitive data can be stored and processed, making international cloud deployments more complicated.
If this is useful, the full report goes much further. Have a look ↓
Sample pages from an earlier edition. The report has been updated with new data since.
Can smaller AI cloud startups really compete with Microsoft, Amazon and Google?
Smaller AI cloud startups can compete with Microsoft, Amazon and Google in specialized computing services, but the financial gap between them makes direct competition across the entire cloud market extremely difficult.
CoreWeave shows that there is genuine demand for specialized providers. Its revenue grew from $229 million in 2023 to $1.9 billion in 2024 and $5.1 billion in 2025.
That represents more than 22-fold growth in two years.
The company also ended 2025 with $66.8 billion in contracted revenue backlog, suggesting that large customers are willing to commit substantial spending to specialized AI infrastructure.
Other companies, including Lambda, Crusoe and Nebius, are pursuing opportunities in GPU cloud services, AI-focused data centers and dedicated computing infrastructure.
However, Microsoft, Amazon and Google have advantages that extend beyond access to chips. They operate global networks, sell many other cloud products, maintain established enterprise relationships and can finance expansion through large existing businesses.
Google's June 2026 investor presentation reported $20 billion in first-quarter Google Cloud revenue and a 33% operating margin. That profitability provides resources that a much smaller infrastructure provider cannot easily match.
Specialized AI clouds can still win customers by offering dedicated capacity, better support or attractive economics for particular workloads.
Could companies end up building too many AI data centers?
AI infrastructure could become overbuilt in certain regions or hardware categories even if overall demand for artificial intelligence continues growing quickly.
The risk comes from the gap between how long infrastructure takes to build and how quickly AI technology changes.
Alphabet's expected $180-190 billion in capital expenditures for 2026 shows how aggressively major companies are investing. NVIDIA's rapid revenue growth and CoreWeave's large contracted backlog also suggest substantial demand for computing resources.
But these figures describe different stages of the same market. NVIDIA sells computing systems, cloud providers purchase those systems, and customers pay to use the resulting infrastructure.
Adding all those financial figures together would count some of the same economic activity several times.
A data center can also take years to develop, while the GPUs it is designed to accommodate may change considerably before the building opens.
Faster processors, more efficient inference software or unexpected changes in customer demand could make some planned facilities less attractive than initially expected.
Location matters too. A region could have excess computing capacity while another suffers from power shortages and long connection queues.
The strongest evidence currently supports continued growth in AI infrastructure demand. There is much less public evidence about the long-term utilization and profitability of every project being announced. And a growing market can still leave investors with expensive, underused assets.
If you want more recent data on this point, see the latest figures in our AI Infrastructure report ↓.
Want the whole picture? Check our report ↓
Sample pages from an earlier edition. The report has been updated with new data since.
What are the biggest unsolved problems in AI infrastructure today?
The biggest unsolved problems in AI infrastructure today are electricity access, the economics of building and operating data centers, constrained chip supply and the difficulty of running increasingly complex AI workloads efficiently.
Electricity ranks first because it can completely prevent new computing capacity from opening. The IEA's warning that approximately one-fifth of planned data center capacity could face grid-related delays makes the scale of this problem clear.
Financing comes next. As we saw earlier, CoreWeave's 2025 results combined extraordinary revenue growth with $1.23 billion in net interest expense. Building capacity quickly can create enormous financial obligations long before the equipment generates sufficient returns.
Advanced chip manufacturing remains another significant constraint. Epoch AI estimated that the four largest AI chip designers consumed nearly 90% of high-bandwidth memory and advanced packaging supply in 2025, leaving relatively limited capacity outside that group.
Inference efficiency, networking and cooling are also increasingly important. Better GPUs can lose part of their economic advantage when networks, memory systems or resource scheduling cannot keep up.
Some problems have more mature solutions than others. Liquid cooling is already commercially available, while electricity grid construction involves longer timelines and more organizations. Software optimization can improve utilization, but predicting the economics of expensive infrastructure over multiple hardware generations remains difficult.
Our ranking reflects how strongly each problem limits growth, how widely it affects the industry and how much uncertainty remains around the available solutions.
| Rank | Unsolved problem | Evidence | Current severity |
|---|---|---|---|
| 1 | Electricity and grid access | Around 20% of planned capacity potentially delayed | Critical |
| 2 | Infrastructure financing and profitability | CoreWeave reported a $1.17B net loss in 2025 | Critical |
| 3 | AI chip manufacturing constraints | Four designers consumed around 90% of HBM and CoWoS supply | Critical |
| 4 | Inference costs and GPU utilization | Token prices have fallen dramatically, but complex workloads remain costly | High |
| 5 | Networking and GPU coordination | NVIDIA networking revenue grew 142% in fiscal 2026 | High |
| 6 | Cooling and power density | New rack-scale systems require liquid cooling | High |
| 7 | Supplier concentration | Heavy dependence on NVIDIA, TSMC and major memory manufacturers | High |
| 8 | Reliability and operational failures | IT and networking caused 23% of impactful outages in 2024 | Moderate to high |
| 9 | Environmental resource consumption | Electricity demand keeps growing despite efficiency gains | High |
| 10 | Geopolitical restrictions | Multibillion-dollar NVIDIA inventory charges linked to export rules | High |
Final verdict: What is the biggest unsolved problem in AI infrastructure?
The biggest unsolved problem in AI infrastructure is currently securing enough electricity to run new data centers, while making sure those facilities can generate attractive returns over their lifetime.
Electricity is the clearest immediate obstacle. The IEA expects global data center consumption to approach 950 TWh by 2030, yet grid limitations are already threatening a substantial share of planned projects.
Chip manufacturing is another serious constraint, particularly in high-bandwidth memory and advanced packaging. NVIDIA, TSMC and major memory suppliers continue to play unusually important roles in determining how much computing capacity the industry can produce.
But the longer-term question is financial. Companies are investing hundreds of billions of dollars in infrastructure even though GPU prices, hardware efficiency and AI workloads can change dramatically within a few years.
We have strong evidence that AI demand is growing. We also have strong evidence that chips are becoming more efficient and that software is making computing cheaper. What remains much less certain is whether every major infrastructure provider can maintain healthy returns while continuously upgrading expensive equipment.
For now, electricity is the biggest physical bottleneck. Making AI infrastructure consistently profitable is the more difficult long-term problem.
If you want more recent data on this point, see the latest figures in our AI Infrastructure report ↓.
From our AI infrastructure report, updated regularly. Have a look ↓
Sample pages from an earlier edition. The report has been updated with new data since.
Our methodology
This analysis examines the biggest unsolved problems in AI infrastructure using recent financial disclosures, energy-demand projections, semiconductor supply estimates, technical benchmarks and data center operating research. We compare the constraints on building computing capacity with the economic challenges of operating that capacity profitably.
We divided the assessment into physical infrastructure, semiconductor supply, computing performance, operational reliability, financial sustainability and external constraints. Within each category, we prioritized measurable developments such as electricity consumption, grid connection delays, packaging and memory availability, GPU networking performance, capital expenditures, financing costs and reported outage losses.
We distinguished current operational constraints from longer-term risks. Electricity access and semiconductor supply can immediately prevent new capacity from becoming available, while falling hardware prices, changing workloads and financing obligations can affect profitability over several hardware generations. We also separated demand indicators, such as revenue growth and contracted backlog, from evidence of actual profitability or sustained hardware utilization.
The final ranking reflects three considerations: how directly a problem restricts infrastructure expansion, how widely it affects the industry and how much uncertainty remains around possible solutions. Technical improvements were considered alongside their practical deployment requirements, costs and limitations. Company-specific financial results were used as concrete examples of broader pressures, rather than assumed to represent every infrastructure provider.
For electricity and grid constraints, the principal references include the IEA's updated Energy and AI outlook, its analysis of AI and energy security, the Lawrence Berkeley National Laboratory's 2024 United States Data Center Energy Usage Report, and the IEA's Electricity 2026 assessment of electricity grids.
For semiconductor capacity and computing hardware, we relied on Epoch AI's analysis of advanced packaging and HBM supply constraints, NVIDIA's fiscal 2026 financial results, the GB200 NVL72 technical specifications, and NVIDIA's technical analysis of rack-scale GPU interconnects. Epoch AI's estimated supply shares are treated as research estimates, not official manufacturer allocations.
To assess efficiency and inference performance, we used Stanford HAI's 2025 AI Index, Google Cloud's Ironwood TPU technical analysis, and MLCommons' MLPerf Inference v6.0 results. We treated historical improvements in hardware efficiency and inference prices separately from changes in total electricity consumption and the cost of completing complex AI tasks.
For financing and profitability, our main sources were CoreWeave's fiscal 2025 financial results, Alphabet's June 2026 investor presentation, and Alphabet's second-quarter 2026 Form 10-Q. We examined reported revenue, operating profitability, interest expense, capital spending and contracted backlog without treating future contracted revenue as revenue already earned.
For operating reliability, environmental impact and geopolitical constraints, we consulted the Uptime Institute's 2025 Annual Outage Analysis, Google's 2025 Environmental Report, and NVIDIA's SEC financial disclosures concerning export restrictions. These sources informed our assessment of outage costs, energy and water consumption, and the financial consequences of changing semiconductor trade rules.
Our conclusions reflect the cumulative evidence available through October 2026. The ranking is a comparative assessment of current severity and unresolved uncertainty, rather than a forecast that every bottleneck will develop at the same pace or affect every operator equally.
Written by the New Market Pitch teamWe track new markets so founders and investors can move faster. About us
