How big is the AI safety market really?
In our AI safety market deck, you will find everything you need to understand the market
SUMMARY
The AI safety market is worth about $3 billion today if we count the dedicated commercial products and services used to secure, govern, evaluate and control AI systems. Pure frontier-model safety is much smaller; most spending there still happens inside AI labs, governments and research organizations rather than through external vendors.
The unusually useful part of the current market data is that two fairly different estimates now converge around the same number. Gartner puts technologies specifically used to secure AI at $2.84 billion in 2026, while MarketsandMarkets puts the broader AI trust, risk and security management market at $3.09 billion.
Most commercial AI safety spending is not going into alignment research. It is going into fairly practical enterprise problems: stopping data leakage, controlling employee AI use, detecting prompt attacks, governing agents, monitoring model behavior and preventing AI applications from doing things they should not do.
Cybersecurity is therefore becoming the commercial center of gravity. Palo Alto Networks, Check Point and F5 have collectively spent more than $1 billion on disclosed acquisitions of specialist AI-security companies, showing that established security vendors already see AI protection as a product category worth buying into.
The startup evidence is also getting harder to dismiss. HiddenLayer reported more than tenfold ARR growth and more than 50 new platform customers in a year, while evaluation companies such as Braintrust and Patronus AI have raised large rounds around products that enterprises are already using.
Evaluation and governance are turning into software markets because AI systems need continuous supervision rather than one-off approval. The shift becomes more important with agents: once an AI system can call tools, alter files, access databases or execute workflows, companies need controls that operate while the system is running.
Regulation is beginning to create a spending floor. The EU AI Act now imposes evaluation, risk-management, incident-reporting and cybersecurity obligations on the most consequential general-purpose models, while California is building formal structures for independent AI verification and auditing.
Frontier-model safety is economically larger than its external vendor revenue suggests. Anthropic and Accenture each expect to invest at least $1 billion over five years in building AI safely, but much of that spending will remain inside labs or embedded partnerships rather than appearing as revenue for standalone safety companies.
The market could plausibly move beyond $10 billion around the start of the next decade without any single safety category becoming enormous. Security, agent controls, governance, evaluation, observability, compliance and independent testing can all expand at the same time as enterprise AI deployment spreads.
So the clean answer remains roughly $3 billion today. The more interesting change is that AI safety is shifting from a collection of research programs and policy discussions into an operating expense attached to deploying AI in the real world.
This market map, featured in our AI safety market deck, highlights top companies and startups in the AI safety market
How big is the AI safety market today?
The broad commercial AI safety market is worth roughly $3 billion today, while the much narrower market for frontier-model safety is still only a fraction of that.
That $3 billion estimate is unusually well supported for such a young category. Gartner's latest work puts spending on technologies specifically used to secure AI at $2.84 billion. MarketsandMarkets currently estimates AI trust, risk and security management at $3.09 billion. The two studies use different definitions, yet they land within about 9% of each other.
Gartner's breakdown is especially useful because it shows where companies are actually spending the money. It estimates $508 million for AI application security, $433 million for controlling how employees and companies use AI, $275 million for dedicated AI governance platforms and $251 million for AI gateways. Another $1.37 billion sits across other products built specifically to secure AI.
MarketsandMarkets takes a somewhat broader view. Its $3.09 billion estimate includes AI governance, risk and compliance software, AI assurance and evaluation, observability, runtime security, consulting, implementation and independent testing.
So we would use approximately $3 billion as the best current estimate for the dedicated commercial market around making AI safer, more governable and more secure.
Pure frontier AI safety is considerably smaller. Alignment research, dangerous-capability testing, catastrophic-risk evaluation and independent testing of frontier models are carried out by a small group of AI labs, specialist organizations, governments and research institutions. Much of that work happens internally and therefore never becomes external market revenue.
| Current AI risk and safety category | Estimated annual spending |
|---|---|
| AI application security | $508M |
| AI usage control | $433M |
| AI governance platforms | $275M |
| AI gateways | $251M |
| Other technologies for securing AI | $1.37B |
| Gartner total for securing AI | $2.84B |
| MarketsandMarkets AI TRiSM market | $3.09B |
| MarketsandMarkets previous-year AI TRiSM estimate | $2.23B |
Why are AI safety market estimates so wildly different?
AI safety market estimates vary so much because researchers often give similar names to markets that contain completely different products.
The easiest example is AI governance. Gartner currently estimates dedicated AI governance platform spending at $492 million under a broader governance-market definition. Its separate securing-AI segmentation counts $275 million of governance spending. MarketsandMarkets has previously published estimates approaching $1 billion for wider governance categories.
The difference mostly comes from what gets counted. A narrow governance platform might inventory AI systems, track risks, enforce policies and document compliance. A wider study can also count consulting, model monitoring, privacy technology, MLOps, LLMOps, auditing and other software that existed before the current AI boom.
Cybersecurity creates an even bigger distortion. MarketsandMarkets values the wider generative-AI cybersecurity market at $8.65 billion for 2025. That figure covers areas including threat intelligence, SIEM, encryption, access control and conventional cybersecurity products that either protect AI or use AI themselves.
Calling that whole $8.65 billion market "AI safety" would make the category look far larger than the dedicated market really is.
We therefore use a fairly strict test: would most of the product or service still exist in essentially the same form if companies were not deploying modern AI systems? If the answer is yes, we should be careful about counting all of its revenue as AI safety.
That is why the roughly $3 billion AI-specific security, risk and governance estimate is more useful than many larger numbers circulating around the market.
If you want more recent data on this point, please see our latest AI safety market report.
As this chart shows, and as featured in our AI safety market deck, search interest in AI safety has been growing steadily
How much AI safety spending is really cybersecurity?
Cybersecurity currently takes the largest share of commercial AI safety spending.
Gartner's numbers make that fairly obvious. AI application security, AI usage controls and AI gateways alone account for almost $1.2 billion of its $2.84 billion estimate. Parts of its remaining $1.37 billion category also cover security technologies, so the real cybersecurity share is higher.
Companies are buying these products to solve very concrete problems. They want to stop employees from sending confidential information to public models, detect prompt injection, prevent models from exposing protected data, control what AI agents can access and block malicious inputs before an application acts on them.
The acquisition market points in the same direction. Palo Alto Networks paid $635 million for Protect AI. Check Point subsequently spent about $202 million on Lakera, which protects generative AI applications and agents against prompt injection, data leakage and model manipulation. F5 paid $145.2 million for CalypsoAI and then added SurePath AI for another $50.1 million. Cisco had already bought Robust Intelligence, whose technology became part of Cisco AI Defense.
Across Protect AI, Lakera, CalypsoAI and SurePath alone, buyers spent more than $1 billion. That total excludes Cisco's purchase because the final price for Robust Intelligence was not officially disclosed.
Large security companies are telling us pretty clearly what they value: runtime protection, model security, AI discovery, attack prevention and agent controls. Commercial AI safety currently looks much more like an emerging cybersecurity category than a commercialized version of academic alignment research.
| AI security company | Buyer | Disclosed purchase price |
|---|---|---|
| Protect AI | Palo Alto Networks | $635M |
| Lakera | Check Point | ~$202M |
| CalypsoAI | F5 | $145.2M |
| SurePath AI | F5 | $50.1M |
| Total disclosed value | >$1.03B | |
| Robust Intelligence | Cisco | Undisclosed |
| Faculty | Accenture | Undisclosed |
Are AI safety startups becoming real businesses?
Some AI safety startups are now showing real commercial traction rather than relying mainly on venture funding and future expectations.
HiddenLayer gives us one of the clearest recent examples. The AI security company raised a $100 million Series B after reporting that annual recurring revenue had grown by more than 10 times in the previous year. It also said it had added more than 50 platform customers across banking, insurance, pharmaceuticals, airlines, government, technology and other sectors.
Those numbers tell us more than the funding round itself. A $100 million investment can reflect expectations. Tenfold ARR growth and dozens of new enterprise customers show that organizations are already paying for the product.
Evaluation companies are attracting meaningful capital too. Braintrust raised $80 million to expand its AI evaluation and observability infrastructure. Patronus AI raised $50 million after building products around model and agent evaluation.
The exits are even harder to dismiss. Palo Alto Networks' $635 million Protect AI acquisition, Check Point's roughly $202 million Lakera deal and F5's purchases of CalypsoAI and SurePath show that established security companies now assign meaningful strategic value to specialist AI protection technology.
Revenue disclosure remains frustratingly thin. Most private AI safety companies do not publish sales, margins or customer-level spending, so we cannot yet measure the sector the way we can measure mature cybersecurity categories.
Still, the market has moved beyond research projects and pre-revenue startups. Specialist companies are winning enterprise customers, growing recurring revenue and being acquired for hundreds of millions of dollars.
If you want more recent data on this point, please see our latest AI safety market report.
This chart, featured in our AI safety market deck, shows annual venture capital investment in AI safety startups
Is AI evaluation becoming its own market?
AI evaluation is becoming a genuine software category as companies realize they cannot reliably run generative AI and agents without constantly testing what those systems do.
Gartner now treats AI evaluation and observability as a distinct area. It estimates that LLM observability currently appears in only about 15% of generative-AI deployments but expects that share to reach 50% by 2028.
A separate Gartner forecast expects 40% of organizations deploying AI to use dedicated AI observability tools by 2028. Those forecasts are uncertain, but they capture a real technical problem: traditional software is generally deterministic, while generative systems can behave differently across prompts, contexts and model versions.
That creates demand for products such as Braintrust, Patronus AI, Arize and other evaluation platforms. Companies now test hallucinations, task success, policy compliance, bias, tool use and agent behavior before deployment and again in production.
AI evaluation overlaps heavily with safety, although we should not count all evaluation revenue as safety revenue.
Checking whether a support bot gives customers accurate opening hours is mostly a quality problem. Testing whether the same bot can leak private customer data or be manipulated into ignoring its restrictions is clearly a safety and security problem.
The boundary will stay messy, but evaluation is becoming one of the main commercial routes through which companies pay for safer AI.
Is AI governance finally becoming real software?
AI governance is now moving from spreadsheets and policy documents into software that can continuously control what AI systems are allowed to do.
Under Gartner's broader governance-market estimate, companies are expected to spend about $492 million on AI governance platforms this year and more than $1 billion by 2030.
That is still a small software category. The interesting part is how quickly the product itself is changing.
Early AI governance often meant keeping inventories, writing policies, documenting model decisions and preparing compliance reports. Those functions remain useful, but companies deploying hundreds of models and AI applications cannot realistically govern everything through committees and manual reviews.
Current platforms increasingly connect directly to AI systems. They can discover which models employees are using, classify risk, enforce approval rules, track model changes and sometimes stop prohibited behavior automatically.
The vendor mix also shows that governance is becoming mainstream enterprise software. Specialist companies such as Credo AI, Holistic AI and ModelOp now compete alongside IBM, SAP, ServiceNow and OneTrust.
Runtime governance is particularly important for agents. A policy document cannot stop an autonomous system halfway through an unauthorized action. Software can potentially block the tool call, require human approval or remove the agent's permissions.
MarketsandMarkets now expects AI security and runtime protection to be the fastest-growing part of the wider AI trust, risk and security market, at a projected 33.1% annual rate through 2031.
Governance is starting to look less like an annual compliance exercise and more like a control layer that stays active while AI systems are running.
This chart, featured in our AI safety market deck, shows how HiddenLayer is positioned in AI safety
Is regulation already creating AI safety revenue?
Yes, AI regulation is now creating work that companies have to pay for rather than merely encouraging them to behave responsibly.
The EU AI Act gives us the clearest large-scale example. Providers of general-purpose AI models face documentation and transparency obligations. Providers of models considered to present systemic risk also have to conduct model evaluations, assess and mitigate systemic risks, report serious incidents and maintain adequate cybersecurity safeguards.
The AI Office's enforcement powers for these general-purpose AI obligations became applicable in August 2026.
That changes the economics of safety. Adversarial testing, risk assessment, compliance documentation, incident tracking and model governance all require people, software or external service providers.
California is moving even more directly toward an independent assurance market. The state has created a framework for independent verification organizations that can assess AI systems and models for safety and risk, together with a registry for AI auditors.
More recently, California's governor directed state agencies to accelerate those systems and consider stronger requirements around embedded independent evaluators, verification of frontier-model safety frameworks and risk assessments, and potential emergency shutdown mechanisms.
We should not assume that every new AI rule creates a dollar of new software revenue. Large companies will perform some compliance internally, and existing legal or cybersecurity teams can absorb part of the workload.
But a regulatory floor now exists in major markets. Companies developing or deploying sensitive AI systems increasingly have to show what they tested, what risks they found and what controls they use. That creates a much more durable market than voluntary safety commitments alone.
If you want more recent data on this point, please see our latest AI safety market report.
Are companies actually worried enough about AI risk to pay for protection?
Yes, enterprises are paying because AI failures can now create ordinary business problems such as leaked data, cyberattacks, bad automated decisions and regulatory breaches.
We can see that behavior more clearly in customer growth than in surveys about executives' intentions.
HiddenLayer says it added more than 50 platform customers in one year while growing annual recurring revenue more than tenfold. Those customers came from sectors including finance, insurance, government, pharmaceuticals, transportation and technology.
Large security vendors are also turning AI protection into standard product lines. Cisco built Robust Intelligence into Cisco AI Defense. F5 is combining CalypsoAI and SurePath with its wider application-security platform. Palo Alto Networks is folding Protect AI into its security portfolio. Check Point is doing the same with Lakera.
These companies already sell to thousands of chief information security officers. Once AI protection becomes another module inside an existing security platform, companies do not need to discover an entirely new vendor category before they can start spending.
Current buying behavior is strongest where the risk is easy to explain financially: stolen data, unauthorized agent actions, malicious prompts, exposed intellectual property and compliance failures.
More abstract safety problems still have much weaker commercial demand outside the small number of organizations building frontier models.
This chart, featured in our AI safety market deck, shows annual funding in AI safety startups
Will AI agents make the AI safety market much bigger?
AI agents should increase safety spending substantially because an agent can turn a bad model output into a real action before a human catches it.
A chatbot might hallucinate an answer. An agent connected to company systems can potentially send an email, modify code, access a database, approve a workflow, call another service or execute a transaction.
That creates several new things companies need to control: which tools the agent can use, what data it can access, when it needs human approval, how its actions are logged and what happens when its behavior suddenly changes.
The products being launched lately already reflect that shift. HiddenLayer is using part of its new $100 million funding to expand agentic runtime security, including protection for autonomous coding agents. Check Point bought Lakera partly for its ability to protect agentic applications. F5 describes CalypsoAI as relevant to both generative and agentic AI.
MarketsandMarkets now explicitly cites autonomous, tool-using agents as one of the main reasons it expects the wider AI trust, risk and security market to grow from $3.09 billion today to $11.61 billion by 2031.
The World Economic Forum previously found that 82% of surveyed executives expected their organizations to adopt AI agents within one to three years. Adoption forecasts deserve caution, but even partial realization of that expectation would greatly increase the number of AI systems capable of interacting directly with business infrastructure.
Agent security could become one of the biggest additions to the current AI safety market, particularly if companies move from supervised copilots to systems that can work for long stretches with limited human intervention.
How much are frontier AI labs spending on AI safety?
Frontier AI companies are now committing hundreds of millions of dollars a year to safety-related work, although we still cannot cleanly separate safety spending from wider research and engineering budgets.
The freshest evidence is unusually large. Anthropic and Accenture recently announced that each expects to invest at least $1 billion over five years in building AI safely.
If those commitments were distributed evenly, the combined amount would average at least $400 million a year.
The partnership is particularly relevant because some of that work will be carried out by embedded evaluators from Accenture's Faculty business. Those evaluators are expected to work alongside Anthropic's teams on red teaming, alignment assessments and testing safeguards.
Anthropic describes embedded evaluation as a new model in which external evaluators gain access comparable to employees. That is a meaningful departure from the more limited external testing arrangements frontier labs have typically used.
Other laboratories also maintain substantial internal safety programs. OpenAI conducts dangerous-capability evaluations and safeguard testing through its preparedness process. Google DeepMind has frontier-safety frameworks and dedicated safety research. Frontier labs also spend on model security, red teaming, interpretability, abuse prevention and evaluations that overlap with product engineering.
We cannot aggregate those budgets properly because the companies do not report safety expenditure in a standardized way. Salaries, research compute and security engineering may serve both safety and product-development purposes.
Total economic spending on frontier AI safety is therefore already much larger than the revenue earned by independent frontier-safety vendors. Most of that money simply remains inside the labs.
If you want more recent data on this point, please see our latest AI safety market report.
This chart, featured in our AI safety market deck, compares the main business model options for AI alignment research labs
Can independent AI safety testing become a real business?
Independent frontier AI testing is starting to develop the structure of a real professional-services market.
Until recently, much of the most serious external frontier-model evaluation came from research organizations, nonprofits and government institutes. Groups such as METR and Apollo Research built expertise in areas including autonomy, deception and dangerous capabilities, while governments created their own testing capacity.
The Anthropic-Accenture partnership changes the commercial picture. Faculty, which Accenture acquired earlier this year, will place evaluators inside Anthropic to test models, run red teams, conduct alignment assessments and inspect safeguards.
California is moving in the same direction from the regulatory side. The state has now created formal structures for independent AI verification organizations and registered auditors. Its latest policy push also raises the possibility of independent evaluators working directly inside frontier labs.
That combination is worth watching closely. Frontier companies are beginning to pay external specialists for deep model access at the same time that regulators are defining who can qualify as an independent evaluator.
Cybersecurity developed a similar ecosystem around penetration testing and external audits. Financial markets have independent audit firms. Safety-critical industries use external certification and testing.
Frontier AI may eventually need its own version of those businesses.
We are still early. The number of frontier-model developers is tiny, standards remain unsettled and there is no mature recurring-revenue model yet. But independent AI testing now has both paying customers and regulatory support, which makes the category much more commercially credible than it was a few years ago.
Is government AI safety spending actually meaningful?
Government AI safety spending is meaningful for frontier research, but it remains small next to commercial cybersecurity and AI infrastructure.
The UK AI Security Institute offers the clearest budget benchmark. The institute says it is backed by £66 million in funding per financial year, alongside access to substantially larger government compute resources.
That is a serious research budget. It still illustrates how small dedicated public AI safety institutions remain compared with major technology markets.
A single private acquisition, Palo Alto Networks' $635 million purchase of Protect AI, was worth many times the institute's annual cash funding. Anthropic and Accenture's combined five-year safety commitments are larger again.
Public spending plays a different role, though. Government institutes can perform frontier testing, develop evaluation methods and produce standards that would be difficult for young private companies to fund on their own.
Government programs also create customers for commercial safety companies. HiddenLayer, for example, works with U.S. government and defense organizations and has been selected to participate in a Department of Energy initiative involving AI security for nuclear energy and critical infrastructure.
Government money will probably remain much smaller than enterprise spending while having an outsized influence on how the market defines tests, standards and acceptable safeguards.
This chart, featured in our AI safety market deck, shows revenue breakdown by customer segment in the AI safety market
Is AI safety big compared with the AI industry itself?
AI safety is still a small layer around a much larger AI economy.
Gartner estimates $2.84 billion of spending on technologies specifically used to secure AI this year. Even before we compare that figure with cloud infrastructure, chips or AI services, the difference in scale is obvious.
The important question is what happens to the ratio from here.
Gartner says spending on securing AI is currently growing about 83% year over year and expects another 68.7% increase next year, taking the category to roughly $4.78 billion.
Those growth rates are far higher than what we normally see in mature security markets. They are also easier to understand when we look at the base: companies are adding a new control layer after already deploying large amounts of AI.
Some safety spending will remain invisible because model providers include safeguards directly in their services, cloud companies bundle controls into platforms and security vendors package AI protection alongside larger contracts.
Even allowing for those hidden costs, dedicated AI safety remains much smaller than the technology it protects.
That gap is precisely why a few years of 30%, 50% or even higher growth can still leave us with a relatively modest market in absolute dollars.
Could the AI safety market really reach $10 billion?
A broad AI safety and risk-management market reaching $10 billion around the start of the next decade now looks quite plausible.
MarketsandMarkets currently projects AI trust, risk and security management to grow from $3.09 billion to $11.61 billion by 2031, equivalent to 30.3% annual growth.
We should not accept the forecast mechanically. Categories this young can change definition halfway through a forecast period, and future estimates naturally become less reliable the further out we go.
The near-term numbers make the direction harder to dismiss, though. Gartner's dedicated securing-AI estimate rises from $2.84 billion to $4.78 billion in just one year. AI usage controls are expected to grow 73%, AI gateways 70.9%, governance platforms 68% and AI application security 67.5%.
Those are forecasts rather than booked future revenue, but current business activity is moving the same way. HiddenLayer reports tenfold ARR growth. Large security vendors have already spent more than $1 billion on four disclosed specialist AI-security acquisitions. Dedicated governance software is approaching half a billion dollars in annual spending. Evaluation companies are raising rounds of $50 million to $80 million. Frontier safety has also just attracted multiyear corporate commitments measured in billions.
The growth is coming from several separate customer needs at once: enterprise security, agent control, regulation, evaluation and frontier-model testing.
Reaching $10 billion therefore does not require one speculative safety technology to suddenly become enormous. Several smaller categories can get there together.
| Market indicator | Current level | Near-term or longer-term direction |
|---|---|---|
| Gartner securing-AI spending | $2.84B | $4.78B next year |
| MarketsandMarkets AI TRiSM | $3.09B | $11.61B by 2031 |
| AI application security | $508M | $851M next year |
| AI usage controls | $433M | $749M next year |
| AI governance platforms | $275M in Gartner's security segmentation | $462M next year |
| AI gateways | $251M | $429M next year |
| Dedicated governance platforms, wider Gartner estimate | $492M | >$1B by 2030 |
| Anthropic + Accenture safety commitments | ≥$2B over five years | New frontier-safety capacity |
| Four disclosed specialist AI-security acquisitions | >$1.03B combined | Consolidation already underway |
If you want more recent data on this point, please see our latest AI safety market report.
This chart, featured in our AI safety market deck, shows how prompt injection defense platform technology has evolved over time
So how big is the AI safety market really?
The best current estimate is about $3 billion for the dedicated commercial AI safety, governance and security market, with pure frontier AI safety still representing a much smaller slice.
Three things make that estimate more credible today than it would have been even a year ago.
First, independent market estimates are converging. Gartner's $2.84 billion securing-AI figure sits close to MarketsandMarkets' $3.09 billion AI trust, risk and security estimate.
Second, real companies are generating evidence of customer demand. HiddenLayer has reported more than tenfold annual recurring revenue growth and over 50 new platform customers. Major cybersecurity groups have spent more than $1 billion on disclosed acquisitions of Protect AI, Lakera, CalypsoAI and SurePath.
Third, frontier safety itself is beginning to commercialize. Anthropic and Accenture have each committed at least $1 billion over five years to building AI safely, including a new embedded-evaluator model. California is building formal systems for independent verification organizations and AI auditors. The EU is already enforcing safety, evaluation and risk-management obligations for the most important general-purpose models.
We therefore see roughly $3 billion of identifiable annual commercial revenue today, alongside additional internal spending by AI labs, governments and companies that does not show up in market-revenue estimates.
The narrow frontier-safety market remains below the billion-dollar scale in external annual revenue based on the evidence currently available. Enterprise AI security, governance, evaluation and runtime controls account for most of the money.
That distinction is essential. Calling every cybersecurity product, consulting engagement or internal AI research budget "AI safety" can produce much larger numbers, but the result stops telling us much about the actual market.
Around $3 billion is the cleaner answer for now.
What has changed lately is the speed. Gartner currently sees spending on securing AI growing more than 80% this year. Specialist vendors are showing real recurring revenue. Agents are creating new runtime risks. Regulators are forcing companies to document and test safeguards. Frontier labs are putting much more money into independent evaluation.
The AI safety market is still small today. It is becoming expensive enough that companies can no longer treat it as a side project.
OUR METHODOLOGY
This analysis estimates the size of the AI safety market by separating dedicated commercial AI safety activity from the much larger universe of cybersecurity, AI software, consulting and internal research that can sometimes be placed under the same label. The central market-size anchor is therefore revenue and spending that is specifically attributable to securing, governing, evaluating or controlling AI systems.
We use Gartner's securing-AI market estimate and MarketsandMarkets' AI Trust, Risk and Security Management market as the two main commercial benchmarks. They use different boundaries, so we do not simply average them. Their convergence around $3 billion is useful because it gives us a range that can then be checked against individual security, governance, evaluation and runtime-control categories.
We keep commercial market revenue separate from acquisition values, venture funding, government budgets and internal frontier-lab spending. Those numbers are used to test whether demand and investment around the category are becoming economically meaningful, but adding them directly to market revenue would mix stocks, flows and internal costs and would overstate the size of the external market.
We also distinguish broad enterprise AI safety from frontier-model safety. Enterprise security, governance, observability and evaluation already have thousands of potential corporate customers. Frontier alignment, catastrophic-risk evaluation and dangerous-capability testing serve a much smaller group of model developers and institutions, and a large share of that work is still performed internally.
For the forward view, we treat published forecasts as scenarios rather than guaranteed outcomes. The $10 billion question is tested against current category growth, acquisition activity, customer evidence, agent deployment, regulation and the emergence of independent evaluation rather than accepted simply because one market-research firm publishes an $11.61 billion forecast for 2031.
We prioritized current primary sources where possible: market-research releases for category definitions and spending estimates, company filings and announcements for acquisitions and funding, government and regulator pages for legal obligations and public spending, and direct company disclosures for customer and ARR growth. Where precise private-company revenue is unavailable, we do not manufacture an estimate.
Key sources used for this analysis include: Gartner's 2026–2027 securing-AI spending breakdown, MarketsandMarkets' AI Trust, Risk and Security Management market study, Gartner on AI governance-platform spending, Gartner on dedicated AI observability adoption, Gartner on LLM observability, HiddenLayer's $100 million Series B and customer-growth disclosure, Palo Alto Networks on its Protect AI acquisition, Check Point on its Lakera acquisition, F5 on SurePath AI and its AI Security Platform, Anthropic on its embedded-evaluation partnership with Accenture, Accenture's corresponding AI-safety investment announcement, the European Commission's guidance on general-purpose AI obligations, the European Commission's AI Act enforcement timeline, California's framework for independent AI verification, the UK AI Security Institute, and the World Economic Forum's research on enterprise AI-agent adoption and governance.
In our AI safety market deck, we identify pain points entrepreneurs should prioritize
