AI Agents: what are the biggest challenges now?

Why are AI agents suddenly becoming so hard to deploy?

AI agents are becoming harder to deploy because they can now do enough real work for their mistakes to have real consequences.

A chatbot giving a bad answer is annoying. An agent can take that answer and use it to modify a database, send an email, change a customer record, execute code or trigger another system. OpenAI defines agents around exactly this distinction: the model controls part of the workflow, decides what to do next and uses tools to pursue an objective.

We can already see how far behavior has shifted. Anthropic's research on Claude Code found that around 79% of conversations in one large analysis were classified as automation rather than augmentation, compared with 49% for ordinary Claude conversations. In a later study of roughly 400,000 Claude Code sessions, users generally kept control over higher-level decisions while Claude handled much more of the execution.

Enterprise adoption is moving in the same direction. Deloitte's latest survey found that companies expect agents to become much more autonomous over the next few years, while 74% of executives surveyed expect close to half of their business processes to be rebuilt or redesigned around them.

Companies already know agents can perform useful work. The harder part now is making that work reliable and safe enough to repeat.

What actually counts as an AI agent today?

An AI agent today is a system that can decide what to do next and take actions toward a goal, rather than simply producing an answer for the user.

That definition excludes a lot of products marketed as “agents.” A chatbot that retrieves a document and summarizes it has very little autonomy. A coding system that searches a repository, edits files, runs tests, notices a failure and tries another approach clearly has much more.

The useful distinction is the amount of judgment and authority handed to the model. At one end, the AI proposes an action and waits. Further along, it chooses tools and completes several steps. At the more autonomous end, it can keep working until it decides the task is finished or needs human help.

This distinction becomes important when people talk about agent adoption because very different systems get bundled into the same number. An agent allowed to draft a refund request carries much less risk than an agent allowed to issue the refund itself.

Type of system	What the AI can do	Main risk
Assistant	Suggest an answer or action	Wrong recommendation
Tool-using agent	Choose and call tools	Wrong action
Workflow agent	Complete several dependent steps	Errors accumulate
Autonomous agent	Continue with limited intervention	Mistakes can spread before anyone sees them
Multi-agent system	Delegate work between agents	One agent can pass bad information to another

Why does AI agent reliability fall apart on long tasks?

AI agent reliability is currently the biggest technical problem because small mistakes compound quickly once a task contains many dependent steps.

A model can look excellent when it performs one clean action. Real agents often need to understand the request, find information, choose the correct tool, enter the right parameters, interpret the result, update their plan, verify what happened and decide what to do next.

Imagine that an agent succeeds 95% of the time at every independent step. Over 20 equally important steps, the probability of getting all 20 right falls to about 36%. Real workflows are obviously more complicated than that toy calculation, but it explains why seemingly strong models can become surprisingly fragile over long runs.

Recent research keeps finding the same weakness from different angles. Microsoft tested agents under workplace multitasking and saw baseline completion fall from 16.7% with 12 concurrent tasks to 8.7% with 46. Another production-oriented study of more than 10,000 agent trajectories found success falling roughly geometrically as dependent steps accumulated, approaching zero on one tool-use task around 16 steps.

The failures are also difficult to diagnose. Microsoft's AgentRx research examined 115 failed agent trajectories across API workflows, incident management, web tasks and file operations. Researchers found that identifying the point where a run first became unrecoverable was often difficult, especially across long trajectories and multi-agent interactions.

An agent may choose the wrong file, interpret it plausibly, call the right tool for the wrong reason and keep building on the mistake. Seeing the final failure does not tell an engineering team whether the agent misunderstood the user, retrieved bad information, chose the wrong tool, passed invalid arguments or incorrectly decided that its work was complete.

Production agent systems therefore need detailed traces of tool calls, arguments, outputs and intermediate decisions. Without that visibility, teams can see that the agent failed without knowing where the failure really began.

If you want more recent data on this point, please see our latest agentic AI market report.

How long can AI agents actually work on their own now?

The strongest AI agents can currently handle work that would take a person hours, but their reliability still falls well short of what most companies would accept for unsupervised production work.

METR's task-horizon research gives us one of the clearest measures. Instead of asking whether a model passes a benchmark, METR estimates how long a software, machine-learning or cybersecurity task can be before the agent's probability of completing it falls below a certain level.

Those horizons have grown dramatically. Frontier systems have progressed from tasks measured in seconds and minutes to tasks that represent many hours of human work.

The important detail is the probability attached to the headline number. METR's widely discussed 50% horizon tells us the duration at which the agent is expected to succeed only half the time. METR also cautions that its estimates beyond roughly 16 hours remain less certain because there are fewer extremely long tasks in the benchmark.

Anthropic has observed a similar pattern in real usage. In one analysis, estimated success was around 60% for tasks taking humans less than an hour but closer to 45% for tasks that would take people five hours or longer. Interactive conversations degraded less sharply, which makes sense because users can keep redirecting the model.

Agents can now attempt genuinely long jobs. The harder problem is getting them to finish those jobs reliably without someone repeatedly correcting them.

Are AI agent benchmarks making agents look better than they really are?

Some AI agent benchmarks are currently giving companies too much confidence because they remove exactly the messiness that breaks agents in production.

Microsoft's WAREX research demonstrated this by introducing network problems, website changes and other real-world instability into familiar browser tasks. Agent performance dropped once the environment became less predictable.

Evaluation itself can also be surprisingly unreliable. In separate Microsoft research, judging the actual files created by agents agreed with human evaluation around 90% of the time. Using screenshots and action logs to judge those same outcomes produced agreement closer to 40%.

Static benchmarks create another problem. Enterprise software keeps changing. Interfaces move, permissions change, APIs evolve and company procedures get rewritten. An agent tested on one frozen version of that environment can encounter something materially different soon afterward.

For production systems, repeated trials, failure severity, latency, cost, policy violations and performance under changing conditions tell us much more than one headline success rate.

Common metric	What it can hide
One successful run	Run-to-run instability
Average benchmark score	A severe failure mixed with harmless mistakes
Stable test environment	Website, API and network changes
Final answer only	Dangerous intermediate actions
Short tasks	Compounding mistakes
Static test set	Changing real-world workflows

If you want more recent data on this point, please see our latest agentic AI market report.

Is prompt injection much more dangerous with AI agents?

Yes. Prompt injection is considerably more dangerous with AI agents because attackers can now try to manipulate actions, permissions and data access, not just the text an AI produces.

The simplest example is a browser agent reading a webpage containing malicious instructions hidden inside content that the user genuinely asked it to inspect. The agent has to consume the webpage to complete the task, yet some of that content may be trying to redirect its behavior.

OpenAI's recent security work describes the strongest prompt-injection attacks increasingly as a form of social engineering. Filtering suspicious strings cannot be enough when a malicious instruction can be written in ordinary language and embedded in legitimate content.

OWASP's agentic-security work identifies a wider family of risks around the same problem: goal hijacking, tool misuse, privilege escalation, memory poisoning, data exfiltration and cascading failures between agents.

The practical response is layered security: the model should resist manipulation, while the surrounding system restricts access and puts extra checks around dangerous actions.

Can companies safely give AI agents access to real systems?

Companies can safely give AI agents access to real systems only when that access is tightly limited and tied to the specific job the agent is doing.

Broad permanent permissions are especially risky because agents can misunderstand instructions, encounter manipulated content or combine information in ways nobody intended.

The problem becomes obvious with ordinary business tasks. An HR agent may legitimately have access to compensation information. That does not mean it should be able to insert that information into an email to any employee. A customer-service agent may need to inspect billing records while still requiring approval before issuing an unusually large refund.

Microsoft's CI-Work benchmark recently tested this kind of contextual privacy inside enterprise workflows. Across frontier models, privacy-violation rates ranged from 15.8% to 50.9%, while sensitive information leakage reached as high as 26.7%.

More worrying, stronger task performance sometimes came with more privacy violations. Simply making the model larger or giving it more reasoning did not fix the problem.

That pushes agent security toward narrowly scoped access, temporary credentials where possible, separate authorization for high-impact actions and clear boundaries between information the agent may read and information it may disclose.

If you want more recent data on this point, please see our latest agentic AI market report.

Why is AI agent memory still so unreliable?

AI agent memory is still unreliable because remembering everything is easy while remembering the right thing at the right moment is much harder.

Long-running agents accumulate huge amounts of information: user instructions, documents, failed attempts, tool results, conversations, previous decisions and changes in the outside world. Keeping all of it in context can make the agent slower, more expensive and sometimes less accurate.

Microsoft's recent memory research shows how quickly this becomes difficult in group settings. GroupMemBench tests agents inside multi-person conversations rather than simple one-to-one chats. The strongest system reached only 46% average accuracy. Accuracy on updating knowledge fell to 27.1%, while ambiguity handling reached 37.7%.

The hardest memory problem is often deciding which version of a fact remains valid. A customer changes address. A deadline moves. A manager reverses an earlier instruction. An employee changes teams. An agent that retrieves the old information perfectly can still make the wrong decision.

Security adds another complication. OWASP treats memory poisoning as a separate agentic risk because malicious or simply incorrect information that survives in persistent memory can influence later sessions.

Useful agent memory therefore needs provenance, expiry, permissions, conflict handling and deliberate forgetting. Larger context windows help with some tasks, but they do not provide those controls by themselves.

Can AI agents actually handle a normal day at work?

AI agents still struggle badly with a normal day of messy, overlapping work, even when they look impressive on isolated tasks.

Microsoft built Multi-Horizon Task Environments specifically to test that gap. Instead of asking an agent to complete one job, researchers gave it five-hour simulated work sessions containing 12 to 46 concurrent tasks. Each task required around 10 to 30 dependent steps.

As the workload increased, completion across baseline systems fell from 16.7% to 8.7%.

The agents struggled for familiar reasons. Memory filled up. Details from one task leaked into another. Some jobs depended on earlier jobs finishing first. Priorities had to be reconsidered continually.

Microsoft then built CORPGEN around hierarchical planning, isolated sub-agents, layered memory and learning from previous successful runs. At the highest load, completion reached 15.2%, compared with 4.3% for the baseline configuration used in that comparison. That is roughly a 3.5x improvement.

The result is encouraging, but the absolute level keeps the hype in perspective. Even the stronger system still completed only a minority of these difficult workplace tasks.

Are MCP and A2A finally fixing AI agent interoperability?

MCP and A2A are rapidly fixing the connectivity problem between AI agents, tools and other agents, while trust and coordination remain much less solved.

MCP's growth has become difficult to ignore. Its maintainers recently reported close to half a billion monthly downloads across Tier 1 SDKs, while both the TypeScript and Python SDKs had passed one billion cumulative downloads. The latest specification also added stronger authorization features, header-based routing, a stateless protocol core and a formal extension framework.

A2A has developed quickly alongside it. The Linux Foundation reported more than 150 supporting organizations after roughly its first year, including AWS, Google, Microsoft, Salesforce, SAP and ServiceNow. Production usage already spans areas such as financial services, insurance, supply chains and IT operations.

Together, the two standards address different parts of the stack. MCP helps agents connect with tools and data. A2A helps agents communicate and coordinate with other agents.

That also makes multi-agent systems easier to build. A planner can delegate to a research agent, a coding agent and a verifier, but every handoff creates another place where context can be lost or a bad result can be trusted by the next agent.

OWASP already treats cascading failures between agents as a distinct risk. Microsoft has also found that architecture matters heavily in multi-agent setups: memory that helps an orchestrator plan and delegate is different from the memory individual agents need to complete their own tasks.

Connecting agents is getting easier very quickly. Reliable coordination still has a long way to go.

Are AI agents still too expensive to use at scale?

AI agents can still become expensive at scale because one completed task may require dozens of model calls, tool calls, retries and verification steps.

Token prices have fallen sharply, which helps. But a normal AI response might involve one request and one answer, while an agent may repeatedly plan, search, retrieve documents, call an API, inspect the result, reconsider its approach and check its own work.

The model bill is only part of the cost. Browser infrastructure, search, databases, sandboxed compute, third-party APIs, observability and human review all add to the total.

Retries deserve more attention than they usually get. An agent that costs $0.50 per attempt but succeeds half the time may be a worse product than one costing $2 per attempt that almost always finishes correctly. A cheap agent stops looking cheap pretty fast when an employee has to spend ten minutes inspecting every result.

Gartner's current work on agent economics makes the same point from a business perspective: companies need measures tied to actual work and capacity rather than treating token spend or simple automation counts as proof of ROI.

The useful unit is increasingly cost per correctly completed outcome.

Cost	Why agents amplify it
Model inference	One job can trigger many reasoning calls
Context	Long-running agents repeatedly carry or retrieve state
Tools	Browsers, APIs, search and compute can add separate charges
Retries	Failed runs consume resources without completing the work
Verification	Additional models or people have to check important outputs
Engineering	Permissions, evaluation and monitoring require infrastructure

Why are companies deploying AI agents before they can govern them?

Companies are currently deploying AI agents much faster than they are building the controls needed to manage those agents safely.

Deloitte's multinational survey of 3,235 business and technology leaders found that only 21% of organizations had mature agentic governance. Roughly four out of five therefore lacked mature controls such as clearly defined decision boundaries, real-time monitoring and complete audit trails.

A more recent Deloitte survey found another gap. Only 5% of organizations considered their business processes highly prepared for agents, while just 15% had scaled orchestrated multi-agent systems across functions.

These figures explain why governance is becoming such a practical bottleneck. Companies have to decide which actions an agent can execute without approval, who is responsible when it makes the wrong decision, which data it can combine, how long its memory should persist and when the workflow must stop automatically.

Human review alone will not scale either. If every agent action needs approval, most of the promised automation disappears. Reviewing only occasional actions creates a different risk because people can become used to approving streams of mostly correct work.

Deloitte's latest findings support a hybrid approach: 75% of surveyed executives said human-agent collaboration would create more value than agent-only automation.

Do companies need to rebuild workflows around AI agents?

Yes. Companies that simply drop AI agents into old human workflows will probably capture only a fraction of the potential value.

Deloitte's latest survey makes the mismatch unusually clear. Only 5% of organizations said their processes were highly prepared for agents, while 74% of executives expected close to half of business processes to be redesigned or rebuilt around them within four years. Nearly two-thirds were already reconsidering parts of their business model.

We can see why from a simple office process. A human might receive information by email, copy it into another system, check a policy document, ask for approval and then re-enter the answer somewhere else. Automating every one of those steps separately preserves most of the original friction.

An agent-oriented process can retrieve the relevant information itself, check the policy, prepare a decision, run deterministic validations and ask a person only when something unusual or high-risk appears.

For many businesses, workflow design may become a bigger constraint than model intelligence.

If you want more recent data on this point, please see our latest agentic AI market report.

Is bad enterprise data becoming a bigger problem than weak AI models?

For many enterprise AI agents, messy data is already a bigger practical problem than the underlying model.

A prototype can use a clean database, a few selected documents and carefully designed APIs. An employee rarely gets such a tidy environment. Real organizations have information spread across email, shared drives, spreadsheets, old software, messaging platforms, ticketing systems and undocumented habits.

Agents depend on that information to act. If an agent retrieves an outdated policy and merely summarizes it, the damage may be limited. If the same agent uses that policy to approve a payment or reject a customer request, stale data becomes an operational error.

Access rules make the problem even harder. The important question is increasingly whether the agent should use a piece of information in a particular context, rather than simply whether the agent can technically retrieve it.

Microsoft's CI-Work results show why. Frontier models frequently failed to respect contextual privacy boundaries even when they were capable of finding the correct information. Better reasoning alone did not remove the failures.

Companies working seriously with agents therefore need much better control over data freshness, ownership, lineage, access and audience. Without that foundation, upgrading the model can improve reasoning while leaving the underlying information problem untouched.

What happens when an AI agent does not realize it is wrong?

AI agents are still weak at recognizing some of their own failures, especially when the task has no clear test for success.

Self-correction works surprisingly well in environments that provide objective feedback. A coding agent can run tests. A data agent can check whether an output matches a schema. An IT agent can verify whether a service came back online.

Open-ended business work provides much less feedback. A research agent may miss the strongest counterargument without knowing it. A sales agent can misread why a customer objected. A strategy agent can give a coherent answer to the wrong business question.

There may be nothing in the environment that tells the agent it failed. That's the awkward part.

This helps explain why software engineering remains one of the strongest areas for agents today. Code gives the model an unusually rich verification loop: compilation, tests, static analysis, runtime behavior and explicit errors.

Microsoft's work on scientific agents shows the harder side of the problem. Agents performed much better on well-defined analysis workflows than on tasks requiring genuinely novel insights and open-ended exploration.

The near-term boundary for autonomy may depend heavily on verifiability. Where agents can check their own work against something objective, we can give them much more freedom.

Are AI agents actually producing enough ROI yet?

AI agents are already producing real ROI in some workflows, but the evidence is much stronger in coding and structured operational work than across the enterprise as a whole.

McKinsey's enterprise research shows agentic coding among the first use cases reaching meaningful scale. Around one-fifth of organizations in its survey were scaling coding agents, rising to 31% among larger companies. Thirty-two percent also said agentic coding had allowed them to skip buying at least one software product or feature because they could build the functionality internally.

That is a stronger economic measure than a vague claim that employees “saved time.” Avoiding a software purchase or materially increasing completed work has an identifiable financial value.

The wider AI picture remains much murkier. Gartner's current research on AI value still emphasizes that many companies lack the financial controls and measurement systems needed to understand what they are getting back. One Gartner survey found only 44% of organizations had implemented financial guardrails or AI FinOps practices.

Hours saved alone are weak evidence of ROI. Correctly completed work, throughput, human-intervention rates, avoided costs, additional revenue and full operating cost tell us much more.

Evidence	How useful it is for judging ROI
More agent usage	Shows demand, but little about financial value
Hours saved	Useful capacity estimate, but may never become cash savings
Higher completed-task volume	Strong operational evidence
Lower human handling	Direct evidence of automation
Avoided software or labor cost	Clearer financial benefit
Higher revenue or lower total cost	Strongest business evidence

Will better AI models fix most agent problems?

Better AI models will remove a lot of current agent failures, but several of the hardest problems will remain even if model intelligence keeps improving quickly.

METR's task-horizon results show how much raw capability growth matters. Stronger models can stay on task longer, choose tools better and recover from more mistakes.

Yet Microsoft's recent research gives us several examples where architecture matters just as much. CORPGEN produced gains of up to 3.5x under heavy workplace multitasking through planning, memory isolation and experiential learning. CI-Work found that larger models or additional reasoning did not reliably solve contextual privacy failures.

A more intelligent model also cannot decide an organization's permission policy by itself. It still needs accurate data, secure credentials, reliable tools, auditability and clear rules for what requires approval.

Model capability remains crucial, but orchestration, memory, security, evaluation and workflow design increasingly decide whether that capability survives contact with real work.

If you want more recent data on this point, please see our latest agentic AI market report.

So what are the biggest challenges for AI agents now?

The biggest challenge for AI agents today is making long-running autonomous work reliable enough that companies can trust the result without checking every step.

Reliability sits first because nearly every other problem gets worse when an agent works for longer. Mistakes compound, memory fills up, tools fail, external content can manipulate the system and the agent may not notice that its own plan has drifted.

Security comes immediately after. Agents are gaining access to browsers, company data, codebases and business systems faster than organizations are learning how to control that access. Microsoft's recent enterprise privacy results, with violation rates between 15.8% and 50.9%, make this a very concrete problem.

Memory remains weak too. GroupMemBench's best system reached only 46% average accuracy in multi-person conversations. Real-world workload is another barrier: Microsoft's workplace experiments showed baseline task completion falling to 8.7% as agents had to juggle 46 concurrent jobs.

Evaluation then complicates the picture because clean, static benchmarks can make agents look much stronger than they behave when websites change, networks fail, workflows overlap or success becomes ambiguous.

Companies also have to solve the organizational layer. Deloitte currently finds just 5% of surveyed organizations saying their processes are highly ready for agents, despite executives expecting agents to reshape a large share of those processes.

Finally, the economics have to work after retries, tool calls, monitoring and human supervision are included.

These problems do not look like evidence that agents have hit a dead end. Long-horizon capability keeps improving, MCP and A2A are becoming real infrastructure, and measurable value is already appearing in coding and structured operations.

For now, capability is running ahead of dependability. That is the gap the AI agent industry still has to close.

Who is the author of this content?

NEW MARKET PITCH TEAM

We track new markets so founders and investors can move faster

We build living "market pitch" documents for emerging markets: AI, synthetic biology, new proteins, and more. Instead of outdated PDFs or hallucinated LLM answers, our clients get a clean, visual, always-updated view of what's really happening: key players, deals, regulations, and signals that matter. Learn more about us.