Autonomous Vehicles: what are the biggest challenges now?

Why are autonomous vehicles suddenly much harder to dismiss?

Autonomous vehicles have crossed an important line: fully driverless transport is now a real service used by ordinary customers, so the debate has moved from “can the car drive itself?” to “can this work almost everywhere and at a sensible cost?”

Waymo currently gives fully autonomous rides in 14 cities after adding public riders in Denver, San Diego and Tampa. The company says it has completed more than 20 million autonomous trips overall and is serving hundreds of thousands of commercial rides every week. In China, Pony.ai ended its second quarter with 1,975 robotaxis and $12.1 million of quarterly robotaxi revenue, up 691% from a year earlier. Its fare-charging revenue grew even faster, by 849%.

Trucking is crossing the same line. Kodiak ended its second quarter with 35 customer-owned driverless trucks, more than 40,000 paid driverless operating hours and over 20,000 cumulative loads delivered. This week, the company said its safety case for driverless long-haul operations was 93% complete.

Those numbers are much more useful than another slick demonstration. We can finally judge autonomous vehicles on recurring rides, paying customers, fleet utilization, safety over hundreds of millions of miles and the cost of adding new territory.

The uncertainty has shifted with that progress. A car can already drive itself extremely well in certain places. What we still do not really know is how quickly that competence can spread across cities, climates and road systems without costs or failure rates rising with it.

If you want more recent data on this point, please see our latest autonomous vehicle market report.

Are autonomous vehicles actually safer than human drivers now?

The best autonomous vehicles are already beating human drivers on several important safety measures, at least inside the places where they have been thoroughly tested.

Waymo offers the largest dataset. Its analysis of more than 220 million fully autonomous miles found 94% fewer crashes causing serious or fatal injuries than a comparable human-driving benchmark, 82% fewer injury crashes overall and 93% fewer injury crashes involving pedestrians.

Company data always deserves scrutiny, so the more interesting confirmation came from the Insurance Institute for Highway Safety. IIHS researchers compared Waymo's driverless operation in San Francisco, Phoenix, Los Angeles and Austin with human drivers in the same cities. After restricting the comparison to crashes that people would normally report to police, Waymo vehicles had a 68% lower crash rate per mile.

That result is strong enough for us to say something clearly today: proven Level 4 robotaxis can already be safer than human driving inside mature operating areas.

The limit is equally important. We have excellent evidence for one unusually mature system across a growing but still selected collection of cities. We have far less evidence showing the same safety performance across heavy snow, poorly marked rural roads, unfamiliar countries or every competing autonomous stack.

Cruise showed how quickly that distinction can become painful. After a pedestrian was hit by another car and thrown into the path of a Cruise vehicle, the robotaxi pulled over while dragging the pedestrian about 20 feet. California suspended the company's driverless operations, federal regulators later criticized Cruise's disclosure of the incident, and GM eventually stopped funding Cruise as a standalone robotaxi business.

Safety evidence	What we can reasonably conclude
Waymo: 220M+ fully autonomous miles analyzed	The dataset is large enough to study serious crashes rather than demonstrations
Waymo: 94% fewer serious/fatal-injury crashes	Its mature Level 4 service is beating the human benchmark on the most severe outcomes
IIHS: 68% lower comparable crash rate	Independent research points in the same direction
Cruise pedestrian incident	One rare failure can still destroy regulator confidence very quickly

Why is autonomous-vehicle safety still so hard to prove?

Autonomous-vehicle safety remains surprisingly difficult to measure because two fleets can report very different crash totals even when the fleet with more crashes is actually safer per mile.

NHTSA itself warns against ranking companies from its automated-driving crash database. Operators drive different numbers of miles, work in different places and have different systems for detecting incidents. A robotaxi packed with cameras and telemetry can record minor collisions that would never appear in ordinary human-driver statistics.

The human baseline is messy as well. Minor crashes are under-reported, while serious crashes are much more consistently recorded. IIHS had to screen Waymo incidents specifically for crashes comparable to those people normally report to police before calculating its 68% difference.

Location can distort the numbers just as much. Ten million miles around wide suburban roads in good weather tell us something different from ten million miles through dense downtown streets full of cyclists, construction crews and pedestrians. A company can even make its safety rate look worse by deliberately entering harder territory.

So the metric we should watch now is whether the serious-crash rate stays low while the operating environment gets harder.

Are rare edge cases still the hardest technical problem for self-driving cars?

Rare edge cases are still one of the hardest problems in autonomous driving because events that look statistically tiny become ordinary operational problems once a fleet drives millions of miles every week.

Consider a failure that appears once every million miles. For a small test fleet, engineers might hardly ever see it. A fleet driving four million autonomous miles each week could encounter roughly four such situations every week. Expand that by another factor of ten and an apparently exotic problem becomes something operations teams face almost every day.

This is why the last fraction of driving reliability is so expensive. Emergency vehicles stopping in unusual positions, temporary traffic signals, police hand gestures, damaged lanes, debris, partially blocked intersections and unpredictable pedestrians each represent a small part of driving. Combined, the long tail never really ends.

Autonomous-driving companies are leaning much more heavily on AI to attack that problem. Pony.ai says its PonyWorld 2.0 world model can create richer training environments while reducing the extra engineering needed for each new city. Kodiak says its simulation system can already run more than one million simulations per hour at a compute cost of roughly one-tenth of a cent each.

Scale helps because every unusual encounter creates new training material. It also raises the bar. A software update that improves average performance while quietly making one rare scenario worse is unacceptable when the fleet is carrying passengers or hauling freight without a driver.

Can self-driving cars really work outside carefully mapped cities?

Geographic generalization is probably the biggest technical challenge in autonomous vehicles now because excellent driving in one city still does not mean a company can switch on another city like a new cloud region.

Waymo's latest expansion shows both the progress and the remaining friction. Public autonomous rides have now reached 14 cities, while the company is preparing many more locations. Yet its Munich rollout still begins with manually driven mapping, followed by testing with autonomous specialists and a phased validation period before commercial service.

That tells us something important about autonomous driving today. New cities are getting easier, but they still need local work.

Weather sits inside the same problem. Phoenix, Austin, Los Angeles, Guangzhou and Shenzhen provide complicated urban driving, yet they do not reproduce the winter conditions of Munich, Chicago or Minneapolis. Heavy snow, road spray, fog and low sun affect sensors while also changing road markings, braking and human behavior. For commercial autonomy, the useful question is how many normal local hours the fleet can safely serve rather than whether a car has ever completed a drive in snow.

Pony.ai is giving us another useful test. Its China fleet has operated through heavy rainstorms, while the company is now running fare-charging robotaxi operations in Doha through a partnership with national transport operator Mowasalat. It is also preparing larger overseas deployments through Uber. Different road rules, driving behavior and climate make those international operations much more informative than another expansion inside a familiar Chinese district.

The next major proof point is simple: whether city number 20 requires far less mapping, validation and engineering work than city number two.

If you want more recent data on this point, please see our latest autonomous vehicle market report.

Are remote operators secretly driving autonomous cars?

Remote operators are not secretly driving leading robotaxis from control rooms, although autonomous fleets still rely on humans when unusual situations need extra context.

Waymo gives unusually specific information about this. Its remote-assistance staff answer requests initiated by the automated driving system rather than continuously steering vehicles. The remote agent can provide information, and the Waymo Driver decides how to use it.

The staffing ratio is revealing. Waymo said roughly 70 remote-assistance agents were on duty worldwide at a time when its fleet contained around 3,000 vehicles. That comes to about one active support worker for every 43 vehicles, although agents are shared across the network rather than permanently assigned to particular cars.

That ratio is already fundamentally different from one paid driver per taxi.

The real question is what happens as fleets multiply. If every extra million miles produces the same number of assistance requests, remote operations eventually become expensive. The software needs to learn from recurring interventions so that fleet size grows faster than the support team.

Connectivity matters too. Waymo reported median one-way latency of roughly 150 milliseconds to U.S. operations centers and 250 milliseconds to overseas assistance. The vehicle still has to remain safe when that connection disappears completely.

Can robotaxis actually make money now?

Robotaxi unit economics are starting to work in real markets, but we still have no proof that a giant autonomous fleet can produce attractive company-level profits after paying for the technology and physical operation behind it.

Pony.ai currently gives us some of the clearest public numbers. Robotaxi revenue reached $12.1 million in its second quarter, compared with $1.5 million a year earlier. Fare-charging revenue rose by more than eightfold. The company also says its latest robotaxis have reached positive unit economics in parts of its Chinese operation.

That is meaningful progress. Paid rides are finally large enough that we can study actual transportation economics instead of estimating what a theoretical robotaxi might earn.

Yet Pony.ai's overall financials show why vehicle-level breakeven is only one milestone. During the same quarter, total company revenue was $36.2 million while operating expenses reached $72.1 million, including $56.2 million of R&D. Gross profit was only $6.4 million.

GM's decision on Cruise gives us the opposite case. GM concluded that scaling a robotaxi network would require too much additional time and capital and said restructuring Cruise would cut annual spending by more than $1 billion.

A successful robotaxi therefore needs enough paid rides per day to cover depreciation, sensors, compute, charging, cleaning, tires, repairs, insurance, remote assistance and local fleet infrastructure.

Current economics evidence	What it shows
Pony.ai robotaxi revenue: $12.1M in one quarter	Paying autonomous rides are becoming a real revenue line
Robotaxi revenue: +691% year over year	Commercial usage is scaling far faster than the previous base
Pony.ai operating expenses: $72.1M	Positive vehicle economics do not yet equal a profitable autonomy company
GM: >$1B expected annual savings from Cruise restructuring	Full-stack robotaxi development can consume extraordinary amounts of capital

Are autonomous vehicles still too expensive to build?

Autonomous vehicles are getting cheap enough to deploy in large fleets, although the cost of making every critical system redundant still keeps Level 4 cars well above ordinary production vehicles.

Pony.ai has set a useful benchmark. The company wants the full cost of a China-market robotaxi, including the vehicle and autonomous-driving hardware, to fall below RMB230,000 by the middle of next year. That would put a complete robotaxi much closer to normal passenger-car economics than the heavily modified research vehicles used earlier in the industry's development.

Production scale is already changing as well. Hundreds of seventh-generation Pony.ai robotaxis have entered daily operation, while Chinese manufacturers and autonomy companies are discussing vehicle programs measured in thousands rather than dozens.

Sensors deserve some perspective here. Lidar used to symbolize the supposedly impossible cost of autonomy because early units could cost tens of thousands of dollars. Automotive lidar prices have fallen dramatically, while purpose-built compute and centralized vehicle architectures are spreading across the industry.

The harder cost problem is redundancy. A Level 4 vehicle cannot simply beep and hand the steering wheel back when a camera, computer, power circuit or braking component fails. Removing the human fallback forces manufacturers to build fallback capability into the machine.

Hardware cost still matters, but utilization and operating cost now look more decisive.

Can robotaxi fleets expand city by city without becoming operationally messy?

Robotaxi fleets are expanding much faster today, but each new city still brings enough physical work that autonomy does not yet scale like ordinary software.

A ride-hailing company such as Uber usually does not own every car, charge it, wash it, replace its tires or repair its sensors. A vertically integrated robotaxi company may have to do all of those things while also repositioning vehicles, handling lost property, assisting passengers and responding to accidents.

The industry is already adjusting. Pony.ai has been pushing a joint-deployment model where local partners can provide vehicles, dispatch, customers and operational infrastructure while Pony.ai supplies autonomy. Its European agreement with Uber covers more than 2,000 planned robotaxis across five cities. In Doha, national transport operator Mowasalat runs the customer-facing service through its Karwa app.

Waymo has also used Uber in markets including Austin and Atlanta rather than insisting on controlling every part of the ride-hailing stack itself.

Waymo's geographic growth is accelerating. It moved from a handful of mature markets to fully autonomous rider service across 14 cities, while cities including London, Tokyo, New York, Boston, Chicago, Detroit and Seattle sit in its broader expansion pipeline.

Pony.ai is scaling from another direction. Its fleet reached 1,975 robotaxis by mid-year, with a target above 3,500 by year-end. The company says agreements under discussion across international markets cover more than 4,000 additional vehicles.

Factories can manufacture another 500 cars relatively quickly. Each new market still needs permits, local validation, charging, maintenance, dispatch, emergency-response procedures and enough demand to keep those cars busy.

The useful metric from here is launch time: how quickly companies can move from first local testing to broad driverless commercial service without rebuilding the operating stack each time.

If you want more recent data on this point, please see our latest autonomous vehicle market report.

Is regulation slowing autonomous vehicles down now?

Regulation is becoming one of the main limits on autonomous-vehicle expansion because companies can increasingly build driverless systems faster than governments can create one consistent set of rules for deploying them.

California makes the problem easy to see. Testing permission, driverless deployment and paid passenger service fall under different regulatory processes. Airports add another layer of approval. A vehicle can therefore be technically capable of running without a driver while still being unable to carry paying passengers in a particular place.

The current California passenger-service permits make the gap concrete. Waymo has driverless deployment authority, while Zoox and WeRide remain in driverless pilot categories there.

Other countries add entirely different approval structures. As Waymo prepares Munich, for example, local validation, German rules and European vehicle standards all become part of the deployment process. Pony.ai's operations in Doha likewise depend on a partnership with Qatar's national mobility provider.

Cybersecurity adds another regulatory issue as vehicles become software-defined. A Level 4 fleet depends on sensors, onboard computers, cellular communication, cloud services, remote support and over-the-air updates. UNECE has already built cybersecurity and software-update management into international vehicle regulation, while NHTSA treats cyber risk as part of automated-driving safety.

One software vulnerability can theoretically affect a whole fleet, and one poorly validated update can change driving behavior across many vehicles at once.

What the industry lacks today is a broadly harmonized approval path. Common safety-case standards, comparable crash reporting and clearer rules for software changes would make international expansion much easier without lowering the safety bar.

Can one autonomous-vehicle crash still destroy a whole robotaxi program?

A single autonomous-vehicle crash can still cripple a robotaxi program because regulators have to ask whether the same software failure could exist across every similar vehicle in the fleet.

Cruise remains the clearest example. Its 2023 pedestrian incident followed years of testing and expansion, yet the fallout eventually included California's suspension of driverless operations, a federal investigation into reporting practices and GM's decision to stop pursuing Cruise as a standalone robotaxi business.

The size of the reaction can look strange beside human-driving statistics. Human drivers cause enormous numbers of crashes every year without society reconsidering human driving as a category.

Autonomous fleets create a different risk. Thousands of vehicles can share the same perception model, planning code and software update. A strange failure in one car may therefore reveal a repeatable weakness rather than one person's mistake.

Centralization also gives autonomous vehicles an advantage. Engineers can diagnose a defect once and update the fleet. Human driving has no equivalent mechanism for instantly correcting the same behavior across thousands of drivers.

That makes transparency extremely important. Companies need detailed event logs, fast root-cause analysis and a credible way to remove or update vehicles when a new failure appears. Cruise damaged itself further when regulators concluded that important details about the pedestrian incident had not been properly disclosed.

Why do people still distrust self-driving cars?

People still distrust self-driving cars because safety data has improved much faster than public confidence, and most people have never experienced a fully autonomous ride themselves.

The latest J.D. Power U.S. Mobility Confidence Index makes that gap unusually clear. The overall confidence score sits at only 39 out of 100, essentially unchanged from 2024. Fewer than one in four respondents say they feel comfortable riding in a fully self-driving vehicle.

People are becoming more informed without becoming much more relaxed. In the same survey, 58% correctly understood what full vehicle automation means, up from 43% two years earlier. Safety remained the leading concern for 60% of respondents, emergency handling for 58% and difficult weather or traffic for 51%.

Direct experience appears to change the reaction dramatically. Earlier J.D. Power robotaxi research found average rider satisfaction above 8.5 out of 10. People who had actually ridden in a robotaxi also showed much higher confidence in full autonomy than people who had not.

Trucking exposes the contradiction even more clearly. In the latest survey, 46% were relatively comfortable with goods being transported autonomously, but only 16% felt comfortable sharing the road with fully autonomous semi-trucks.

Public acceptance currently looks more like an exposure problem than an information problem. Reading that an autonomous car is safer and sitting inside one while it calmly handles traffic produce very different levels of trust.

Are robotaxis easier to solve than fully self-driving personal cars?

Robotaxis are much easier to solve than genuinely self-driving personal cars because the operator can decide exactly where the vehicle works and keep tight control over every car in the fleet.

A robotaxi can reject a trip outside its validated area. It can stay parked during conditions its software has not yet mastered. A fleet operator can inspect sensors, clean cameras, check tire condition and withdraw a vehicle as soon as diagnostics detect a problem.

A privately owned self-driving car faces a much harsher expectation. Buyers will want the vehicle to handle unfamiliar roads, holidays, rural trips, winter weather and years of ordinary wear. They will not think much of a car that works beautifully in their home city but gives up halfway through a road trip.

This difference helps explain why Level 4 commercial fleets have moved ahead of broadly available consumer autonomy. Waymo can add one verified territory at a time. An owner expects the car to follow the owner.

Tesla is testing a different path by linking consumer-vehicle autonomy with a robotaxi model. GM went another way after winding down Cruise's standalone robotaxi expansion and moving more autonomous-driving work back toward personal vehicles.

For now, geofenced robotaxis look like the more tractable commercial problem. A personal car that can truly drive itself almost anywhere remains a much tougher standard.

If you want more recent data on this point, please see our latest autonomous vehicle market report.

Will autonomous trucks scale faster than robotaxis?

Autonomous trucks could scale faster on selected freight corridors because highway driving is more repetitive and the economics of removing driver-hours are especially strong.

Aurora shows why the business case can be attractive. Its autonomous network has expanded beyond the original Dallas-Houston corridor toward routes linking major freight markets such as Fort Worth, El Paso, Phoenix and Laredo. Long routes create a direct advantage because autonomous trucks are not constrained by a human driver's hours-of-service limits in the same way.

Kodiak is approaching the market from a particularly practical angle. Its customer-owned driverless fleet has reached 35 trucks, with more than 40,000 cumulative paid driverless hours and over 20,000 loads delivered. The company said this week that its safety case for driverless long-haul operations had reached 93% completion, up from 91% one month earlier.

Highway autonomy still carries serious risks. An 80,000-pound truck needs extremely reliable braking, perception and fallback behavior, while depots and city endpoints introduce much messier driving than the interstate itself.

Public acceptance is also weak. As seen above, J.D. Power found only 16% of surveyed consumers comfortable sharing the road with a fully autonomous semi-truck.

Still, trucking does not need to solve every road in America to create a large business. Repetitive freight lanes connecting major logistics hubs could support substantial autonomous volume long before a truck can drive itself from any warehouse to any destination.

Will better AI finally solve self-driving cars?

Better AI is speeding up autonomous-driving development dramatically, but it still cannot remove the need to prove that a physical system behaves safely when the world surprises it.

The technical direction across the industry has clearly shifted. Pony.ai is using large world models to generate and learn from a wider range of scenarios. Kodiak describes its newer approach around large neural architectures that combine perception with richer physical reasoning. Simulation systems can now expose these models to huge numbers of variations that engineers could never recreate manually on public roads.

That should help the long tail. Instead of writing explicit rules for every unusual object or road layout, models can learn general patterns from much larger datasets.

Transfer between cities may improve as well. A system that genuinely learns concepts such as occlusion, yielding and pedestrian intent should need less location-specific engineering than one built from thousands of handcrafted local rules.

Physical autonomy has a much harsher error budget than most generative AI. A model can produce a bad paragraph without hurting anybody. A driving model making a wrong assumption about a pedestrian, cyclist or stopped vehicle has almost no room for the same kind of error.

This is why verification remains so important even as the models improve. Companies need to know what changed after every major software update, which rare scenarios improved and whether something that previously worked became worse.

What are the biggest challenges facing autonomous vehicles now?

The biggest challenges facing autonomous vehicles today are geographic generalization, long-tail safety and economics; the industry has already moved well beyond proving that a car can drive itself under favorable conditions.

The safety evidence is stronger than many people realize. Waymo has accumulated hundreds of millions of fully autonomous miles, and independent IIHS research found a substantially lower comparable crash rate than human driving. We no longer need to speculate much about whether a Level 4 system can work safely at meaningful scale. We have evidence that one can.

The uncertainty begins when we ask how transferable that result is. Expanding into Munich still involves local mapping and validation. New climates change sensor conditions. New countries change road rules and driver behavior. Every extra market also brings permits, depots, charging, fleet support and emergency-response procedures.

Then comes the money. Pony.ai's robotaxi revenue is growing extremely fast and its latest fleets are beginning to report positive unit economics in parts of China. At the same time, the company still spends far more on operations and R&D than its robotaxi business generates. Cruise showed how expensive the gap between working autonomy and scalable autonomy can become.

Our judgment is sharp. The industry's central unsolved problem is now industrialization: preserving the capability while spreading it across more roads, climates, fleets and business models.

The winners now have to preserve today's best safety performance across much broader environments, launch new cities with progressively less local work and make each vehicle produce enough useful miles to justify the infrastructure around it.

If you want more recent data on this point, please see our latest autonomous vehicle market report.

Who is the author of this content?

NEW MARKET PITCH TEAM

We track new markets so founders and investors can move faster

We build living "market pitch" documents for emerging markets: AI, synthetic biology, new proteins, and more. Instead of outdated PDFs or hallucinated LLM answers, our clients get a clean, visual, always-updated view of what's really happening: key players, deals, regulations, and signals that matter. Learn more about us.