Which AI training data startups are becoming essential to model labs?

Why do frontier AI labs still need so many humans to train better models?

Frontier AI labs currently need humans most where their models are already strong, because easy labeling has lost value and genuinely difficult expert work has become the bottleneck.

A few years ago, a large training-data contract could involve thousands of people labeling images, moderating text or ranking fairly obvious chatbot answers. These days, the work increasingly goes to software engineers, scientists, doctors, lawyers, accountants and other specialists who can create problems that frontier models still struggle to solve.

The economics tell us how far this shift has gone. Reuters has documented Scale AI paying specialists to work on difficult generative-AI tasks that can cost tens of dollars per individual annotation. Handshake built an AI-training business approaching $1 billion in gross annualized revenue after connecting labs with graduate students and PhDs. Mercor says more than 30,000 experts across medicine, law, engineering and consulting now contribute to the human intelligence behind leading models.

The work itself has changed just as dramatically. Turing's recent Frontier Agentic STEM dataset contains 805 expert-authored tasks across physics, mathematics, chemistry and biomedicine. Turing says producing one task required an average of 58 hours of expert work. Its newer CyberStrike benchmark contains 200 security tasks where even the strongest tested frontier-agent configuration solved fewer than one-third.

That gives us a better definition of the market. AI labs are spending less attention on collecting huge quantities of obvious answers and more on finding the small number of people capable of creating problems that today's models cannot reliably solve.

When does an AI training-data company become essential to a model lab?

An AI training-data company becomes essential when removing it would noticeably slow a model lab's research and post-training work, rather than simply forcing the lab to buy labels somewhere else.

Revenue helps us measure this, but revenue alone is a poor test. A company can process hundreds of millions of dollars of contractor payments without owning anything particularly difficult to replace. We care more about whether the supplier repeatedly works with frontier labs, can recruit expertise the lab cannot easily assemble itself and increasingly owns the tasks, environments, benchmarks or verification systems around that expertise.

The strongest companies are all moving in that direction. Mercor now builds professional benchmarks and reinforcement-learning environments. Scale says almost half of its newer data-training projects involve RL environments. Snorkel shifted from selling labeling software to delivering finished datasets and simulated environments. Turing is building frontier-calibrated datasets and verifiers. AfterQuery focuses heavily on teaching agents how professionals actually complete work.

The closer these companies move to the experiment itself, the harder they become to swap out. Buying 100,000 labels is procurement. Building an environment that lets a model repeatedly practise a complex accounting workflow, verifies whether every step was done correctly and keeps generating useful failures for the next training run sits much closer to research infrastructure.

Is Mercor becoming the most important independent AI training-data company?

Mercor currently has the strongest overall case among independent AI training-data startups because it combines enormous spending from model labs with an increasingly serious research layer.

The commercial growth has been exceptional. Mercor's gross annualized revenue reached about $2 billion this year after being roughly $760 million at the end of the previous year. TechCrunch reported that it doubled in roughly four months on the way there. The company says it works with the five largest AI labs.

Those headline figures need one important qualification. Mercor operates partly like a marketplace, so a large share of customer spending goes to the experts doing the work. Gross annualized revenue therefore overstates what Mercor keeps. That does not weaken the demand signal, however: billions of dollars are now moving through one company from sophisticated AI buyers to human experts.

More interestingly, Mercor keeps moving deeper into model development. Its APEX benchmarks test AI on real work in accounting, software engineering, investment banking, consulting and corporate law. APEX-Agents requires models to operate across applications and produce professional deliverables. Mercor's latest accounting work also compares agents with real junior accountants rather than using another artificial academic benchmark.

We can see the strategy quite clearly. The expert marketplace supplies people who understand the work. Those experts help Mercor identify hard tasks, write rubrics and build environments. The resulting benchmarks reveal where models still fail, creating another source of training work.

Few competitors currently connect all of those pieces at Mercor's scale.

Mercor metric	Current evidence	What we learn
Gross annualized revenue	About $2B	Model-lab spending has reached very large scale
Expert network	30,000+ experts	Mercor can cover many professional domains
Frontier-lab reach	Says it serves the five largest AI labs	Demand is spread across competing labs
Professional benchmarks	Accounting, software, law, banking, consulting	Mercor can find capability gaps as well as produce data
RL environments	Already part of the offering	Mercor is moving deeper into post-training
Latest research direction	Human-vs-agent professional performance	The company is tying training data to real economic work
Main weakness	Marketplace-heavy economics	$2B of gross spend is much larger than Mercor's retained revenue

If you want more recent data on this point, please see our latest AI training data market report.

Is Surge AI still one of the companies model labs cannot easily replace?

Surge AI still belongs near the top of the AI training-data market because frontier labs have already shown that they will spend enormous amounts with the company over long periods.

Forbes reported that Surge generated around $1.2 billion of revenue in 2024 while remaining largely bootstrapped and profitable. Customers publicly associated with Surge have included Anthropic, Google, Meta, Microsoft and Mistral.

That history is important. Mercor, Handshake, Micro1 and Snorkel have exploded more recently, while Surge was already operating at billion-dollar scale before the current rush into professional training data became obvious. It managed to become deeply embedded with a remarkably small group of highly demanding customers.

Surge is harder to analyze than Mercor because it discloses much less about its research operation. We can see Mercor publishing APEX benchmarks, Scale publishing evaluation research and Turing continuously releasing frontier-calibrated task sets. Surge gives outsiders fewer ways to see exactly how its work has evolved.

We would still put Surge in the first group today. A billion-dollar relationship base with competing frontier labs carries more weight than a polished product page. The remaining uncertainty concerns how much of Surge's advantage now comes from research and environment design rather than exceptional execution of bespoke data projects.

How did Handshake become a billion-dollar AI training-data business so quickly?

Handshake became a major AI training-data supplier because it already controlled something model labs suddenly wanted badly: a huge, verified pool of highly educated people.

Handshake spent more than a decade building a university recruiting network before AI training became a meaningful part of the company. It now says that network reaches around 18 million students and alumni, including roughly three million graduate-level candidates and more than 500,000 PhDs.

When model labs began searching for specialists who could create doctorate-level physics problems, review medical reasoning or grade expert financial work, Handshake did not need to construct an expert network from zero.

The Information reported that Handshake's gross annualized AI-training revenue jumped from only around $5 million to $10 million shortly after launching the business to $550 million at the start of this year and nearly $1 billion a few months later. Sacra subsequently estimated total gross annualized revenue at roughly $1.1 billion.

The speed of that transition is hard to dismiss as a temporary side business. AI training has effectively transformed Handshake's economics.

What we still have to prove is whether Handshake can own more of the intellectual layer around the experts. Access to hundreds of thousands of PhDs is unusually valuable today. If Handshake also becomes good at constructing tasks, environments and evaluation systems around them, its old university network becomes far more difficult for newer competitors to copy.

If you want more recent data on this point, please see our latest AI training data market report.

Is Snorkel AI becoming a real frontier-model supplier?

Snorkel AI has become one of the clearest breakout suppliers to frontier model labs, with its new data business growing more than eighteenfold in roughly a year.

The freshest evidence is unusually strong. Snorkel recently raised $350 million at a $3.5 billion valuation and said its annualized revenue run rate had crossed $375 million. Reuters reported that the same business was generating only around $20 million a year earlier.

That growth followed a major change in what Snorkel sells. The company grew out of Stanford research and spent years selling software that helped customers create training data. It then launched a data-as-a-service business that delivers completed datasets and reinforcement-learning environments directly to labs.

The distinction matters when comparing Snorkel with marketplaces. Handshake and Mercor report large gross spending figures because money passes through them to contractors. Snorkel says expert costs sit inside its cost of goods sold because the customer buys the finished data product. We therefore should not put its $375 million next to Mercor's $2 billion and conclude that Snorkel is five times smaller in any strategically meaningful sense.

Snorkel's recent work also looks increasingly research-heavy. The company is involved in agent benchmarks and environments covering terminal use, software engineering and other complex workflows. It has started describing itself as a frontier AI data lab, and lately the operating results make that description much more credible.

Company	Recent scale indicator	Where the advantage currently comes from
Mercor	~$2B gross annualized revenue	Expert network, evals, benchmarks and RL environments
Surge AI	~$1.2B reported 2024 revenue	Deep relationships and high-end data operations
Handshake	~$1B AI-training gross annualized revenue	Huge university and PhD network
Micro1	>$500M annualized revenue	Rapid expert sourcing and data production
Snorkel AI	~$375M annualized revenue run rate	Finished datasets, research and RL environments
Turing	$300M reported revenue before its latest research expansion	Technical talent and verifier-heavy frontier datasets
AfterQuery	>$100M disclosed run rate earlier this year	Professional agent training and environments

Is Turing becoming important to model labs again?

Turing is becoming much more interesting to frontier model labs because its technical talent network is now producing some of the hardest verifiable training tasks we can find in the market.

Turing was better known for connecting companies with software developers. Reuters reported earlier that its revenue had tripled to around $300 million, with OpenAI, Google, Anthropic and Meta among the customers it named.

Its more recent work looks very different from ordinary engineering outsourcing.

Turing's Frontier Research Lab now builds datasets specifically calibrated so that today's strongest models still fail often enough to produce useful training signal. Its Agentic STEM collection has 805 tasks spanning 94 subdomains. The company reports an average of 58 hours of expert work per task. Its machine-learning and data-science set contains 304 tasks tested over 7,296 agent rollouts. KernelQuest adds 100 GPU-programming problems with human-certified performance targets.

The verification work is just as revealing. In KernelQuest, Turing found 170 cases among 1,200 agent runs where a system appeared to reach the required speed but failed other checks. A simpler benchmark would have counted those as successes.

That is exactly the kind of failure a frontier lab cares about now. Strong models are increasingly good at producing outputs that look correct. The harder job is proving that the result really is correct.

Turing's commercial scale remains below the biggest marketplaces, but its technical data is becoming unusually hard to commoditize.

Why is AfterQuery getting so much attention from AI labs?

AfterQuery has become one of the fastest-rising AI training-data startups because it started directly with professional reasoning, agent workflows and RL environments instead of spending years in commodity labeling.

The company said earlier this year that it had already passed a $100 million annualized revenue run rate within roughly 14 months of inception and that leading AI labs were using its datasets and environments. More recently, TechCrunch reported a financing valuing AfterQuery at around $3.2 billion, more than ten times the $300 million valuation attached to its Series A only a few months earlier.

That valuation jump is an investor signal rather than proof that AfterQuery is indispensable. The underlying product tells us more.

AfterQuery focuses on encoding how strong professionals make decisions and complete work across fields such as financial reasoning, software engineering and enterprise workflows. That places the company directly in one of the biggest unresolved problems in post-training: teaching agents to execute long sequences of professional actions rather than merely answer professional questions.

The company is still young enough that we should use a lower confidence level here than we do for Mercor or Surge. Model-lab enthusiasm can move extremely quickly, and a $3.2 billion valuation does not prove durable customer dependence.

Still, very few startups have moved from formation to nine-figure run-rate claims and serious frontier-lab work this quickly. AfterQuery has earned a place in the group we need to watch closely.

If you want more recent data on this point, please see our latest AI training data market report.

Is Micro1 becoming a serious competitor to Mercor and Handshake?

Micro1 is now a serious AI training-data competitor because its annualized revenue has moved from roughly $100 million to more than $500 million in less than a year.

TechCrunch reported the fivefold increase over about eight months. More recently, Micro1 raised roughly $100 million in a financing that reportedly valued the company around $4 billion. The customer base includes frontier AI labs alongside companies such as Microsoft, Amazon and robotics company 1X.

Micro1 originally built around recruiting and vetting technical talent, which gives it a route similar to Mercor and Handshake: use software to find unusually capable people, then turn their expertise into training material.

The company is now extending that model into expert datasets, evaluations and training environments rather than simply supplying contractors. That direction makes strategic sense, but Micro1 has less publicly visible research output than Mercor, Turing, Snorkel or Scale.

So we would separate commercial momentum from indispensability. Micro1 already has the first. The second will depend on whether labs start relying on infrastructure, benchmarks or specialist networks that they cannot easily reproduce elsewhere.

Is Scale AI still essential after becoming so closely tied to Meta?

Scale AI remains essential to parts of the frontier-model ecosystem, but its Meta relationship has weakened the one advantage that once made Scale uniquely powerful across the entire market: neutrality.

Meta invested roughly $14 billion for a 49% stake in Scale and brought founder Alexandr Wang into Meta to lead major AI work. Scale remained a separate company, yet rival labs immediately had reason to reconsider how much sensitive research they wanted running through a supplier tied so closely to one competitor.

Reuters reported that Google, previously Scale's largest customer and expected to spend around $200 million with the company that year, planned to move work to other suppliers. OpenAI had also been reducing its reliance on Scale.

Scale's AI-data technology has not faded. Quite the opposite. Scale said earlier this year that nearly half of its new training-data projects involved reinforcement-learning environments. More recently, Scale Labs released AgentEnv, the framework it uses to construct environments for agents, and continued publishing work around coding benchmarks, reward hacking and agent evaluations.

Scale therefore remains one of the technically deepest companies in the category. Its position has simply changed. Meta can treat Scale as a highly strategic piece of its AI operation, while Meta's competitors have an extra reason to keep Mercor, Surge, Handshake, Snorkel and other independent suppliers healthy.

The episode also taught us something broader: model labs consider the information flowing through data suppliers sensitive enough that ownership structure can change procurement decisions.

If you want more recent data on this point, please see our latest AI training data market report.

Why are AI labs buying from several training-data companies at once?

Frontier AI labs currently need several training-data suppliers because no single company has every expert, every environment or every type of data they need, and relying on one vendor creates unnecessary strategic risk.

The spending pattern backs that up. Mercor, Handshake, Snorkel and Micro1 have all expanded rapidly during the same period, while Surge and Scale remain very large businesses. One supplier has clearly not been taking all the growth.

The actual training problems explain why. A lab building a coding agent might need thousands of experienced software engineers, terminal environments and automatically verifiable tasks. Medical reasoning requires doctors and specialist reviewers. Professional agents need accountants, lawyers or bankers who understand how real deliverables are produced. Cybersecurity requires experts capable of building attack and defense environments where success can be measured objectively.

Different suppliers have different recruiting networks and research strengths, so labs can route work accordingly.

There is another reason to spread spending around. Training-data orders can reveal what a lab is working on. A sudden request for thousands of high-level chemistry tasks, complex browser environments or accounting workflows says something about the capabilities the lab is trying to improve. The reaction to Meta's Scale investment showed that customers care about who can see those requests.

As seen above, Mercor's roughly $2 billion gross run rate coexists with billion-dollar-scale Handshake and Surge businesses and very fast growth at Snorkel and Micro1. That looks much more like a portfolio of important suppliers than a market converging on one universal vendor.

Model-lab problem	Suppliers with a strong position today	Why several suppliers can coexist
Large expert workforce	Mercor, Handshake, Surge, Micro1	Their professional networks differ
Scientific and technical tasks	Turing, Mercor, Snorkel	Deep domain knowledge is difficult to generalize
RL environments	Scale, Mercor, Snorkel, AfterQuery, Micro1	Labs need many different software and workflow environments
Professional agent training	Mercor, AfterQuery, Micro1	Each profession creates different workflows and verification problems
Large bespoke data programs	Surge, Scale	Operational reliability matters at very high volume
University and research expertise	Handshake	Its academic network is unusually difficult to reproduce
Frontier benchmarks and evals	Scale, Mercor, Turing, Snorkel	Labs need independent ways to expose new model failures

Why are RL environments becoming such a big part of AI training data?

RL environments are becoming central to AI training because frontier models increasingly need to learn how to complete work inside software, where a correct final sentence tells us very little about whether the agent can actually do the job.

Imagine training an accounting agent. A useful task may require the model to inspect emails, open several spreadsheets, identify discrepancies, modify the right files, calculate an answer and leave the work in the correct state. The environment needs realistic software, realistic data, resettable starting conditions and a reliable way to judge whether the model completed the task properly.

That is much harder to build than a question-and-answer dataset.

Scale's disclosure that almost half of its newer training-data projects involve RL environments shows how quickly procurement has shifted. Mercor, Snorkel, AfterQuery and Micro1 are all pushing into the same area independently, while Turing increasingly delivers tasks inside executable terminal environments.

The economics could become attractive for the best providers because environments can create repeated training episodes. Once a difficult professional workflow has been reconstructed properly, a lab can run models through variants of it again and again rather than paying for one static answer.

This also gives suppliers more ways to become embedded. Replacing a vendor that delivered a batch of text examples is relatively straightforward. Replacing hundreds of tested environments, verifiers and task-generation pipelines woven into the lab's reinforcement-learning process is a much bigger job.

Will synthetic data eventually kill the human training-data boom?

Synthetic data will probably shrink a lot of routine human labeling, but it currently makes top human experts more valuable rather than making companies such as Mercor, Snorkel or Turing irrelevant.

Frontier labs can already use models to generate enormous numbers of examples, variations and candidate solutions. That changes the amount of manual production required.

The difficult part has moved elsewhere. Someone still has to decide which problems are worth generating, whether the synthetic tasks contain hidden shortcuts, whether the proposed answer is genuinely correct and whether the reward function is teaching the model the behavior researchers actually want.

Turing's recent benchmark work gives us a good concrete example. In KernelQuest, many agent runs appeared to satisfy a speed target but failed deeper verification. Synthetic generation can cheaply produce more code attempts, while expert-designed checks are what separate real progress from a model exploiting an incomplete metric.

Snorkel is explicitly combining experts with synthetic generation and model-driven quality control. That hybrid approach is likely closer to where the industry ends up than either extreme of “humans do everything” or “models generate all their own training data.”

Human labor per training example may therefore fall substantially. Spending on the people who design the hardest tasks, construct reliable verifiers and catch sophisticated failures can still rise at the same time.

Which AI training-data startups are actually becoming essential to model labs?

Mercor, Surge AI and Handshake look closest to essential independent suppliers today; Snorkel AI and Turing are moving into that group through deeper technical work, while AfterQuery and Micro1 have become the strongest fast-rising challengers.

Mercor has the best combination of scale, breadth and visible research progress. The company already handles enormous spending from labs while building benchmarks, evaluations and reinforcement-learning environments around real professional work.

Surge earns its place through proof rather than publicity. It built a billion-dollar business serving a small set of highly sophisticated AI customers well before the latest training-data boom. Until those customers demonstrate that they can easily replace it, we should treat that history seriously.

Handshake has a different advantage. Hundreds of thousands of PhDs and millions of graduate-level people already sit inside a network that was built long before frontier labs desperately needed them. Its next challenge is turning that supply advantage into deeper technical infrastructure.

Snorkel is currently making the fastest case for promotion into the top group. Its new data business grew more than eighteenfold in roughly a year, its run rate has passed $375 million, and its product has moved directly into finished datasets and simulated training environments.

Turing looks strongest where technical verification matters most. Its recent scientific, coding, cybersecurity and machine-learning datasets are precisely the sort of problems frontier models still fail despite looking increasingly competent on older benchmarks.

AfterQuery and Micro1 deserve more caution because their histories are shorter. Both have grown remarkably quickly, but we still need more evidence that labs depend on assets unique to them rather than simply buying large quantities of high-quality work while demand is booming.

Scale sits slightly outside this ranking. It remains one of the deepest AI-data organizations in the world and is especially strategic to Meta, but its close relationship with Meta has encouraged rival labs to build alternatives rather than letting Scale become the universal supplier it once appeared capable of becoming.

So the market today is producing several essential suppliers rather than one new monopoly. The companies pulling ahead are the ones that can repeatedly find a frontier model's next weakness, bring in humans who genuinely understand the problem, turn their expertise into a trainable environment and prove that the model got better afterward.

If you want more recent data on this point, please see our latest AI training data market report.