The Data Supplier Behind the Models: Surge AI

I first heard of Surge AI from a podcast interview with Edwin Chen, right as they were raising their first round of funding. A few of Edwin’s points left a strong impression on me [1]:

  • He dislikes playing the Silicon Valley hype game, preferring to do the right thing over things that merely look impressive
  • He opposes the approach of throwing labor at data labeling, instead focusing on producing the highest-quality data and improving efficiency by optimizing the process
  • He points out that public benchmarks (such as LMArena, which is based on user preferences) don’t work

I strongly agree with his views. For one, I also dislike formulaic startup paths. On top of that, I once spent a stretch of time working on AutoML, where I trained many models intensively to push performance, so I know firsthand how important understanding the data is to a model’s results. Out of interest in this company, this article surveys Surge AI’s development, team, business, and core views.

Surge AI’s Development and Team

Surge AI was founded in 2020 (after the release of GPT-3). Unlike the path most Silicon Valley companies take, founder Edwin didn’t take VC money; instead, he built the company with the savings from his job at a big tech firm, and made it profitable from the very start. In the interview, Edwin said he dislikes Silicon Valley’s status game—fundraising, hype, tweeting and building cliques, and so on—and that he wanted to do the right things on his own terms, without being beholden to anyone after taking VC money.

From the beginning, Surge AI offered high-quality data labeling (unlike cheap, large-scale crowdsourcing). Later it added RLHF data and workflows, along with professional services such as quality control, complex rubric design, domain-expert labeling, fast experimentation interfaces, and red-teaming tools. This full stack of capabilities was deeply involved in training Claude: Anthropic needed to scale up human feedback for RLHF, but was bogged down by the drudgery of “where to find qualified labelers, how to build a quality-control system, how to maintain the labeling tools,” which slowed the pace of research. Surge AI took over this entire pipeline [2]—using its own human-in-the-loop algorithms for quality control (because language models are “extremely sensitive” to low-quality data, and a single bad batch can set research back), supplying expert labelers spanning math, coding, law, medicine, and business, and opening up directly to customers the RLHF methodology distilled from hundreds of prior internal experiments.

The company’s 2024 revenue had already surpassed 1billion,andinJuly2025itplannedtoraise1 billion, and in July 2025 it planned to raise 1 billion for the first time, at a valuation of around $30 billion.

CEO Edwin graduated from MIT and previously did data-science and machine-learning work at Twitter (search/ad quality), Google, and Facebook/Meta, giving him a deep understanding of data. The whole company’s headcount is extremely lean: the core team is fewer than 100 people, focused on building the toolchain, the platform, and evaluation systems. Counting part-time staff and consultants, it’s around 250 people. Beyond that, Surge AI also partners with other contractors to handle labor-intensive tasks.

Business and Case Studies

There isn’t much public information about Surge AI, but a few case studies can be found on their blog.

GSM8K, Built for OpenAI (2022)

This is a dataset of 8,500 grade-school math problems, all of them word problems, for example:

After Bobby gets his paycheck, Darren will have twice as much money as Bobby. Bobby currently has 40andwillreceivea40 and will receive a 16 paycheck. How much money does Darren have now?

GSM8K served OpenAI’s reinforcement-learning team in training models’ mathematical reasoning, so the constraints on each problem were very specific: every problem needed 2 to 8 solution steps, used only the four basic arithmetic operations, involved calculations small enough to do in one’s head, and had an integer final answer—while the scenarios had to be diverse enough to avoid repetition [3]. To ensure quality, Surge AI prioritized recruiting problem-writers with STEM degrees and solid math skills, and had each writer’s first five submissions reviewed by another qualified Surger. Two pipelines guarded quality—correctness and diversity. For correctness, every problem was solved independently by two Surgers, and any time their answers disagreed, the team focused on checking whether the wording was ambiguous. For diversity, sentence embeddings were computed for the problems and pairwise cosine similarities taken, dropping any problem above a similarity threshold to prevent scenario collisions. The dataset was later widely adopted by papers such as Google’s PaLM and Chain-of-Thought, becoming one of the foundational benchmarks for measuring mathematical reasoning.

Training and Evaluating Claude (2023)

In Claude’s training and evaluation, these Surge AI features were used [2]:

  • Quality control: human-in-the-loop algorithms intercept low-quality samples before they enter training
  • Domain-expert labeling: professional labelers across math, coding, law, medicine, business, STEM, and more, letting the model learn high-quality preference feedback in each domain
  • Fast experimentation interfaces: an API and RLHF interface let researchers launch labeling tasks and iterate on experiments quickly, without first spending a lot of time polishing labeling guidelines
  • Red-teaming tools: a dedicated team proactively probes the model for safety vulnerabilities

In other words, what Surge AI delivered wasn’t just “labeling data,” but the entire infrastructure RLHF requires—labeler recruitment, quality control, an experimentation platform, and red-teaming, all packaged together—so that Anthropic’s researchers could focus on the research itself.

AdvancedIF, Built for Meta Superintelligence (2025)

Today’s public benchmarks are actually disconnected from real-world scenarios; everyone focuses on metrics that are easy to evaluate (such as character counts) rather than metrics that are genuinely useful. In building AdvancedIF, an instruction-following evaluation system, together with Meta Superintelligence Labs, Surge AI followed these principles [4]:

  • Build true evaluation targets rather than simple proxy targets (for example, a programmatically verifiable constraint like “avoid the letter C” is easy to measure, but genuine writing and instruction-following ability is hard)
  • Have prompts and scoring rubrics written entirely by human experts rather than synthesized by an LLM
  • Make the evaluation flexible enough, instead of treating it as a simple multiple-choice question
  • Treat complex multi-turn conversations as a feature, because users are also messy in each turn (for example, changing a constraint midway, or asking the model to update its plan based on earlier context)

To let these human-written rubrics score at scale, the team trained a verifier on thousands of expert annotations; its F1 agreement with human scoring reached 0.728, 41% higher than scoring directly with an LLM. The payoff was direct too: Meta used these human-written rubrics as the reward signal for RL to train Llama 4 Maverick, and the model achieved a 6.7% absolute improvement on AdvancedIF, generalizing even to other benchmarks it hadn’t been trained on. Notably, even frontier models still fail these real instruction-following tasks 22%–30% of the time, showing that the gap between “optimizing for character counting” and “optimizing for real intent” remains large.

Other Model Evaluation Analyses

Beyond the customer case studies above, Surge AI’s blog has plenty of interesting analyses, such as model evaluations on finance and coding tasks.

In the finance evaluation, Surge AI brought in experts—former traders, investment bankers, and risk managers—and designed more than 200 real-world scenarios across seven domains, covering Basel capital calculations, commodities trading, PowerPoint/Excel file manipulation, and more, testing GPT-5, Claude Sonnet 4.5, and Gemini 2.5 Pro [5]. The results weren’t pretty: GPT-5 led with a 47% success rate, Claude at 26% and Gemini at 24%, and in expert scoring over 70% of the responses fell into the “mediocre to poor” range. Getting specific: asked to produce a slide deck on financial risk, Gemini claimed it had generated the slides but couldn’t produce them, Claude omitted required sections, and only GPT-5 completed the whole deck—though it lacked an explanation of risk mitigation; asked to build an Excel financial forecast, Gemini simply fabricated calculations and returned a broken link, Claude broke a formula midway, and GPT-5 got the numbers right but wiped out all the formatting; and on Basel capital modeling, all three models used the wrong loss rates and missed the offset rules for hedging trades. What these models lack isn’t textbook knowledge, but the “street smarts” to bring that knowledge to bear in professional practice.

In the coding evaluation, Surge AI used SWE-Bench’s Bash-Only setting (solving real GitHub issues using only shell commands, where even the strongest models pass only around 67%) to test Gemini 2.5 Pro, Claude Sonnet 4, and GPT-5 [6]. The task itself was just to fix a bug where astropy’s HTML table writer ignored the formats argument—the correct fix takes only two lines of code—but the three models’ hallucination trajectories were completely different. After seeing truncated file output, Gemini invented a nonexistent BaseWriter class and a pile of fictional methods, and over 39 turns and 693 lines of changes chased hallucinated line numbers (claiming lines 362–365, when it was actually line 440) in circles, insisting “the core logic of my fix is correct,” ultimately failing to recover. Claude made a similar inheritance-relationship error early on, but realized the runtime error signaled a deeper problem, so it stopped to re-evaluate and re-investigate, and finally found the correct fix. GPT-5 avoided hallucination entirely by explicitly verifying its assumptions: on hitting a truncated file, it first flagged the information gap and then dug in, solving the problem cleanly on the first try. The three failure modes can be summarized as: failing to recognize missing information, lacking a verification mechanism, and doubling down on a wrong assumption.

Comparison of hallucination behavior across GPT-5, Claude, and Gemini on finance and coding tasks
Comparison of hallucination behavior across GPT-5, Claude, and Gemini on finance and coding tasks

There’s an even simpler example [7]: a teacher named Josephina just wanted to extract the body text of Section 4 from a government PDF into Word, stripping out footnotes and captions—and both ChatGPT and Gemini botched it. ChatGPT first returned a “summarized version” instead of the full text without asking; then its parsing code spat out content beginning with garbled “47,” repeatedly forgot the “remove footnotes” requirement, and every fix spawned a new problem (fixing the spacing broke the line breaks, deleting footnotes broke the formatting); after 25 minutes of fiddling, the teacher gave up. Gemini was even more absurd: with no file uploaded, it hallucinated that it was analyzing random academic papers, fabricated a nonexistent Word download link, and even went off to access a random Google Drive file. It all came back to the same old problems: not verifying tool output, not proactively asking about ambiguities, not remembering instructions across turns, and starting to make things up whenever uncertain.

Core Views

Public Benchmarks Have Many Problems

Take LMArena: you enter a question, evaluate two answers, and then pick the best one.

This actually optimizes for human attention rather than for ground truth, because ordinary people don’t fact-check—whichever answer is longer or has flashier emojis gets the vote (ChatGPT has this mechanism too). That’s actually quite unreasonable, yet it’s hugely popular in the community and treated as gospel; customers and investors look at performance on this leaderboard, which forces developers to optimize toward an unreasonable evaluation system [8].

The evidence Surge AI offers is solid too: after reviewing Arena’s votes, they disagreed with 52% of the samples and strongly disagreed with 39% of them; voters would prefer answers with hallucinations (such as a wrong quote from The Wizard of Oz) and mathematically impossible plans (such as a cake pan whose dimensions don’t add up). Worse still, the mechanism can be gamed—big labs A/B test dozens of variants and pull in a disproportionate share of battle data (Google and OpenAI each account for about 20%, while 83 open-source models combined make up just 30%), and can exploit the roughly 9% of monthly duplicate prompts to overfit the leaderboard; Meta’s Maverick would even fob off simple questions with emoji-laden “non-answers.” On top of that, many other benchmarks have a host of problems too: 30% of the answers in “Humanity’s Last Exam” are wrong, and 36% of the answers in HellaSwag are wrong. If even the evaluation metrics have problems, then the models trained against them won’t be good either.

Today’s Agents Are Still Far From Human Common Sense

Surge AI dropped nine frontier models into a simulated PC retailer, “Corecraft, Inc.,” and tested them on 150 customer-service tasks, distilling a tiered framework of agent capabilities [9]. If we frame these capabilities into several tiers, none of today’s models reach human-level common-sense reasoning:

  • Basic tool use, planning, and goal setting (breaking a task into steps, picking the right tools, executing in order)
  • The ability to adapt to real-world conditions (whether it can switch strategies after a first attempt fails; for example, if a customer makes a typo in a query, whether it handles it correctly)
  • Staying grounded in the actual environment (staying tightly aligned with the current conversation’s date and context, not fabricating details out of thin air)
  • Common-sense reasoning (inferring unstated needs; for example, in a customer-service task, proactively looking up the membership tier based on the account information the user provides, in order to offer personalized pricing)
Agent capability tier framework: from basic tool use to human-level common-sense reasoning
Agent capability tier framework: from basic tool use to human-level common-sense reasoning

The test results bore out the framework: even GPT-5 and Claude Sonnet 4.5 failed more than 40% of the agent tasks, with failure points spread across different tiers—weak models couldn’t even use basic tools (for example, Nova 1 Pro passed “gold” straight in as a customer ID instead of first calling search), while strong models tripped up on higher-tier reasoning (for example, Gemini would just accept an empty search result rather than trying different parameters; Kimi K2 flipped the date back and forth between 2024 and 2025 mid-task; GPT-5 failed to infer from context that “Sarah Kim” referred to that very customer). In other words, today’s models can execute a coherent plan, but still struggle to genuinely reason about unfamiliar situations—still far from human-level autonomy.

References

  • Business