🔍 Read the full analysis: Claude Fable 5.2 Through Three Benchmarks: Revisiting The AI Math on ThorstenMeyerAI.com
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
A November 12 analysis of three benchmarks found Claude Fable 5.2 above fitted progress trends, with the clearest margin on freelance projects. Its lead on Epoch AI’s index falls within the index’s uncertainty range, while its RoboDojo result comes from Anthropic and has not been independently verified. The results do not show that the model can complete most tested freelance projects to client standards.
The analysis, published by Thorsten Meyer AI, compared Fable 5.2’s benchmark scores with trends among earlier record-setting models. It reports that the model scored 41.3% on the Remote Labor Index, against a predicted 27.2%; 173.4 on Epoch AI’s Capabilities Index, against a trend estimate of 171.1; and 64.9% on RoboDojo, 8.8 points above the authors’ fitted trend. These are comparisons to modeled trends, not direct measures of broad real-world performance.
The three results have different levels of support. For the Remote Labor Index, the report estimates that the gap would remain between 13.0 and 14.4 percentage points under alternative reasonable trend fits. The Epoch index’s published uncertainty range for Fable 5.2 is 169.9 to 177.5, which includes values below the trend estimate. RoboDojo’s score was reported by Anthropic rather than verified as a leaderboard entry; the analysis says alternative fitting choices put the gap anywhere from 3.8 to 23.3 points.
The report says its method used prior benchmark record-setters, excluded Fable 5.2 from the trend fits, and estimated the trend at the model’s November 9 release date. It used logistic curves for percentage-based results and a straight line for Epoch’s index. The published comparison chart gave a slightly different Remote Labor estimate—27.1% and a 14.2-point gap—which the authors say may reflect differences in model dates.
Claude Fable 5.2 Through Three Benchmarks
A November analysis found scores above historical trends, but the strength of the evidence differs across tests.
Three Different Signals
Scores exceed fitted trends, but the comparisons do not carry equal weight.
| Benchmark | Score | Trend comparison | Evidence |
|---|---|---|---|
| Remote Labor Index | 41.3% | 27.2% estimate | Freelance project results |
| Epoch AI index | 173.4 | 171.1 estimate | Lead within uncertainty range |
| RoboDojo | 64.9% | 8.8 points above trend | Reported by Anthropic; unverified |
Freelance Tasks: A Clearer Lead
The test measured whether substantial project work met a professional, client-acceptable standard.
Tasks came from 23 categories and 358 Upwork freelancers.
Reviewers judged these deliverables at least as good as professional work.
The result does not show that most tested projects met the standard.
How to Read the Results
- Remote Labor exceeded its fitted trend by 14.1 percentage points.
- The analysis estimates the gap remains 13.0–14.4 points under alternative fits.
- Epoch’s 169.9–177.5 range includes values below its trend estimate.
- RoboDojo’s gap varies with fitting choices, and its score lacks independent verification.
What Would Strengthen the Evidence?
Readers and businesses can watch for evidence that confirms results beyond this analysis.
- Independently verified RoboDojo results
- Further benchmark submissions
- Updated data and documented test procedures, model settings, and uncertainty estimates
Freelance Benchmark Shows the Widest Gap
The Remote Labor Index result stands out because it measures whether AI-generated work meets a client-facing standard across substantial projects, rather than performance on a short, isolated question. The test includes 240 projects from 358 Upwork freelancers across 23 categories, including game development, product design, architecture and data analysis. A project passes when trained reviewers judge the AI deliverable at least as good as a professional’s work—what a reasonable client would accept.
Even there, a score above trend should not be confused with a majority of work being automated successfully. Fable 5.2 passed 99 of 240 projects; most projects did not meet the stated standard. The report also notes that Fable 5.2’s rate is more than double its predecessor’s 17.92%, a notable benchmark change but not proof that every type of freelance work is similarly affected.
For readers tracking AI capabilities, the comparison offers a more qualified picture than a simple ranking. A strong result on one task suite, a marginal index lead, and an unverified vendor-reported robotics score should not be treated as equivalent evidence. The distinctions matter to companies weighing whether to assign work to models, researchers comparing progress, and workers assessing possible changes to their fields.
Products Worth Considering
Bose QuietComfort Wireless Noise-Cancelling Headphones, White Smoke
Kindle Paperwhite Signature Edition 32GB E-Reader, 7-Inch Display, Seaglass Green
Apple AirPods Pro 3 Wireless Earbuds with Active Noise Cancellation
As an affiliate, we earn on qualifying purchases.
How the Three Tests Differ
The Remote Labor Index was developed by Scale AI and the Center for AI Safety. Scale AI’s data says human freelancers spent an average of 28.9 hours per project, with a median of 11.5 hours, and projects averaged $632.60 in value. According to Anthropic’s system card, Scale AI ran the Claude models at high effort. The leaderboard lists Fable 5.2 at 41.3%, ahead of Claude Opus 5.5 at 21.25%, GPT-6 Astra at 20.83% and Claude Fable 5.1 at 17.92%.
Epoch AI’s index combines results from more than 50 benchmarks to track model capabilities as individual tests approach their score ceilings. Epoch cautions that the index’s absolute values do not carry meaning on their own, though comparisons between models can be useful. Its uncertainty interval for Fable 5.2 is particularly relevant here: the reported trend advantage of 2.4 points is small relative to the range of plausible scores.
RoboDojo measures performance on simulated robot tasks. The source analysis says Fable 5.2’s 64.9% figure is an internal Anthropic run, not a verified leaderboard submission; it also identifies a higher listed result, Q0-Preview at 68.9%. The analysis’s trend method compares each benchmark’s previous record-setters at their release dates, but the size of the RoboDojo gap changes substantially when the fitting approach changes.
““Absolute ECI values are meaningless by themselves, but meaningful comparisons can be made between models.””
— Epoch AI
Uncertainty Varies by Benchmark
The analysis is not a peer-reviewed study, and its trend comparisons depend on which earlier models are included, how release dates are assigned, and how the curve is fitted. The authors say four agent products on the Remote Labor leaderboard—three Manus versions and ChatGPT agent—were omitted because Epoch AI’s dataset did not provide release dates for them.
The Remote Labor Index leaderboard supplies no confidence interval for its automation rate. The report offers a rough sampling estimate of about plus or minus six percentage points for a 41% score across 240 projects; that is the authors’ estimate, not an official interval. The Epoch result is less decisive because its published range extends below the trend. The RoboDojo figure remains harder to assess without an independently verified leaderboard entry, and the report finds that its calculated margin changes widely with fitting choices.
The source article disclosed that it was researched and drafted with AI assistance, including Claude, the model discussed. It also said Anthropic, Scale AI, Epoch AI and the RoboDojo team did not review or sponsor the article. The available material does not establish how performance on these benchmarks translates to routine deployments or to tasks outside the tests.
Independent Verification Is the Next Test
The next useful evidence would include independently verified RoboDojo results, further benchmark submissions, and updated Remote Labor and Epoch data that allow researchers to test whether the reported gaps persist. The source material does not identify a scheduled verification or a new evaluation date.
Readers should watch for changes in the benchmark leaderboards and for documentation of test procedures, model settings and uncertainty estimates. Until then, the most defensible takeaway is limited: Fable 5.2 substantially exceeded the fitted Remote Labor trend in this analysis, while the Epoch lead is uncertain and the RoboDojo comparison remains dependent on an Anthropic-reported score and modeling choices.
Key Questions
What is the main finding about Claude Fable 5.2?
A November 12 analysis found Fable 5.2 above historical trend estimates on three benchmarks, with the clearest margin on freelance projects. The strength and reliability of the comparisons differ by test.
How did Fable 5.2 perform on the Remote Labor Index?
It passed 41.3% of 240 projects, or 99 projects, under the index’s client-acceptance standard. The analysis’s trend estimate was 27.2% on the model’s release date.
Does the Epoch index confirm a clear lead?
No. Fable 5.2 scored 173.4 against a trend estimate of 171.1, but Epoch AI’s reported uncertainty range for the model is 169.9 to 177.5, including results below the trend.
Is the RoboDojo result independently verified?
Not in the source analysis. It describes the 64.9% result as an Anthropic internal run rather than a verified leaderboard entry, and says reasonable trend-fitting alternatives produce widely different gaps.
Does the freelance score mean most projects were completed successfully?
No. The reported 41.3% pass rate means most projects did not meet the test’s client-acceptance standard, despite the result being well above the analysis’s historical trend estimate.
Source: ThorstenMeyerAI.com
