Anthropic has spent the run-up to its IPO arguing that AI is improving fast — but not so fast that its own safety rules have been tripped. On Thursday, the independent evaluator it invited inside said the math doesn't hold up.
According to Tech Times, METR, the nonprofit evaluator given inside access, found the company's models have been improving about 1.9 times faster since October 2025 than before. On Anthropic's composite capability index, scores climbed roughly 15 points a year before the break and about 29 points a year after it.
That number matters because Anthropic drew a line in its own safety policy. Its Responsible Scaling Policy sets a "2x Acceleration Threshold" — where AI progress roughly doubles, substantially because AI is automating AI research. Anthropic's August risk report said acceleration was less than a factor of 2. METR calls that argument "overstated": crossing by July 15 was plausible, though most researchers at both organizations lean toward not crossed yet.
The gap comes with caveats: robustness checks swing from 1.1x to 2.9x depending on one modeling choice, and including rival models pulls the estimate to 1.4x. METR guessed roughly 80% of the acceleration comes from AI itself, with substantial uncertainty. What it could say confidently is that faster training-compute growth did not drive the change.
The bigger problem is what nobody outside Anthropic can check. Anthropic held unilateral redaction rights over the roughly 100-page report and removed most of the evidence METR found most informative — pipeline estimates, bottleneck analysis, future-model forecasts — to keep rivals from benefiting, it said. It also cut a planned check of how it splits compute between capabilities and safety work midway through.
The timing could hardly be worse for Anthropic. Its amended IPO filing says future models "may imminently meet" the threshold, and Reuters reported Thursday that some investors doubt the roadshow — expected the week of November 9 — can still lead to a November listing. METR found no bottleneck that would slow progress in the next 12 months, and warned acceleration could speed up further over the coming year.
What to watch is whether Anthropic lets anyone check the next claim. CEO Dario Amodei has argued evaluators should publish key findings without the company's editorial control, and six METR employees are now embedded inside Anthropic under a separate October arrangement. The test will be whether the next assessment arrives with its evidence intact.
— via Tech Times