AI正大幅加速AI研究，安全准备不足

This post discusses a public METR report on AI R&D acceleration at Anthropic. I did not work on this project or see the unredacted report.1 All opinions are my own and do not represent METR or the report authors.

Thanks to Ryan Greenblatt for discussion, feedback, and help with modeling, and Daniel Kokotajlo, Eli Lifland, and Hjalmar Wjik for comments.

Earlier this week, Anthropic stated in their S1 that they “may imminently meet” the threshold of “dramatically accelerated progress due to AI” as defined in their Responsible Scaling Policy. In their latest RSP (effective July 2026), say this threshold will be met when:

- They observe or expect overall capabilities progress to be going twice as fast as the fastest sustained rate before AI was contributing much to AI R&D.2

- This is plausibly attributable to AI automating AI R&D, rather than acceleration in compute or other inputs.3

For the last few months, some of my colleagues at METR have been working on a deep-access investigation to understand how much Anthropic is being accelerated by its AI systems. From my reading of their report and Anthropic’s RSP, it looks like the “dramatic acceleration” threshold has already been crossed.4

The report authors found that:

- Between October 2025 and October 2026, capabilities progress at Anthropic has been going about twice as fast as it had been over the previous twelve months. The measurement they report is an internal version of the Epoch Capabilities Index called AECI; a wide range of other supporting evidence for acceleration has been redacted. They said that this redacted evidence was "a majority of the evidence [they] found most informative."

- While the pace may have initially started picking up for a mix of reasons, AI automation is probably the large majority of the reason why capabilities are still improving at this faster pace today. Their reasons for believing this have been redacted.

This is a much bigger deal than it might sound at first. I think this report is the most important update we have gotten about AI risk in the last couple of years — much bigger than the Hugging Face attack. Naively, it points toward much shorter timelines than most external researchers (including the AI 2027 team) have projected.

Researchers must be sped up a whole lot

Importantly, if AI is roughly doubling capabilities progress overall, it must be speeding up human researchers by much more than that.

Researchers work on improving algorithms and architectures; Anthropic has sometimes referred to these innovations as “compute multipliers.”5 Overall AI capabilities (“effective compute”) is the product of compute multipliers and training compute. Because the compute multipliers are only one input into capabilities, this means that if AI capabilities are advancing twice as fast overall because of software progress, the rate of growth in compute multipliers would need to be more than twice as fast: perhaps ~2.5-3x faster based on reasonable assumptions.6 Since there’s uncertainty about the size of speedup and how much of it comes from AI, I’ll go with ~2-3x.

Similarly, researcher labor is only one input to discovering these compute multipliers: they need to test their ideas, which requires compute for experiments. If we assume a very simple Cobb-Douglas model where experiment compute and labor are similarly important, that means the uplift to labor roughly needs to be the square of the acceleration in compute multipliers.7 This would imply that we would need a (~2-3x)^2 = ~4-9x speed up in labor to make overall AI capabilities go twice as fast.8

However, once capabilities start speeding up significantly, this also means that low-hanging fruit is plucked at a correspondingly faster rate. Each new compute multiplier takes more and more work to discover and implement (i.e., “ideas are getting harder to find”). Before there was much acceleration from AI, labor roughly grew at some roughly fixed exponential rate from hiring more humans. Combined with growth in compute, this compensated for diminishing returns enough to maintain the baseline rate of capabilities progress. But to maintain a new faster rate of capabilities progress without scaling compute faster, we would need a new faster rate of growth in overall labor.

Since the human labor force is stuck growing at more or less the old rate, the uplift from AI would need to get larger and larger over time just to sustain capabilities progress at its new rate. That means the uplift today, after several months of accelerated progress from AI,9 needs to be even higher than ~4-9x, the estimate for what it would take to double progress at the start: maybe the necessary labor uplift is ~10x. This is very uncertain, and substantially more or less is plausible. But it lines up surprisingly well with informal self-reports from Anthropic staff, who say they are now getting a ~12x uplift from the use of internal models.

When you’re picturing a 10x uplift, you should not be imagining that every researcher is now a team of ten researchers — as anyone who has ever done anything knows, a team of ten does not get ten times as much valuable work done per hour as one person. Instead, it’s as if every researcher at Anthropic is in a time warp bubble where they experience ten days for every day that passes in the outside world. As if AI lets them think, read, speak, type, and do everything else ten times faster.10 Anthropic probably wouldn’t have been able to speed up its research this much by hiring any number of people.11

This could compound on itself extremely rapidly

A reasonable default expectation is that as AI capabilities continue to improve, AI researchers will be sped up more and more from automation: if today it takes them a month to get as much work done as they used to in a year, soon it will take a week, and then perhaps a day. If this increasing labor uplift is strong enough to overcome diminishing returns, then in the absence of other bottlenecks it would translate into even larger speed ups in the rate of AI progress: progress will start going 4x faster, and then 8x, and then 16x, and so on.

The report authors were very uncertain about this question, and say that their picture changed several times with new information. But ultimately, they did not find evidence of bottlenecks or steep diminishing returns that would get in the way of compounding acceleration.12 They also report seeing some (confusing) redacted evidence indicating that acceleration could already be above 2x and increasing over time.

If we take it at face value that AI is now nearly doubling the rate of AI capabilities progress and assume acceleration from AI is smooth, this naively implies that we would reach extreme superintelligence within a year. It seems that around a year’s worth of AI progress got us from modest acceleration to the point where AIs now make AI progress roughly twice as fast. If acceleration from AI is smoothly increasing, then at this new rate, we should expect that we will double the rate of progress again within six months. At that point, it would take three months to double it again, then six weeks.13

In theory, these doubling times would add up to infinite capabilities within one year. In reality, at some point we would reach the effective ceiling on software efficiency. But I expect these limits to be very far from where we are now.

If we go through this feedback loop, I expect that the AI systems that come out the other side will be able to very quickly develop novel weapons and very easily overpower human civilization in a conflict. If these systems were seriously misaligned, I think they would likely disempower humanity, and this could lead to human extinction.

The industry is not prepared to handle this safely

The “dramatically accelerated progress due to AI” is such an important and extreme milestone because it is a strong concrete indicator of extreme capabilities in the very near term. Indeed, in an earl