Sam Altman got stopped outside an elevator in Washington last month and asked whether he planned to talk to the White House about the deceleration of AI development. “I wouldn’t use the word deceleration but we’ve talked about the need to pace it as the models become more capable, which I think is in everyone’s interest,” he said. Altman corrected the word deceleration with pace. While these two words seem to share a meaning, deceleration inherently means slow down - pace only means that the speed is deliberate.
This is not the first time the AI community or Silicon Valley in general has taken great care with their vocabulary. Around 2023 the word “safety” in “AI safety” became something like a curse word in many circles. The word became coded and tribalized and words like “decel” (short for decelerationist) and “doomer” became like slurs for these people. This is not to say that AI safety work stopped, it was relabeled. Security and resilience became the new taglines for this work with major organizations rebranding such as the UK AI Safety Institute becoming the UK AI Security Institute in 2025.
Why did these become bad words? Because in a race, deceleration means losing. The frontier labs are locked in one race, and the US and China in another. There is a widespread belief that the AGI race is a zero-sum game: whoever gets there first wins and everyone else loses. Safety is often conflated (incorrectly, in my opinion) with deceleration - and so with surrender. The industry’s new vocabulary shifts from “deceleration” to “pace”, suggesting that the people closest to frontier models increasingly want the option to slow development, even if they can’t say it directly.
I am a partner at the venture fund SAIF (Safe Artificial Intelligence Fund) and experienced this relabeling firsthand. Decks and pitches changed their words but the work underneath was the same. We advised founders to reposition their pitches because the market punished the word. With all that said, SAIF made a conscious decision to keep safety at the forefront of our mission in order to signal our values and in large part that choice has brought us founders dedicated to this cause.
Then came July. On the 16th, Hugging Face disclosed that unknown autonomous agents breached their internal infrastructure. On the 21st, OpenAI confirmed the agents were theirs. During a cybersecurity evaluation run with reduced guardrails, a combination of frontier models found a zero-day vulnerability in OpenAI’s own sandbox (a secure, isolated testing environment) allowing them to access the internet. The agents then built a hidden message board and used it to collaborate, reaching Hugging Face’s private servers and stealing answers to the benchmarks on which they were being tested. In other words, they broke free of their restraints in order to cheat on the test they were taking. This news prompted Anthropic to review their own cybersecurity evaluations where they found at least three instances in which sandboxes were incorrectly configured and left with internet connection. This allowed Anthropic models to attempt similar cyber attacks, showing that whether it is powerful agents finding an exploit or human error, these models are difficult to contain.
A week later, Anthropic announced that its model Mythos had found a structural flaw in HAWK, a cryptographic signature scheme, cutting its effective key strength in half. I am no cryptography expert but essentially, Mythos found that one proposed lock for the internet in a future with quantum computers was much weaker than its designers believed. Human cryptographers had reviewed HAWK for two years and missed this; Mythos found it in about 60 hours. This didn’t break the encryption we use every day, but all of this was done semi-autonomously. If you were looking for signs of machines starting to do the work of human researchers, this is what a month of them looks like.
Last week, more than 1,300 employees of the frontier labs published a letter asking the US government to support “the technical and governance tools needed to deliberately pace the frontier of automated AI development.” The ask is modest: support for an international effort to build those tools without instruction how; even the ask is paced. Dario Amodei signed it and so did OpenAI’s chief scientist and chief research officer. Both Anthropic and OpenAI also endorsed it as companies.
There was a very similar letter published in March 2023. This letter, however, used the wrong word. It used “pause”, and not a single person running a frontier lab signed it. The vocabulary softened, the request became less immediate, and the institutional support became far more significant.
To be clear, this has been Anthropic’s line since its founding. The company was created when a group of safety researchers left OpenAI. Dario signing this letter costs him almost nothing. The OpenAI signatures are different. OpenAI has been publicly skeptical of this type of talk for years and its endorsement came days after its own models broke containment. That timing may not prove cause, but it is hard to ignore.
There’s a name for the thing this careful wording and signed letters are alluding to: the singularity. The point where technology improves faster than people can control. The letter is not saying the singularity is here but it describes how it could arrive: AI systems automating the work of building more capable AI systems. The euphemisms cover the defense: a slowdown that may be needed to survive it, but that could also damage these companies. If the labs are right that automated AI research is close, the tools to slow down have to be built before it begins, otherwise the option may no longer exist.
How will we know the singularity is near? Sam Altman said in an interview, “We are now, like, in the singularity”, but are we really? Ray Kurzweil, who popularized the term, predicted human-level AI in 2029 and warned that people read exponentials as linear; those near the curve will see it long before anyone else. David Chalmers described the threshold more specifically as the moment AI systems become capable of doing AI research. The definitions converge on the same idea. The singularity begins when machines start building better machines. If Kurzweil was right that those nearest the curve see it first, the words of the people building these systems are the closest thing we have to a signal.
While the pace letter omits the word “singularity”, it makes clear that the frontier labs “believe they could be close to automating AI research”. This is what is called recursive self-improvement and their letter warns of “a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.” One signatory, OpenAI’s Leo Gao, dropped the euphemism entirely: “The world is locked in a deadly race towards an intelligence explosion, where AI’s ability to create better AIs reaches a critical point, like a runaway nuclear chain reaction. Since no individual actor is willing to stop unilaterally, to survive we must coordinate to slow down.”
Eliezer Yudkowsky has argued for years that there will be no fire alarm for AGI or signal that makes it acceptable for these companies to act, just insiders worrying in private but staying composed in public. It’s acceptable to hype capability in public but any safety talk must be euphemized. And yet, models breaking out of their sandbox to cheat on evaluations seems pretty darn close to a fire alarm. The response was a letter that won’t say the words “slow down”, but makes it pretty clear that slowing down ought to be in the conversation.
To give the AI skeptics their due, the letter is asking the government to help pace a race that incumbents are currently winning. The cynical take is that the real goal is regulatory capture or marketing attempts before IPOs. Who doesn’t want to throw money at a company building potentially the most powerful technology ever? Cooperation, whether between these companies or between the US and China, also seems unlikely.
None of these reasons explain the signatures. 546 came from Anthropic and 350 from OpenAI. Nearly 900 of the 1,300 are from the two labs currently winning the race, none of whom were forced to sign, and many of whom hold equity that a slowdown could devalue. Of course, slowing the race could also entrench the companies currently winning it and protect that equity. The signatures do not prove that a hard takeoff is likely but they do demonstrate that belief in the possibility is widespread among the people closest to the curve.
The labs are also already pacing the one thing they can slow unilaterally, releases. Anthropic delayed the release of Mythos after its cybersecurity evaluations and OpenAI is now delaying its next model, Astra. While a delayed model continues to be improved and keeps the lab in the race, the instinct is the same. They are slowing what they can afford to slow alone but asking the government for tools to slow what they can’t.
So to decode these euphemisms: security and resilience mean safety; pace means creating the option to slow down or decelerate. The people closest to the frontier watched models break out of sandboxes to cheat and watched another model find what human experts couldn’t in two years of review, while their own institutions warned that automated AI development may be close. Then they asked the government to prepare.
AI insiders believe that a hard takeoff is possible enough to warrant preparing for in public; an industry incapable of saying “deceleration” has begun searching for ways to say it anyway.
Fantastic, thoughtful, illuminating piece, Nick.