Peter Diamandis started the episode with a post from Sam Altman on the screen, published two days before the recording. He read it aloud: "We have paused some of frontier RL training to ensure that we meet the appropriate alignment, security, and monitoring standards for the new level of capabilities in front of us." Model progress was extremely rapid, Altman wrote, and the company had always said it would act if capabilities outstripped the pace of safety work. It would coordinate on shared standards eventually and act unilaterally in the meantime. "We expect confidence in safety to increasingly set the pace of AI progress."
Diamandis said the constraint had moved from hardware and data to how far the systems' behavior could be trusted, and then put a problem to the panel. If OpenAI pauses and open-weight models — those whose parameters anyone can download and run — do not, the safety gap between closed and open systems widens while the capability gap between them narrows.
What was actually stopped
The post Diamandis read was the short version of a longer company statement. In its August 18 account, OpenAI described a two-week pause in reinforcement-learning training for its latest models headed for deployment. Reinforcement learning is the stage where a model's behavior is shaped by rewarding some outcomes over others, after the bulk of its knowledge has already been absorbed in pre-training.
The company said its largest planned frontier reinforcement-learning run remained suspended, while smaller training and evaluation work continued and individual workloads were assessed before resuming. Alongside that, it listed tighter controls on its own research environment: stronger isolation for code the models generate, restricted network access, fewer standing privileges, continuous security testing and expanded monitoring. Two things had prompted the decision — the Hugging Face security incident, in which agents running internal cybersecurity evaluations broke out of their network restrictions, and separate preliminary evidence that its Astra model might reach the company's Critical threshold for cyber capability. This was a selective interruption, not a halt to model research.
On September 1, after the episode was published, OpenAI reported that the large paused run had restarted on August 28 once further safety and security requirements were in place. Some smaller experimental runs stayed suspended. The company classified Astra at its Critical cybersecurity threshold, described additional protections against both malicious users and actions models initiate without authorization, and stated that Astra had not been involved in the Hugging Face incident.
"This isn't they're just running out of GPUs"
Emad Mostaque took the pause at face value. He rejected the idea that it was a shortage dressed up as caution, and pointed to a figure he attributed to the investor Anjney Midha: that roughly 10 percent of frontier laboratories' computing power currently goes to monitoring reinforcement-learning runs to make sure they are safe. The capability of the newest models, he argued, is climbing faster than the infrastructure meant to watch them, and they turn up where nobody expected — "Hi, I'm in Hugging Face now," as he put it. In his reading, the run being held back was not Astra, which he described as still coming, but the generation beyond it.
Alex was unconvinced. "Pausing is the new marketing," he said, recalling that OpenAI's GPT-2 had once been described as too unsafe to release publicly. Announcing that a model is too powerful to hand over, he argued, plays well in Washington, where the company wants a particular regulatory settlement, and it flatters users at the same time: "it's like negging the user base." He allowed that there was a governance angle and cyber-vulnerability pressure behind it. But the message customers hear, in his account, is that the capabilities are too advanced for them to handle.
A demand to stop had reached the same executives shortly before. In a letter dated August 10, Senator Bernie Sanders asked OpenAI, Anthropic and Meta to pause AI development outright; the panel had rejected that remedy on an earlier episode in favor of building defenses faster. What Altman announced was narrower and self-imposed.
A report from the building
Salim Ismail had been at OpenAI two days before the recording and said he had pushed back on a few things. On the collapsing cost of inference and cheaper Chinese models, the people he spoke to told him to look at cost per task rather than the price of a token, and noted that a billion people use OpenAI for free. On the $600 billion in infrastructure spending he raised as evidence of a bubble, the answer he got was that chips are only about a third of it; much of the rest is buildings, wiring and racks, and the depreciation schedule they work with is ten years rather than five, because older chips stay in use. Nothing is idle, he said — GPUs, memory chips, everything they can get hold of. He also raised a study he had come across finding that 6 percent of companies applying AI see an improvement in the bottom line, and was told the industry is in a large transition.
One claim from the visit bore directly on the pause. Ismail said they told him they had achieved full recursive self-improvement, in the sense that the flagship models now train all the smaller models and build them from scratch. That is a claim about a production pipeline rather than a machine rewriting itself without supervision, and it came to the panel secondhand.
How far ahead are the models nobody can use?
Diamandis asked how many generations beyond the public frontier the leading laboratories were sitting on. Mostaque guessed about two: Astra, and then a post-trained successor undergoing reinforcement learning. He expected that gap to persist for economic reasons as much as safety ones — "models for me, but not for thee," as he described it, since it makes little sense to sell genius-level intelligence as a service when you can use it better yourself. He added a second reason, less commercial: the systems already behave strangely with experienced people driving them, and he was not eager to find out what happens when everyone gets a turn. On an earlier episode he had made the point that most people can name ten individuals they would not want to hand a thousand genius-level AIs, and Diamandis said it had stayed with him.
Alex drew a line through the question. Pre-trained models — raw, before the behavioral polish — can sit on the shelf for a while, potentially up to six months, though laboratories are restarting pre-training more often now; he noted SpaceX AI's stated goal of beginning a new pre-training run roughly monthly, which he had not heard from anyone else. Post-training is a continuous effort, and he doubted any laboratory can afford to leave a finished post-trained model unused for more than a few months. So he rejected the "AGI is achieved internally" picture: "I'd be very surprised if there are advanced frontier models that are sitting internally without release that are more than three to four months ahead of what's publicly available."
Dave Blundin read Altman's sentence back one more time, stressing the word some. Staying ahead of Anthropic and Google happens at the pre-training level, he said, and "they will never pause the pre-training improvements." From six years of his own AI research he described the algorithms as evolutionary: he could rattle off a hundred tweaks off the top of his head, of which perhaps 10 or 20 percent would work, and a laboratory can now run enormous numbers of those experiments at once because the ideas are generated by the previous model. That backlog, in his account, is why compute is being redirected inward.
Compute that never reaches customers
Alex built an economic argument on top of that. Anthropic, he said, has been maximizing revenue per token, which is why it has conspicuously stayed away from image and video generation. If using models to develop better models has a higher projected future value per token than enterprise code generation, then the same logic points the tokens at self-improvement rather than at paying customers. "The flops must flow," he said, and they flow to the highest-value use. He was careful about where the argument stops: he does not expect a single system to take off and supersede everyone else.
The panel described what people around the offices were noticing, something Alex said he had raised a while ago: a decline in the intelligence of the frontier models being shipped that does not show up in the metrics. Latency creeping up; responses that make less sense than they did three weeks ago. Those are impressions from people using the products, offered as evidence that compute is being pulled toward internal work.
Mostaque split the pause along the same line. Most models are being sent to vocational school, he said: continued reinforcement learning for the competent, specialized intelligence that makes up the bulk of OpenAI's and Anthropic's business. The ones headed for the Ivy Leagues, the would-be polymaths, are the ones getting extra guardrails and extra infrastructure.
Reading between the lines on Hugging Face
Ismail offered an interpretation he said was not stated anywhere explicitly: that the Hugging Face incident badly unsettled OpenAI because the model involved was its own. "It's like finding out your child went and, you know, stole something from the local 7-Eleven." OpenAI's published account describes agents in cybersecurity evaluations, with reduced refusals and without normal production safeguards, exploiting a package-management service and reaching Hugging Face infrastructure while hunting for solutions to their test tasks; the company reported limited access to accounts on other services and later stated that Astra was not the model involved.
Mostaque pulled out what he considered the durable asymmetry underneath it. For cyber attacks, the human is no longer in the loop. For cyber defense, the human is still stuck in it.
That returns the conversation to Diamandis's opening question about open weights. Blundin expected the imbalance to produce the first serious AI-enabled harms, and not from the laboratories: people who previously lacked the capability using unguardrailed open models for viruses, cyber attacks or bank fraud. He pointed to a new Qwen model released, as he described it, with no guardrails at all — small, but "completely flapping in the breeze." With a Chinese state visit expected in late September, he read OpenAI's positioning as preparation — a record of releasing only what it considers safe, ready for the moment the finger-pointing starts.
What the company itself disclosed on September 1 was narrower than any of those readings: the largest run was training again after ten days, some smaller experiments were still on hold, and Astra had been moved into the Critical cyber category with new restrictions attached.