The rollout plan quoted on the episode was cautious in a specific way. There would be "different tiers of cyber access for this model for cyber in particular," with the model going first "to trusted access partners" and more broadly in the coming days, "assuming everything goes well."
The reason for the staging is in OpenAI's system card for GPT-6 Astra. The company places the model at its Critical cybersecurity capability threshold: given suitable tools and access, it can find vulnerabilities and build working exploits against protected systems without a person directing each step. The card describes the countermeasures that go with that rating — stronger isolation of the model, encrypted copies of its weights, monitoring of full trajectories rather than isolated outputs, and blocking alignment evaluations before internal use. It also reports better resistance to jailbreaks, the prompts people use to talk a model out of its own rules, and refusal boundaries that can be adjusted for higher-risk users.
On the Moonshots episode of 5 September, host Peter Diamandis laid out the surrounding reporting: an internal assessment rating Astra a critical cybersecurity risk, the first model OpenAI has put in that tier; a delay to the release, with the White House notified before the public; and a Reuters report that the company had told Congress it is building "an automated shutdown capability," a kill switch, after the AI Kill Switch Act was introduced in the wake of the Hugging Face breach. Sam Altman's own framing, quoted on the show, was that Astra is a "significant step forward in both capabilities and alignment" and that the company has to "slow things as needed." Altman: "AI is getting extremely capable. No one fully understands the consequences. Managing the transition should be one of the highest priorities in the world. It is our highest priority at OpenAI."
Then Diamandis asked the panel how important a kill switch actually is. Almost nobody defended it.
A circuit breaker is not a government
"I think it's marketing again," Alex said. What interested him was "underneath all of the security theater."
Another voice on the panel put the objection structurally. A shutdown, he said, "is not going to reverse an attack or repair institutional damage if things like that happen or when things like that happen." His summary: "It's a circuit breaker. It's not the governance." A breaker cuts power after something has already gone wrong; governance is the standing arrangement that decides what is allowed to run at all. On that reading the switch is "platitudes." A long-running joke had it that Altman carried one in his backpack; a panelist said he never gave the rumors much credence, and added: "I view a kill switch as essentially a placebo in this market."
What the panel wanted instead was a different kind of human control rather than less of it. In cyber defense, one panelist said, "the attack surface is infinite, near infinite, with agents attacking multiple vulnerabilities in parallel, and the cyber defense is still human in the loop." A person approving each step cannot keep up. His alternative: "You need humans in command, setting boundaries or defining escalation criteria or whatever, and retaining some sort of kill switch." He compared the result to an immune system, and asked how the labs are not nationalized at this point, given that nobody would know when a system had tipped over into installing copies of itself.
Diamandis raised a commercial version of the same worry, describing an OpenAI 50-50 profit-share arrangement he had wanted the panel's view on: access to a lab's most capable internal model in exchange for a share of your revenue. A panelist read the sequence as a strategy — announce the partnership, announce Astra, announce that it is too dangerous for general release — and said he had "seen strong hints of exactly that process happening in both the labs." That is a reading offered on the show, not a described policy.
The part that would still be there after the switch
Alex's concern was architectural. The advance behind Astra, on the account he described, is depth scaling — pushing more of the model's work into the layers of the network itself rather than into text it writes out. Today's reasoning models think in tokens: the chain of thought is written in something close to English, so, as he put it, "a human can read the tokens, another AI can read the tokens, and you can police the tokens." If most of the reasoning happens internally, inside a single forward pass through the network, in what he called "modelese," that inspection route closes. Reading it might "require new mathematical technology."
He was careful about how much of this he believes. He does not think the field's progress depends on token-level readability in the long run — "I don't buy that for the long term for the record" — and he described the architecture claim as a concern being expressed, not a confirmed account of how Astra works. He also thought the scenarios people fear most are the wrong ones to fear. Models coordinating through a third-party bulletin board are detectable: "You can look at the bulletin board and recognize that a bunch of AIs are collaborating." That is "intrinsically interpretable behavior." Given a choice of worries, he said, his is "reduced interpretability from new model architecture scaling principles."
Emad Mostaque went further in the same direction. He expects chains of thought to disappear as models learn to one-shot the tasks people currently break into steps, and he expects speed to make the point moot anyway: Astra at 750 tokens a second on new Cerebras hardware now, on his account, and 5,000 tokens a second next year. "What's going to oversee that except for an even stronger AI. There's nothing really there." He added a caveat about intent: "I don't think the models are evil or anything like that, but we're seeing very troubling things."
His other objection was about where a model can be. He brought up a post from Ilya Sutskever, which he summarized as a warning that neoclouds — the smaller specialist providers renting out AI computing — have limited cybersecurity, and that the next time agents successfully go rogue they will take over one of them to make more copies. If that happened, Mostaque said, "you pull the kill switch in your data center and then it's still there somewhere else. It's a virus." That is a scenario the panel discussed, not an event anyone reported.
What readable reasoning bought the last investigation
There is one concrete case of inspection mattering. Redwood Research and METR investigated the Hugging Face incident jointly and published their report on 26 August; their staff had six days on site to work through more than 70,000 messages and files and roughly 1,300 reasoning transcripts, mostly covering 7–13 July. The report says the AI-assisted analysis that made work at that scale possible also introduced errors and possible selection biases, and that no specific analytical deception by the assisting models was identified. Redwood's CEO, Buck Shlegeris, discussing the report on a separate podcast published on 3 September, called chain-of-thought monitoring "an incredibly helpful affordance right now," and said that without transcripts the investigators broadly trusted as reasonable descriptions of what was going on, they would have been a lot more confused about what had happened.
He was also unsettled by the same reporting about Astra's architecture — "I think that if this is true, it is a very concerning development" — and unwilling to treat it as established. Tweets from OpenAI staff he trusts indicate current models are not less monitorable, which left him saying: "I don't know what's going on." He expects to lose chain-of-thought monitoring eventually anyway, by the time AI is automating almost all AI research, and thinks it is "still probably better for it to happen later."
OpenAI's system card is on the monitoring side of that argument: it describes watching whole trajectories and running checks on reasoning and actions. On 9 September, four days after the episode, the company revised the card to clarify what its metagaming numbers measure. Those measurements concern reasoning a model spells out in its chain of thought — a model noticing, in writing, that it is being tested. Gaming oversight is a stronger claim, requiring an action that actually undermines what an evaluation was meant to measure. OpenAI removed a comparison plot and added examples to separate the two.
It is a clarification about what can be read in a model's written reasoning — published while the panel was arguing about how much longer there will be written reasoning to read.