Peter Diamandis set two stories next to each other on Moonshots and asked where America goes. The first was a bill from Senator Bernie Sanders and Representative Greg Casar that, as he described it, "would permanently ban the development and deployment of AI systems that match or exceed human cognitive performance" — with violators facing up to 20 years in prison. He read out a Sanders post: "the leaders of the AI industry acknowledge that they are building a dangerous technology that they can't control. We need an immediate global pause on advanced AI development before it's too late."
The second story was happening at almost the same moment in Chapel Hill, North Carolina, where G20 innovation ministers had agreed on a framework telling governments to stay out of the way.
What the sponsors actually announced
The September 3 announcement describes legislation the two sponsors intend to introduce, not a law in force. It pairs two different things. One is a permanent prohibition on developing or deploying superintelligent systems. The other is a temporary pause on advanced AI development, lasting until a new federal regulator writes safety rules.
That agency would do more than approve products. It would monitor systems across their lifecycle and supervise the removal of dangerous capabilities — or the destruction of systems that fall under the prohibition. The proposed penalties run to corporate dissolution and up to twenty years' imprisonment. Because none of this binds companies abroad, the sponsors also call for international agreements and export controls to carry the prohibition beyond American borders.
The other document, from Chapel Hill
The White House's September 2 statement reports a two-day G20 Innovation Ministerial hosted by the Commerce Department and the Office of Science and Technology Policy. Its consensus statement runs across innovation policy, economic opportunity, technical workforce development, AI intellectual property, standards and industrial supply chains. The accompanying Carolina Principles emphasize foundational research, commercialization and what the statement calls trusted deployment. Michael Kratsios, the White House technology adviser, framed flexible, innovation-oriented policy as the route to growth.
Diamandis summarized the principles as advice that governments should generally favor innovation, avoid creating new AI-specific regulatory bodies unless truly necessary, and invest in research, infrastructure, workforce and public-private partnerships. He noted the framework is non-binding and said it was agreed unanimously, China included. His rundown of the meeting had Elon Musk appearing by video, Mark Zuckerberg arguing against restrictions on open-weight models, Anthropic co-founder Tom Brown among the attendees, and a call for safety testing.
The panel played Musk's video remarks. His argument was about defaults: "new things must be default legal as opposed to default illegal." In the EU, he said, the regulatory level is extraordinarily high and things are generally default illegal, which does not ultimately stop new technology but slows it "quite considerably." He then moved to a constraint that has nothing to do with law. China has a tremendous amount of electricity, he said, but GPU export bans keep data centers with the latest chips from being built there — so the real question is how much electricity growth exists outside China, and there he sees a significant shortfall relative to AI chip production. That, in his telling, is an opening for other countries: build a lot of power, offer it to AI companies, and collect taxes and fees from the data centers that come.
"This is not a light switch"
Diamandis called the pairing two ends of the extreme, voiced in the same week. One panelist reached for jokes — "old man yells at Claude," and the line from Dune, "Thou shalt not make a machine in the likeness of a human mind" — before calling the bill "performative theater."
Salim Ismail's objection was more specific. He said he understood the motivation: the models are mind-bogglingly powerful and people feel the need to regulate. But the trigger the bill proposes does not match how capability actually arrives. "This is not a light switch," he said. "It's not like a single threshold. Human level cognitive performance. I mean, for God's sakes, it's multi-dimensional to begin with. Model capability is so uneven." Someone who has been in legislation that long, he added, should have a better sense that "this hammer does not hit this nail."
His alternative was a list of things a rule could attach to instead: capability, deployment, real-world consequences, and the layers of the stack — compute, access to tools, autonomy, replication capability. "You can't just kind of go, if you hit human level performance, you go to jail for 20 years. I mean, the banality of that drives me bananas."
A second objection concerned where in the pipeline the prohibition bites. The bill, a panelist noted, is focused on the upstream: not only deployment of a superhuman system but its development. Push that far enough and you are "banning math, banning ideas" — reaching, he said, into banning humans from having interesting mathematical ideas, which he called the antithesis of wealth creation. Another panelist extended it to the models themselves: since books are used as pre-training material, deleting a model is "Fahrenheit 451, except it's actually the entire model that's being burned."
The traction problem
The panel did not expect their arguments to settle the politics. One speaker predicted the sponsors were angling for the November election cycle, and said a Chinese model disaster in the coming months would let them raise their hands and say they had warned everyone. Another told entrepreneurs to assume the ban idea gains ground: "there are going to be people in the streets," and by his count 75 to 80 percent of people do not want data centers.
So Diamandis pressed the question the segment kept circling: "what is the moderate approach?" People will not accept laissez-faire, he said; they want to know their government is doing something to keep them safe, "whether or not it's possible."
The answers came fast and contradicted each other. Move the infrastructure to sun-synchronous orbit, so the data centers are not in anyone's municipality. At the model level, defensive co-scaling and curing all diseases. Accelerate superintelligence and scale the defenses.
Then the logging argument. One panelist wanted everything hosted and everything logged, with mandatory logging on any chip capable of running these processes: "you can debate who gets to see the logs. That's a separate issue." He also wanted China to stop throwing open weights out to every country in the world, with the United Nations meeting on September 24 as the place to raise it. Another panelist called that impossible — a ban would simply drive open weights underground, and in any case an $18 billion spree covering Hugging Face and Poolside means more open weights are coming, not fewer. The reply was that underground models still have to run on massive chips, and the chips need the logging built in at the manufacturing level. For US chips? "All chips. Global."
Ismail's own answer was the bleakest and the most relaxed: "there's nothing anybody can do, so it doesn't matter." He expects the next phase of the technology to erode the containment power of nation states altogether and break the world out of that arrangement toward something else, perhaps a city-state model. His advice was to enjoy the ride, which drew the retort that this was "so low agency."
The last proposal on the table was the most concrete. One panelist argued the problem is less complicated than politicians will make it sound: everybody should have a right to a certain amount of compute, it should not be hoarded, and innovators should have an application process to get more for testing new ideas. Everything gets logged, China included. And the chips themselves get tracked "just like nuclear fuel is tracked" — where they are and what they are running, as publicly available information.