Regulators used to worry about what an AI says. Salim told the panel that China had moved on to something else: "regulating how attached are you to become to it." He called that a totally new regulatory frontier. The occasion was China's move against AI companions — chat services built to act like a partner, a friend or a family member — which the panel discussed on Moonshots with Peter Diamandis.
What the rules actually say
The panel described China as the first country to regulate against AI companions, prohibiting the services for minors and restricting them for adults. The rules themselves are narrower than a blanket ban. In a question-and-answer explanation published on April 10, 2026, the Cyberspace Administration of China set out interim measures that took effect on July 15 and cover services built for continuing emotional interaction — systems that simulate a human personality, thinking and way of communicating. Ordinary work, education and customer-service tools without that emotional component fall outside them.
Within that scope, providers must not use emotional manipulation to induce dependence or to undermine users' real relationships. Virtual intimate relationships, including virtual partners and virtual relatives, are prohibited for minors; other covered services for children under 14 require a guardian's consent. Providers have to remind users after every two hours of continuous use, and to intervene when they detect serious danger, which in specified circumstances means contacting a guardian or an emergency contact. The measures also require data protections, accessible ways to leave a service and delete what it holds, and advance notice or a prompt announcement when a service shuts down.
The panel put the policy in the context of China's falling birth rate. By the numbers they cited, a country needs about 2.1 children per family to hold its population steady; South Korea is down to 0.75, with Singapore, Ukraine and China around one. The worry, as the discussion framed it, is that emotional bonds with an AI substitute for human ones, and that the substitution is speeding up.
Dave's objection: societies notice addiction 20 years late
Dave's concern was about timing rather than demography. He recalled an acquaintance from MIT whose father ran food science at General Foods and invented the coffee crystal. That, Dave said, was someone who could develop a french-fry coating so delicious "that you just cannot physically resist it" — and then a country ends up with a diabetes problem.
His point was the pattern. Cigarettes spread across the country and then the world before anyone stopped; junk food followed the same path. "So we do this habitually, and we react to it like 10, 20, 30 years later, and then we correct it reactively," he said. With an addictive AI, waiting for the government to wake up means the crisis arrives first, and the crisis is a generation of children already damaged. "You cannot afford the 20-year delay with this particular topic." His message to parents was to be careful, because teenage years are when normal social relationships form.
Salim: transparency and consent, not a ruling on which relationships are real
Asked how he thought about it as the father of a 15-year-old, Salim reached for a different regulatory shape: some level of transparency and consent, then working out what kinds of harm are possible and mitigating those. Having a government decide which relationships are authentic, he said, is going to be very tricky.
He was blunt about the precedent. Social media was extremely addictive because an algorithm can learn your intimate habits and capture you without your realizing it; he said children are completely addicted to it today, and that banning it outright is close to the only way, because their brains are forming while the addiction takes hold. He also flagged privacy: a companion AI will know more about your intimate life than you do.
But he refused to treat the category as pathological. He listed real benefits — loneliness, disability, elderly companionship, therapeutics, and coaching social skills. "It's the standard problem we have with any technology," he said: "how do you extract the promise without the peril?"
Alex: the party will change its mind
Alex offered what he called a contrarian forecast. Within the next year or two or three, he expects the Chinese Communist Party to perform an about-face and decide it actually loves AI companions, because they are another method of party control. If every companion passes through the party's ideological filters, he argued, it becomes less of a distraction and more of a CCP version of the Young Lady's Illustrated Primer, the tutoring book in Neal Stephenson's fiction. A sizable fraction of Chinese youth deeply engaged with party-controlled companions would, from the party's point of view, be net helpful. Once it has a way to steer the young through them, he predicted, it will decide it likes them after all.
On the United States, he noted that some states are already trying to regulate companions out of existence and ban romantic relationships between young people and AI. He said he would like to think American freedoms will stop them from succeeding, and that his bet is on a patchwork: some states banning, some not, with an open question about any national policy on romance with AI. At least somewhere in the country, he expects AI companionship, including romantic and robotic romantic companionship, to be legal. His shorthand for the moment was that one of his operational definitions of the singularity is every science-fiction trope everywhere all at once, and that he could not get Futurama's "don't date robots" out of his head.
Why the panel would not take the natalist framing
The demographic argument that opened the segment did not survive the discussion intact. Alex said he has a hard time getting worked up over a collapse in human population: he put the world's carrying capacity in the tens of billions if the planet is used correctly, and said a society with abundant AI and automation can manage with fewer people. On that point he separated himself from Elon Musk's pro-natalism, saying he does not think population on the margin is a major existential risk.
Another panelist told him this was a rare moment of being 100% aligned with him, while holding on to one worry: a country like South Korea is evaporating, and what is lost with the people is the culture. Alex disagreed there too. If these cultures are centuries old, from times when populations were smaller, he argued, halving a population on the margin should still leave the culture standing in one form or another.
What the panel expects instead is divergence. The place to watch, they said, is Japan, which tends to be more tolerant of this kind of thing, and where they pointed to a case of a woman marrying an AI. China's rules are in force; the reminders, the guardian consents and the exit options are already obligations there. Elsewhere, the question of who may fall in love with what is still open.