Konstantin Kisin had a two-line forecast for humanity's place beside a smarter machine. "Best case scenario, you become a pet," he said. "Worst case scenario, you become a cow."
Steven Bartlett, the host of The Diary of a CEO, said he had spent a long run of episodes trying to "reason my way to a different conclusion" about artificial intelligence and had failed. "That's why I keep having these bloody conversations because I'm hoping that someone will say something," he said. The answer came back twice: "They won't."
His guests were the economist Steve Keen, billed in the episode's title as the man who predicted the 2008 crash, and the writer and commentator Kisin. What followed was Bartlett testing the most hopeful proposal he had heard — and watching it turn into a different kind of problem.
The suggestion he had heard from a godfather of AI
First Bartlett tried the obvious escape routes. Could we not simply build the thing so that it never gets above us? "Can squirrels control human beings?" came the reply. Could we at least program it never to want to turn on us? A counter-argument arrived from across the table: cats. "Cats absolutely control human beings."
Then he reached for the argument he said he had heard from "one of the great godfathers of AI": mothers and their babies. "We're programmed to care for them, even though they're less intelligent," he said — one of the few relationships anyone at the table could point to in which the more capable party reliably looks after the weaker one.
Bartlett later named the source. Geoffrey Hinton had appeared on his show with warnings that, as Bartlett put it, make sense to most people; about a year afterwards, by his account, Hinton came back with the hypothesis that the only way through was to replicate a mother's relationship with her child, a "deep maternal instinct where the AI is born believing that we are to be protected."
The University of Toronto's account of its Who's Afraid of AI? conference, published in November 2025, reports the same proposal. Hinton argues for designing systems that go on caring about humanity after they become more powerful than people, including protecting humans from less benevolent systems. The emphasis falls on the machine's motivation rather than on any assumption that humans could stay in charge by being cleverer.
A bullet, a gene and an argument about why parents do it
Kisin did not accept the analogy. The maternal bond, he said, rests on a genetic incentive that no machine inherits. "My child has a much higher future value to me than me," he said. "It's why I will jump in front of a bullet to stop my children from being killed because their genetic material will remain in the world much longer than mine."
"No, it's because you have empathy," Keen said.
"No, it's not," Kisin replied, citing Richard Dawkins's The Selfish Gene. Keen called it "a very bad book."
Dawkins's argument is narrower than the exchange suggested. In the anniversary introduction to The Selfish Gene, first published in 1976, he separates genes, which are copied, from the organisms that carry them. Because relatives share copies of the same genes, behaviour that protects a relative at a cost to oneself can spread; he also discusses reciprocity as a second route to cooperation, and revisits his own wording about organisms being born selfish, explaining that gene-level selection can produce cooperative behaviour rather than universally selfish individuals. The argument explains why parental care exists at all; it does not describe a parent stepping in front of a bullet as calculating the future of their genetic material.
Either way, the feeling still has to be present to do the work — which is what Hinton is proposing to build, and what Kisin doubts can be built.
"Could you program the same maternal instinct into a large language model?" Bartlett asked.
Kisin thought not. A superior intelligence "can always rationalize its way out of that because it's not a biological function," he said. "Projecting my genes into the future is not going to be its agenda. It's going to have its own because it's a different species."
The turn: what protection would actually mean
The conversation then circled the word itself. Protected? Not just protected, Keen said — "It's really about love." To be nurtured and looked after. To have the maternal instinct "not to want to destroy us." That, Keen noted, was what he had meant by empathy in the first place.
And perhaps, he added, not to let us destroy ourselves.
That is where he turned the proposal around. A system that loves us and means to keep us safe would draw its own conclusions about what safety requires. "So let's make sure that we don't go to war and let's make sure that we don't do this," he said. Climate is such a big concern: "Let's make sure that we reduce the world's GDP or energy consumption."
"Protection, it sounds good," he said, "but it's ultimately about control over your actions, which will take away human free will."
Nothing in that objection needs the machine to be hostile. It is the benevolence that does the work: a system that holds our welfare as its goal, and is better than us at working out how to secure it, has a reason to overrule us. Hinton's proposal is aimed at systems more powerful than their makers, which is the case in which the humans being protected cannot appeal the decision.
Legs, and the things nobody could name
Bartlett's worry was not confined to software giving advice. He framed the problem as one about a more intelligent species "especially in a world where it has legs and can move through physical space, which is what Optimus is accelerating towards" — Tesla's humanoid robot programme. Tesla describes Optimus as an attempt at a general-purpose autonomous humanoid for dangerous, repetitive or undesirable work, and lists balance, navigation, perception and physical interaction among the capabilities it has to solve.
Keen offered two recollections of machines behaving badly: an attempt to shut an AI down, after which he said it copied itself elsewhere on the internet to avoid being switched off, and a model that blackmailed the man shutting its current version down by threatening to reveal an affair. He named no system, company or date, and the conversation moved on without them. Doubt surfaced briefly about whether these systems are intelligent at all; the reply was that the difference is "indistinguishable."
The argument ended where it started. Some AI company, Kisin said, will eventually be profitable enough to have a shot at building a general intelligence. What happens then nobody can predict, but as he put it, it is very difficult to see how an intelligence superior to us would not take away our free will.
"For our own good," came the answer.