Will artificial intelligence serve humanity — or will it spawn a new species of conscious digital beings with their own agenda?
It’s a question that has sparked scores of science-fiction plots, from “Colossus: The Forbin Project” in 1970, to “The Matrix” in 1999, to this year’s big-budget tale about AI vs. humans, “The Creator.”
The same question has also been lurking behind the OpenAI leadership struggle — in which CEO Sam Altman won out over the nonprofit board members who fired him a week earlier.
If you had to divide the AI community into go-fast and go-slow camps, those board members would be on the go-slow side, while Altman would favor going fast. And there have been rumblings about the possibility of a “breakthrough” at OpenAI that would set the field going very fast — potentially too fast for humanity’s good.
Is the prospect of AI becoming sentient and taking matters into its own hands something we should be worried about? That’s just one of the questions covered by veteran science writer George Musser in a newly published book titled “Putting Ourselves Back in the Equation.”
Musser interviewed AI researchers, neuroscientists, quantum physicists, neuroscientists and philosophers to get a reading on the quest to unravel one of life’s deepest mysteries: What is the nature of consciousness? And is it a uniquely human phenomenon?
His conclusion? There’s no reason why the right kind of AI couldn’t be as conscious as we are. “Almost everyone who thinks about this, in all these different fields, says if we were to replicate a neuron in silicon — if we were to create a neuromorphic computer that would have to be very, very true to the biology — yes, it would be conscious,” Musser says in the latest episode of the Fiction Science podcast.
But should we be worried about enabling the rise of future AI overlords? On that existential question, Musser’s view runs counter to the usual sci-fi script.
“I think dumb machines, unconscious machines are, as is proven in today’s world, extremely dangerous — quite capable of annihilating humanity already. Conscious machines don’t add any particular capability that would enhance, at least in my view, their danger to humanity,” he says.
“If anything, I think probably the danger goes the other way: that we’ll mistreat them,” Musser says. “That’s unfortunately also in the history of human beings. We tend to mistreat, until we grow out of that, beings that clearly are conscious.”
The capabilities of generative AI tools like OpenAI’s ChatGPT aren’t that much of a surprise to those doing research in the field — but Musser says researchers have been surprised by how quickly they’re being developed.
He compares the progress in AI to progress in harnessing nuclear fusion. “It’s one of these things that always seems to be in the future, right?” he says. But in the case of AI, the future seems less far off.
“It brings forward the timetable in a lot of people’s minds for what’s known as AGI,” Musser says. “That’s artificial general intelligence, and that is a system that is capable of a wide range of functions, of learning on the fly. Basically a closer stand-in for an animal, or even a human, or even a superhuman brain. So I think it has accelerated things, but it’s been a quantitative, not a qualitative shift for people.”
In Musser’s view, it’s more urgent than ever in the age of ChatGPT for scientists to figure out how our own brains work. “It’s not just a scientific enterprise, out of curiosity,” he says. “We pretty well better understand consciousness if we are to make judgments about the machine systems and ask, ‘Are they conscious?’ Because you need a theory of consciousness just to answer that question.”
Musser focuses on two theories in his book:
- Integrative information theory: Consciousness arises from the internal connectedness of a network, and can be measured on a mathematical scale. Animals possess a level of consciousness that ranks below humans. Inanimate objects like computers could theoretically be conscious, but their information networks would have to be structured with feedback loops analogous to biological nervous systems.
- Predictive coding: Consciousness arose as a biological function aimed primarily at helping organisms to predict what would happen in the near future and help them avoid threats — for example, being able to anticipate a lion’s leap. Machines would have to possess predictive models of their environments that are as detailed as the ones that organisms developed over billions of years.
“They’re not the only theories,” Musser says. “I’m actually writing now an article on global workspace theory, which is another really super-interesting theory. It does suggest consciousness is specific to relatively few species that have a certain kind of cognitive functioning to them.”
Most scientists say the current generation of generative AI falls short of consciousness, by any definition. “ChatGPT, probably not conscious,” Musser says. “But a successor? ChatGPT 5? Maybe.”
The most fascinating angle covered in Musser’s book has to do with the connection between theories of cognition and quantum physics. On one hand, Nobel-winning physicist Roger Penrose has long argued that consciousness is a quantum phenomenon, arising from activity in microtubules within neurons. If that’s the case, true consciousness couldn’t be created using classical computers.
On the other hand, some scientists suspect that having a better understanding of consciousness could lead to a better understanding of quantum physics’ puzzles — specifically, the “observer effect” that plays a part in strange phenomena like superposition and quantum entanglement. Physicists haven’t yet fully solved those puzzles, which is why some of them are getting increasingly interested in AI and the study of consciousness.
“We can begin to talk about specific theories in neuroscience and philosophy of mind that can help on these questions that physics raises, that physics poses, that physics confronts, having to do with the strange and almost paradoxical role the observer seems to play in our understanding the physical world,” Musser says.
Musser is hoping for a breakthrough — not the kind of breakthrough that leads to a Matrix-style dystopia, but the kind that leads to a new scientific understanding on the scale of Albert Einstein’s theory of relativity. Such breakthroughs may come only once or twice in the course of a century.
“I’m excited by the idea that we will do that in partnership with AI,” he says. “You may have heard of the example from radiology, that human radiologists catch like 97% of whatever we’re looking for — tumors or aneurysms or whatever — and AI captures 97%, but it’s a different 97%. So, you put them together and you get something that’s better than either one on its own. I think we’ll have something analogous to that with the use of AI to solve puzzles of consciousness.”
Check out the original version of this item on Cosmic Log to get recommendations for further reading from George Musser, and stay tuned for future episodes of the Fiction Science podcast via Apple, Google, Spotify, Player.fm, Pocket Casts and Radio Public.
For more about the weird beliefs surrounding the OpenAI controversy, read “What OpenAI Shares With Scientology,” by politics/tech commentator Henry Farrell. And for more about consciousness and quantum physics, check out the Fiction Science interview with Robert Lanza about the observer effect, plus our interview with Christof Koch about the neuroscience of consciousness.
My co-host for the Fiction Science podcast is Dominica Phetteplace, an award-winning writer who is a graduate of the Clarion West Writers Workshop and currently lives in San Francisco. To learn more about Phetteplace, visit her website, DominicaPhetteplace.com, and read an excerpt from “The Ghosts of Mars,” her novella in the current issue of Asimov’s Science Fiction magazine.
