The Technium

Conscious or Not


For as long as I remember, people have been arguing about whether machines could be intelligent or not. Many science fiction authors and fans — like myself — felt it was inevitable, only a matter of time. However there were many very smart experts who made very good arguments as to why machines would be fundamentally unable to think or be intelligent. They had high confidence that intelligence was uniquely human. While these arguments appeared sensible, the main fault on both sides of the controversy was that we lacked a good definition of intelligence. The argument was often reduced to relying on something called the Turing Test, which did not actually test for intelligence.

Now in 2026, no one argues that machines could never be smart. We still don’t have a good definition of intelligence, but we have plenty of real life experiences with machines that are smarter than we are in some ways. LLMs outperform the average human in many intellectual tasks, although they fail in others. But since they are getting better by the month, the arguments that they can never be intelligent have disappeared.

So now the argument has shifted to consciousness. A set of very smart people have high confidence that AIs can’t be conscious, or at least not yet. However, everything I know about both the natural world and the world of technology has convinced me that it is possible to create synthetic consciousness. Even though we lack a good definition of consciousness, we’ve learned that the boundary between living systems and technological systems is blurred and overlapping, so we should imagine being able to synthesize anything found in nature. It seems inevitable to me that we will instill consciousness of some types into machines. In a previous essay I wrote of my suspicion that there is a spark of some type of selfhood, or persona, or consciousness in today’s LLM Claude.

Not everyone agrees. There are many smart experts who feel that machines are fundamentally unable to be conscious because they lack bodies, or souls, or a survival imperative, or experience time. Or at least they are not conscious in the way that humans are. Many more experts think that maybe someday in the far future they can be, but that there is no way machines are near consciousness now. In particular, there is great skepticism by very bright and imaginative people that LLMs could be conscious in 2026.

Recently one of the best living science fiction authors, Ted Chiang, wrote a graceful, beautiful article in The Atlantic that argues against the idea that today’s LLMs are conscious. He argues that claiming consciousness in Claude is not only wrong, it’s dangerous because that kind of anthropomorphic might cause humans to rely on AIs to make decisions. But since they aren’t moral, and are only following commercial interests, they will lead humans astray.

Our current arguments about whether AIs are – or can be – conscious is clouded by the fact that we still have no clue what consciousness is, how it can be detected, appraised, verified or quantified. If consciousness follows the pattern of intelligence, as I suspect it will, we’ll eventually come to see that it is not a binary state – either there or not there – but a continuum of many varieties, of multiple types of awareness in multiple degrees, all present on gradients. In that way, gorillas have some types of consciousness, dolphins and dogs have others, large systems like the immune system have dim bits, and even LLMs will have some primitive degrees of it. It is not an either/or state, and not just one type or one dimension. There are a plurality of qualities, a few that are shared widely among different systems, but the mixture of elemental consciousness types, will vary from entity to entity.

We will make species of intelligence with little consciousness, and species of consciousness with little intelligence. And vice versa. The possibility space of possible minds is large and expanding, and the space of possible types of consciousness is probably also as large. Or perhaps, consciousness is a type of intelligence. We have no idea.

With that in mind, I was struck by one statement in Ted Chiang’s piece, where he quotes Anil Seth:

The neuroscientist Anil Seth has noted that no one claims that AlphaFold—the program developed by Google DeepMind to predict the folding of proteins—is conscious, even though its underlying architecture is in many ways similar to that of LLMs like ChatGPT and Claude. This indicates that it’s not any intrinsic property of so-called neural networks that leads people to believe that LLMs are conscious; it’s simply the fact that LLMs emit grammatical sentences and we are accustomed to reading intention into sentences, whereas we are not accustomed to reading intention into the way that amino acids fold into protein molecules.

I claim that AlphaFold does have a sliver of some kind of consciousness that is far from human types. We might call it molecular consciousness. But more importantly Anil and Ted miss a major episode in the evolution of our own consciousness: language. What they call consciousness only arrived when we invented language. Human-type consciousness requires language; and language enables consciousness. We were not fully conscious until we could think using the symbols of language. Language gave us the tools to access our thoughts. The reason we detect more evidence of consciousness in LLMs versus AlphaFold is that the language in Large Language Models contain the same ingredients that we needed for our own sophisticated consciousness.

We have underestimated the power of language. Millions of years ago we invented language to allow us to communicate with each other. That innovation led to intense cooperation and collaboration, which in turn gave humans immense evolutionary advantage, and that in turn led to the creation of a robust culture and increased resourcefulness which created a cycle of yet more communication. The ability to communicate via language was the primary accelerant in the evolution of humans.

But there was a far greater impact from our acquisition of language. The biggest benefit from language was not the ability to communicate with others but the ability to communicate with ourselves. Language allowed us access to our own minds. It gave us a way to manipulate our thoughts. To reflect, to operate on memories, to predict. It gave form to ideas. Language allows introspection, and thus self-improvement. We cannot imagine how we could be conscious without using language. Try to remove words from your own mind. Our intimate self-awareness, morality, purpose, all seem to collapse when the structure of language disappears. Yes, we can have emotions, reflexes, drives, but the kind of sophisticated state we call consciousness is gone.

To be clear, language is more than just verbal words. The born-deaf are conscious, and those afflicted with brain aphasias that block verbal abilities can likewise operate with a self, but without the symbol and syntax of language the reflective, autobiographical, inner development layer of consciousness is thwarted.

Language and consciousness are so wedded in us they are nearly synonymous. So when we give one type of AI a robust language ability but refrain from giving it to another, it should not surprise us that the language-equipped AI exhibits some aspects of consciousness. 

Full, industrial-grade consciousness is not always a benefit. There may be kinds of minds we don’t want to be conscious at all. Is there a reason we want consciousness in the robot driver of a self-driving car? For safety we don’t want it distracted by thoughts of whether it should have majored in chemistry instead of driver’s ed; we want it to just drive.

This debate of whether AIs are conscious will be a long game. Along the way the quest will introduce a lot of uncertainty about our own consciousness. This wholesale investigation into the nature of consciousness will generate the biggest advances in neuroscience, psychology, and philosophy. In the next 25 years we’ll learn more about ourselves than in the last 25,000 years. One hundred years from now we will have a very different idea of what we think humans are.




Comments


© 2023