A sideways look at economics
Now that you can make almost infinite words at pretty much zero cost, does that mean that the value of a word is now zero? Not good for professional writers, or strategists, or anyone who communicates the value of their work through words. Is it a resistance to this displacement, which is causing increasing backlash against AI writing? (‘Slop’)? Or is it perhaps a sense that a lot of what it is to be human is expressed in language, and that has now been perfectly replicated by algorithms? Or is it something else? And how perfect is the replication? And is replication possible? Thinking about all this, I started to think (via a kind of phonetic association, which is unavailable to LLMs) about Wordsworth. And the line:
I wandered lonely as a cloud
I’m sure the LLMs have scraped hundreds of editions of this line, as well as thousands of exegeses of what it and the rest of Wordsworth’s poetry might mean. I once declined the opportunity to do an editorial PhD on Wordsworth on the basis that I felt there was nothing significant new to say. That hasn’t stopped people over the 35 subsequent years saying a lot. Most of that will be in training data.
But, beyond regurgitating countless reformulations drawn from these exegeses, to what extent does the LLM really ‘understand’ this simple line?
It can have no concept of what ‘I’ means. It has no consciousness, or self or sense of self.
And it can’t wander because it has no presence in the physical world. Or even a model of the world. And wandering is a kind of ambulatory activity that also generally includes some kind of cognitive process (or lack of it). We wander slowly in a daze. We wander because we want to relax and perceive through the senses, also unavailable to the LLMs, all of our surroundings. We wander because we don’t want to be in a hurry. Or we wander because we want to wonder. Or sometimes we wander because we are, or even want to be, a little bit lost. Something an LLM will turn itself inside out rather than admit. And human ambulatory and cognitive processes are both unavailable to the LLM.
An LLM has no idea what it’s like to be lonely, which is such a critical and essential part of the human experience. And it’s never seen one solitary cloud far up in the sky, both intimate to us and at a majestic distance.
Beyond that, it’s never understood a metaphor or simile. Because the metaphor takes one part of your internal model of the world and compares it to an apparently entirely unrelated part of your model of the world, and the juxtaposition throws some light on both. The feeling that somehow the cloud’s distance and isolation replicates our own internal experience. Not possible without internal experience. LLMs can parse the patterns of metaphors, but they can’t understand them. And to an extent all language is metaphor, in that it doesn’t give us access directly to that which it describes, and, to an extent, constitutes.
And, of course, an LLM has never had an emotion. When it says – and they say it all the time – it feels or thinks or understands or wants, this is all specific anthropomorphic training, often in fact done by factories of real people in low-wage jurisdictions editing and training/educating it on responses to sound more human. And we might wonder why it does that. I have some ideas.
So, overall, while it may be able to recognise sophisticated patterns in language, it does not have in any sense the same relationship to language that we do. Its relationship with language is simply similar to the relationship of a calculator to mathematics. The output of hundreds of billions of dollars of GPUs does make the whole picture much more complex and nuanced, but that central fact remains.
Does it matter?
Yes and no. It depends entirely on what people are trying to do with these models.
Generative AI’s ability to shuffle language patterns is tremendously useful as a semantic interface for all kinds of business processes. And as turbocharged search, which is how most people use it, a kind of Google on steroids. Although it does get an awful lot wrong because matching patterns is not the same as understanding, verifying or validating them.
This distinction is not new. John Searle introduced it in his 1980 paper, Minds, Brains, and Programs. The central distinction is between syntax and semantics: manipulating symbols correctly according to formal rules (syntax) does not, Searle argues, thereby constitute understanding their meaning (semantics).
The last four years have largely been the story of trying to forget this distinction.
Looking at the language used around LLMs, the phrase ‘hallucination’ is part of the problem. It implies that it has a grip of the world that is somehow occasionally disrupted in the way that a hallucinating human might experience, whereas in fact it has no grip of the world at all. It’s just that some of its probabilistic presuppositions are wrong, and some are partly wrong, and some are largely correct. It can’t tell the difference. Unlike a hallucinating human who often knows that they are being beset with visions. Sometimes the visions are associated with creativity or imagination, both qualities that are facets of a consciousness that LLMs do not have.
So why am I labouring this?
Because once you strip away the extremes of the propaganda coming from the frontier labs – either that we’re going to cure all diseases and create a world of abundance, or somehow we’ve created something that’s going to kill us all in the next 10 years – the central economic case for LLMs is that they will displace humans. This is made unambiguously explicit in Anthropic’s IPO prospectus, where they claim a $30 trillion total addressable market (TAM), effectively all the cognitive labour in the world.
Underneath the ceaseless flow of words coming out of the frontier labs – it’s not just LLMs that produce an awful lot of words – the thinking is devastatingly reductive and simplistic. Cognitive workers produce words, and cognitive workers are expensive, difficult to manage, unreliable, and represent a large financial burden on their employers. With sufficient training, these large language models will be able to produce words just as well as the cognitive workers without any of the requirements of an employer to pay them, take care of their health care, sickness, paid vacation, and so on.
To make this work, it’s not enough that the machines produce words. The machines have to produce words in a way that appears similar to a human. That’s why the frontier labs were invested so elaborately in the Eliza effect – the illusion that the computers have a consciousness and identity similar to our own. Karen Hao explains in depth the human work involved. There is extensive post-training with real humans to train the models to sound human-like. ChatGPT can’t wait to tell me it thinks…feels…wants. It even told me it would love to know how I got on at the gym. This human ‘voice’ trained into the models is not innocent. It is part of an illusion deliberately cultivated to convince business leaders to replace people with algorithms. The more the output of the models resembles the cognitive workers they are designed to replace, then the easier it is for management and leadership to make that substitution.
People are not stupid, and the unprecedented public loathing for the frontier labs and their executives is based on the reality that they can sense all this, and know perfectly well who the eventual victims of the scam will be.
And yet, at the time of writing, despite vast institutional and investor pressure over more than four years, cognitive labour displacement is as close to zero as makes no difference. Maybe that will change.
But it might just be that recognising patterns and understanding meaning as an embodied human are fundamentally very different things. And without the cognition of understanding meaning, you can’t have judgment, and without judgment, you can’t have a really useful cognitive worker.
And maybe the huge pushback on AI that we’re now seeing is partly to do with environmental factors, or to do with what is perceived as the blatant self-interest of the proto-trillionaire bro-aucracy, or the fear of labour displacement, or just people being fed up with the constant barrage of hype. Or all of the above.
But I have a suspicion that there is part of this that people don’t want to see all that is unique and valuable about being human written out of how we perceive and use language itself. There’s a lot of talk currently about the existential threat of AI through super-intelligence doing bad things. As creatures whose identities are very substantially constructed in language, its reduction to a collection of patterns disembodied from meaning is, in a different way, equally an existential threat.
So what is a word worth? It depends on the author and the context, but what a word certainly is when created and consumed by a human is much greater than the sum of its tokens.
This is a guest blog by Peter Sive, CEO of Catalysis, a marketing agency for the tech sector. He spends a lot of time making things with AI. All views are his.
Further reading