LLM's Literally CAN'T "Write". Here's Why.
(AKA... I take a stab at defending my vocation)
It’s been over 3 years since ChatGPT first emerged, shocking us all with its feats of linguistic verisimilitude. Remember those fast food menus reworked as Shakespearean sonnets?
My first reaction was to chuckle at the phenomenon, then dismiss it. This damn machine isn’t actually writing anything! It’s just recognizing patterns and throwing words together! Cool? Sure — in the same way those early screensavers with the moving tubes were cool. But it’s hardly a replacement for actual writing.
Since then, I’ve been forced to confront the reality that, however silly large language models (LLMs) seemed to me, the world at large is taking them seriously. My students started asking why they have to bother writing when AI can create texts for them. Educational leaders started pushing AI into the classroom even as studies showed it diminishes mental acuity. And, perhaps most shockingly, some writers began musing that AI could replace them.
In 2023, the British novelist Adam Roberts told the Guardian he could “retire” if the AI’s texts become indistinguishable from human writing — while also marveling that AI systems write from “a collective subconscious.” Jeanette Winterson also wrote enthusiastically of AI, predicting that, “[a]n alternative intelligence will make art of all kinds – with us, and without us.”
Needless to say, I disagree.
Large language models are entirely incapable of producing literature. They work by scouring the internet, identifying patterns, and producing strings of text that correspond to those patterns. Their output is entirely limited to what exists on the internet, to what people have already written, created, or said.
At first glance, an LLM’s process seems similar to what writers do. We read — hopefully, a lot. We absorb what other writers have said before us. And then, by some magical process that includes the recognition and reproduction of patterns, we create texts of our own.
But a writer’s work contains more than just the repackaged contributions of other writers. The writer’s brain doesn’t simply transform existing material. It adds material, too — material derived not from other writings, but from the writer’s own experience.
Each human being is a miracle of similarity and difference. We share so much with each other — a basic anatomy, a general psychological makeup, an assortment of basic desires and fears. And yet, our mix of circumstance and personality also makes us absolutely different from everyone else in the world.
This combination of similarity and difference is what makes communication in general, and art in particular, so rewarding. We share enough to understand each other, and differ enough to be fascinated by the ideas we can share, the perspectives we can offer, the stories we can tell. This interplay of similarity and difference lies at the heart of art. Every artistic endeavor carries, to varying degrees, an offer of “recognize yourself” and a demand to “recognize me.”
For art to astonish, it has to say something new about either our shared humanity or the creator’s unique perspective. An impressionist painter depicts a sun-splattered garden, and the viewer says, “Wow, now I realize it — that’s exactly how light affects certain landscapes!” Kerouac writes and writes and breathlessly writes — and young readers think, “Damn — so that’s how one can travel, that’s how one can live!” These creations weren’t rearrangements of existing artistic productions. Impressionist painters brought something new to their art, the understanding of how people truly see the world — and that understanding resonates with viewers. Kerouac brought something new to writing — the restlessness of a generation, the speed of bebop — and the work resonates, because readers feel that restlessness, too. An LLM, working only with what already exists, could never make these sorts of contributions.
The mistake people often make when assessing AI art is that they measure the extent to which the AI’s creations look like other pieces of art. But that misses the point. What makes art “art” is its relationship to the human psyche and its capacity for innovation. AI creations can “look” like art, but they can never “be” art, because they’re incapable of drawing directly from a human psyche to create something genuinely new.
LLMs draw their material from existing written materials. Writers draw from something unspeakably more vast: Human consciousness. Every day, I sit at my desk and attempt to turn the magical, bizarre, miraculous experience of being alive into words. It’s a damn-near impossible task. What I feel and experience is so much more complex than even the best poem or paragraph could ever express. My thoughts soar beyond the limits of the language I have to express them. Even now, as I write this essay, I sense how poor a job I’m doing.
What I mean is: The reservoir of human consciousness is mindbogglingly deep. When we write, we plumb those depths, trying to bring at least some of it to the surface. And if we’re doing even a halfway decent job, we bring back something that’s never been surfaced before.
LLMs aren’t plumbing those depths. They only have access to what’s already been surfaced (and uploaded to the web).
As a writer, let me humbly admit: Relative to the magnificence of the world and the complexity of the human psyche, what we’ve managed to surface, what we’ve managed to put into words, is a paltry haul indeed. Poor LLMs. That’s all they have to work with. They lack any access to the original thing.
I’ve got a publication coming in a week or two. Watch this space!