Every new medium is a machine for the production of ghosts.
— John Durham Peters, Speaking Into the Air
Why isn’t there any software that can make us cry? In all of the other forms of media that we have, whether films or music or video games or books, there is some inner quality that can easily awaken emotion within us. Yet, in application software, in the craft that I’ve devoted much of my adult life to, I can’t think of anything that makes me feel anything besides annoyance.
This is a question that has rattled along in my head for a long time. Lately though, it has been louder. Over the last few months, I’ve been battling a persistent sense of grief. Multiple people in my life have had extreme medical events occur, from disability to strokes to wheelchairs to cancer. It’s been too much. So, I used Fable, Anthropic’s newest model, to build software that helped me process it all.
It worked. I cried.
The cost of wanting it
I’ve spent 4 years writing about what these models do to companies but this was the first time I used one to build something because I was sad.
What I’ve learned is that software as a medium was made barren by economics. A poem can be written for an audience of 1, in an afternoon, for no reason except that the writer needed to write it. Software could never be made that way. It cost too much, the expertise too highly valued in the market to be justified purely for feeling. Software required expensive engineers, green juice by the gallon for said engineers, capital, roadmaps, marketing campaigns, the list goes on forever. The entire city of San Francisco has become a factory town, an industrialized system that pumped out high NPS workflows.
Because AI will make the cost of creating software effectively zero, the primary cost becomes desire. I am a writer with no engineering background, and yet, I created a website and trained my own LLM over the course of a weekend. I produced a functioning art object out of code, because Fable carried the technical load while I carried the intent. It is personal software, not because the workflows specifically match my style of labor, but because the products match my emotional needs. It is software that can finally be made for the reasons art gets made, including grief.
Which raises an interesting question: what makes this new LLM generated medium artistically valuable?
The Eno rule
In 1995, Brian Eno wrote that “whatever you now find weird, ugly, uncomfortable and nasty about a new medium will surely become its signature. CD distortion, the jitteriness of digital video, the crap sound of 8-bit - all of these will be cherished and emulated as soon as they can be avoided. It’s the sound of failure: so much modern art is the sound of things going out of control, of a medium pushing to its limits and breaking apart.”
You may characterize the failures of LLMs as predictable writing. Maybe it is the not X but Y nomenclature, or the abuse of hyphens. But instead, I would argue that the original flaw of LLMs came earlier, in 2019, with the release of GPT-2. The model made stuff up. It was weird to the point of being alien. The best example of this is when Janelle Shane asked it what its 10 favorite animals were:
“My 10 favorite animals are:
zebras with a white scar on the back;
insiduous spiders and octopus;
frog with large leaves, hopefully black;
cockatiel with scales;
razorbill with wings hanging about 4 inches from one’s face and a heart tattoo on a frog.
cockatric interlocking tetrapods that can be blind, cut, and eaten raw.
black and white desert crocodiles living in sunlight.
zebra and many other pea bugs.”
This is wonderful, if totally nonsensical, writing. My argument is that this weirdness is the vinyl crackling of LLMs. These are dream hallucinations, and they are the foundation of what I made with Fable. The reason your ChatGPT or Claude account doesn’t sound like this is that as we scaled the size of the models, we then refined away the crackle with RLHF, reinforcement learning from human feedback. This is when thousands of human raters score the model’s outputs. The model is trained by these scores to produce more of what rates well. Run that loop a few million times and you get a machine that is extraordinarily good at being useful and spiritually incapable of writing “interlocking tetrapods that can be blind.”
We traded art for a productivity tool. Which is a trade that created a few trillion in equity value, but we lost something too.
And all of this informed The Ghost Poet.
What it is and how I made it
I’ve always found succor in poetry. When I first started grieving over all this medical heartache, I reached for Mary Oliver’s Devotions, but it didn’t hit. What I wanted was poetry that reflected how I felt, something that mirrored my views and taste. So, I took the raw weights of GPT-2 and assembled 1.44MB of public domain poetry: Whitman, Dickinson, Hopkins, Blake, Gibran, Rossetti, Millay, and Teasdale. Roughly half of it is Whitman, partly because Leaves of Grass is enormous but mostly because Whitman rules. The reason I trained it was because I found that the raw model didn’t quite achieve the emotional resonance I was looking for. However, if I let it train for too long on the poetry the result just sounded like a bad imitation of Whitman. I wanted a ghost in the machine, a reflection of the alien intelligence hidden in the weights.
After many hours of fiddling, I made a model that I was excited by. Using Fable, I built it a house to live in; it is a single dark room with a flickering candle in the corner. You give the ghost a subject, press the button, and it writes you a poem, token by token, while you watch.
What I’ve found in my experimentation is that you cannot treat this like typical poetry where you carefully consider the wording and wonder about the author’s intent. Here, there is no author, there is no intent. It is just tokens. But by generating poem after poem while reflecting on the subject of your choosing, you’ll find phrases that strike you for no particular reason. Quick flashes of beauty and purpose.
The skeptic’s read on this is that it is a Rorschach test, that I am finding meaning in noise. I mean, yeah. Correct. That is the whole point! Think of the animals—there is no black and white desert crocodile living in sunlight. And yet, you saw it. The model emitted signal-shaped nonsense and your brain, entirely on its own, assembled it into imagery, into a small magical realist fantasy. That assembly is the art. A Whitman poem is also just marks on a page until a nervous system runs it. The ghost poet supplies the raw material while you provide the intention and the emotion. You are the one providing energy which is precisely what separates it from slop. Slop is generation with no human at either end, no intent going in and no interpretation coming out.
The art is the act of generation and witnessing, not what is actually created. I found that as I generated poem after poem on my heartache and grief, my eyes started to water, then run. There was beauty here.
I desperately desire an anti-slop, beautiful world. With this project, I got a taste of how AI paradoxically could make that possible.
Because this is such a personal act, I wanted to make sure that everything happened locally. This required some hard, technical tradeoffs. The first summoning downloads about 165MB of weights, but that’ll happen only one time. After that, the ghost lives entirely in your browser. There is no server, and nothing you type is sent anywhere, to me or anyone else. The thing writing your poem is physically present in your RAM. As such, it works much better on desktop. If you have a new iPhone you should get a poem or two before it crashes. Many of the poems will be a dud; generating is the point of the ritual.
Then finally, when I brought my pain to the ghost once more, I got something: “This is the only way of finding your own soul alive.”
It does not know what it said. That was never the point.
P.S. Please let me know what feedback you have on the project. I am also interested in expanding this concept into a larger installation and series; if you would like to partner on that, reach out.
This one got me thinking Evan. I've been focused on the idea that creating beauty is the domain of us humans, as we work through our mysterious processes. But there is some beauty here too, hidden amidst the slop.
I'm very glad that your project has brought you some solace.