Is Evan Full of Shit? And Other Questions Machines Can Now Answer
Some notes on semantic surveillance
I recently asked Grok an important question. “Is Evan Armstrong full of shit?” As an analyst read by tens of thousands of people, I have a moral obligation to be accurate and accountable. You (hopefully) demand this of the content in your life. In an age of AI slop, we have to deliberately seek out sources of beauty and truth.
Until about three years ago, this was a surprisingly challenging thing to know about writers. For bloggers in particular, the topics we cover are so esoteric, and the volume so prodigious, that unless you have a week or two to burn reading past posts, you couldn’t really know if the person you were reading was actually good at their job.
Now, though, we have magic 8-balls powered by GPUs and nerdy math, which can read all that for us. Thus, my question to Grok. Its response?
“He’s a standard opinionated tech analyst who puts more effort into tracking his record than most. Not a pure bullshit artist on the available evidence. Somewhere in the low-to-mid range for a tech newsletter guy who makes predictions and analysis for a living—call it ~15-30%.”
Egomaniac that I am, I found this offensive. I am much better than your standard tech analyst! 30 percent is way too high. So to prove it wrong, I combined the might of ChatGPT and Claude to build the stupidest website in the history of the internet. Behold, my masterpiece:
It ingests everything I have published through April of this year, evaluates the forecasts that I made in this newsletter, and then scores me accordingly. I am very far from a programmer. Still, this type of analysis and presentation were remarkably easy to do. I used ChatGPT’s new Sites feature to build it in about two hours, but most of that was fiddling with the design. The algorithm design and scrape took maybe an hour and I did that with Claude Code.
Obviously, this was a silly exercise. But the point I am trying to make here is a larger one. If we give a language model an archive and a question, it will answer in a way that sounds convincing.
Everything this tool did to me, it can do to anyone with a paper trail. We have lived under mass data collection for years, but much of that data remained practically unreadable. Nobody could read every email, meeting transcript, Slack message, customer-service call, essay, post, or podcast. But now with LLMs anyone can just drop in the data and ask the machine what it thinks.
IsEvanFullOfShit.com is a joke website. But this new capability is something larger, a trend I’m calling “semantic surveillance.”
The business of surveillance
I define semantic surveillance as the automated interpretation of human communication at scale. It is easy to see why this is valuable to organizations.
Worried about nation-state enemies doing influence campaigns against you? Good news: there are “narrative intelligence” companies like PeakMetrics ($16.3 million) and Blackbird.AI ($58 million) that can track which narratives are spreading and who is advancing them.
How about hiring? Since job interviews are just two people talking, they are perfect candidates for this tech. If we semantically surveil the hiring process, we could make more data-driven decisions. Enter Metaview ($50 million, Series B led by GV) which sells interview note-taking and analysis.
Existing platforms are already launching these capabilities within the tools you use every day. Microsoft Purview Communication Compliance ships with Microsoft 365 and runs classifiers across Teams and Outlook for “discrimination, threats, harassment, corporate sabotage, regulatory collusion, and stock manipulation.” It even watches what employees type into Copilot! Zoom records sales calls and scores reps on sentiment, talk ratio, and objection handling. Anytime there is a database with conversations in it, the platforms are trying to monetize them.
In previous eras of software the monetization meant keyword counts and flagging “problem phrases.” (Remember a decade ago when everyone was doing word clouds?) But now with LLMs we can answer questions like, “Using Evan’s Slack messages, help me understand the underlying feelings he has about his manager.” The answer might be complete nonsense. But it’ll be cheap and fast. That is perhaps more important than accuracy. I’ve written in the past how computer vision models make it cheap and easy enough to discipline people’s bodies. LLMs make it cheap and easy enough to discipline people’s communication.
Which raises the only questions that matter. Who ends up wielding this? And what does it mean for everyone who’s ever left a paper trail? (That’s you.)
