Was this newsletter forwarded to you? Sign up to get it in your inbox.
Imagine a neighborhood diner: chrome-edged counters, white enamel mugs, phonebook-thick laminated menus, and, of course, cakes posing in the stage lights of the glass display case near the front.
You bounce in at 7 a.m., fresh from a run, and settle down in a red pleather booth that goes fwhump when you sit. A waiter passes by with a pot of hot coffee—he catches your eye and pours a mugful. For one clean second, the universe purrs.
Mid-menu, your eyes drift back to the cake case: Inside sits a triple‑layer chocolate erupting fudge like lava and a lemon meringue dome wobbling slightly—the dessert world’s seismograph.
“Egg-white omelet,” you say to your waiter. But a slice of the meringue shows up on the table before you finish talking.
“I saw you looking,” says the waiter with a wink and a bow.
It’s got egg whites at least. And hey, fruit! you think. Before you can pick up your fork, three slices of the triple-layer chocolate cake clatter onto the table, spilling the meringue into your lap.
“I ordered an omelet!” you yell, pointing at the menu. But you notice it has changed—the omelets have vanished, the salads have disappeared, even the coffee is gone. The menu has transformed into an endless variety of cake: red velvets, Boston creams, and strawberry shortcakes.
“Here, we only serve what you really want,” the waiter says.
You try to scream, but you can’t. Your mouth is full of cake.
Social media: A diner that serves you what you look at
For 15 years, the internet has been this diner. Social media served whatever our gaze grazed and our fingers clicked—what we call revealed preference—because that’s all the intent it could discern.
In many ways, this was great. We went from a one-size-fits-all media ecosystem to Khan Academy math videos, the ALS ice bucket challenge, BookTok, and this publication. Every would not exist without recommendation algorithms and social media. Instead I’d be pushing pencils at IBM and writing unpublished short fiction on the weekends in my gray little monoculture reality.
And yet, in many ways, this was not great. We tend to click on stuff that makes us outraged or horny. This turned social media into a cornucopia of empty calories: filter bubbles, flamewars, and doom-scrolling. We can’t stop eating, but we feel nauseous and ashamed at the end of the meal.
We thought this was the only way. We called our clicks revealed preferences because they’re supposed to be what we actually want, not just what we say we want. We began to view ourselves in this reduced form, just a clump of base desires masquerading as a well-intentioned human being.
Subscribe to read the full article
Ideas and Apps to
Thrive in the AI Age
The essential toolkit for those shaping the future
"This might be the best value you
can get from an AI subscription."
- Jay S.
Every Content
AI&I Podcast
Cora
Sparkle
Spiral
Join 100,000+ leaders, builders, and innovators
Email address
Already have an account? Sign in
What is included in a subscription?
Daily insights from AI pioneers + early access to powerful AI tools
Front-row access to the future of AI
In-depth reviews of new models on release day
Playbooks and guides for putting AI to work
Prompts and use cases for builders
In-depth reviews of new models on release day
Playbooks and guides for putting AI to work
Prompts and use cases for builders
Dan Shipper
Reviews of new AI models on release day
Comments
Don't have an account? Sign up!
Taking explicit preference into account was always possible, media companies (social and otherwise) just chose not to because it was more profitable to orient to the extremes and play our limbic systems like a fiddle. Remember the glory days of the internet before it was taken over by big companies and advertising? That's AI right now: we just haven't figured out how to best insert ads and other monetization into it yet. The flat monthly fees are working for now, but certainly 1000s of "innovators" are working on how they can wring more money out of AI users as we speak.
If we don't get to AGI first, we're very likely to just get a repeat of the commodification of our attention we already have outside of AI that you describe in such damning terms. Only with AI it could be supercharged to be even more effective at holding our attention because it understands us better, at least in some respect, for some definition of "understand". Imagine custom advertising that knows you well enough to manipulate *you specifically*, in direct ways customized to what it knows about you, e.g. "Don't let little Jimmy sleep without a custom plushy tonight, his nightmares have been getting worse!". Dystopian but plausible. I see little reason for optimism that capitalism will drive AI companies in a different way than it did for social media and the internet economy at large.
Also isn't AI Alignment largely about avoiding true harms, like helping terrorists build bombs, or helping hackers and scammers, or causing mental health issues, etc? I don't think much of that research and intent is oriented toward "Don't put profit over user preference". Though I wish it were!
Enlightening and a delight to read. Thank you, Dan.
Such a great analogy, Dan — it really brings to light the dynamic we're all caught in. We don't intend to be drawn toward the wreck on the side of the road, but attention gets hijacked before we even realize it. Social media has been engineered around that impulse — the irresistible glance — instead of genuine intent.
If AI can help recalibrate us back toward what we actually want to build, learn, and become... that's a game-changer. I think you're tapping into something essential here — the shift we've been needing.
Lovely analogy and I enjoyed reading it a lot. Great choice of words that made it feel like I was right there, especially at the start of the essay.
That said, I feel like there is a slight shortcoming in the logic. The argument goes from social media as revealed preference to AI which takes in our stated preference. This argument works if we were all not on social media anymore and instead mainly used AI chatbots for everything.
The truth, from my point of view, is that we say "AI is so much better at understanding what I want than social", yet we will still continue using social – despite the fact that it makes us feel outraged and envious.
AI reflects our higher selves. Social media panders to our lower selves. Both are now options and we often choose the latter, even when we know it's worse.