dirtyriver - Tumblr blog

dirtyriver · 4 hours ago

Text

How twiddling enshittifies your brain

Hey, German-speakers! Through a very weird set of circumstances, I ended up owning the rights to the German audiobook of my bestselling 2022 cryptocurrency heist technothriller Red Team Blues and now I'm selling DRM-free audio and ebooks, along with the paperback (all in German and English) on a Kickstarter that runs until August 11.

"If your customers are too happy, you're leaving money on the table": it's the rallying cry of the enshittifier, and it's also what a friend of mine was told by a respected professor in a top-tier MBA program.

Enshittification is the theory that if platforms can shift value away from workers, suppliers, users and/or customers without facing consequences, we should expect that they will. A company is a colony organism made up of many differing organelles, some of whom have firm moral centers and good values, but those factions can't win an argument about enshittifying the company's offerings merely by gesturing towards their ethical reservations. To win that argument, the good guys have to be able to appeal to a villain's highest priority: their own self-interest. It's one thing to say, "I'll feel gross if we wreck our product this way," but it's another altogether to say, "We'll go broke – because of fines, or employee defections, or competitor poaching, or interoperable blocking tech – if we do it your way":

https://pluralistic.net/2023/07/28/microincentives-and-enshittification/

Someone in the org is always ready to believe that the customers (or workers, or suppliers) are too happy, and that this represents money left on the table. Customer service can be scaled back, wages cut, free features turned into upsells. Some of capitalism's most imaginative inventors are enshittifiers, dreaming up new ways to sell you to yourself.

The great tragedy of all this is that the more useful and important a service becomes to you, the more the service's proprietors can extract from you. They don't care if you hate them, so long as you love the data, the friends, the productivity, the utility you get from the service more.

Writing in Ethics and Information Technology, Louisiana State's Michael J Ardoline and Muhlenberg College's Edward Lenzo write about another one of enshittification's systematic torments: "The cognitive and moral harms of platform decay":

https://link.springer.com/article/10.1007/s10676-025-09846-1

The authors observe that our technologies quickly turn into cognitive prostheses: as soon as we can externalize some function of our thinking into a technology, we do.

I used to walk around with a hundred phone numbers in my head, now I remember two, maybe three on a good day. Which is fine! Sure, remembering those phone numbers wasn't cognitively useless. I cultivated all kinds of clever mnemonics based on the spatial relationships of the phone buttons, their alphabetical equivalents, the tones they made, and the arithmetic relationships between sequential digits, all of which constituted a kind of cognitive workout. But after the Great Telephone Number Forgettering, I retasked all that cognitive capacity to memorizing and thinking about stuff that's much less arbitrary and far more consequential than phone numbers.

Whenever we adopt a cognitive prosthesis, there's always someone who overweights the value of the old system of unassisted thinking, while ignoring the cool things we can do with the free capacity we get from replacing our fallible and scarce meat-thinkers with something reproducible and external. No one is immune to this: Socrates thought that reading would make us all stupid because we'd lose the discipline of memorizing all works of literature (ironically, we only know that Socrates thought this because Plato wrote it down):

https://wondermark.com/socrates-vs-writing/

Versions of this continue to play out. When I was a kid, there was a moral panic that pocket calculators would make us all innumerate (an argument advanced by people who know so little about mathematics that they think it's the same thing as arithmetic). Now I keep hearing about millennials who can't read an analog clock, a skill that has as much objective utility as knowing how to interpret a slide-rule or convert from Francs to Lire to Deutschemarks. Not actually useless, but entirely bound to a specific time and place and a mere historical curiosity at some later date.

So I love cognitive prostheses. As a perennially disoriented man with innately poor spatial reasoning and consequently no ability to parse a map, I fucking love living in the age of turn-by-turn GPS directions.

If you wanna know how I write 2-3 books per year, blame the cognitive prosthesis of blogging, which forces me to apply rigor to the notes I take, and rewards me with a searchable database of everything I've ever found important, while stimulating a constant mnemonic rejuggling of all those thoughts that crystallizes into an endless stream of novel synthetic insights and road-tested ways to express them:

https://pluralistic.net/2021/05/09/the-memex-method/

My blogging is self-hosted, and for good reason. An asset that important to my personal and professional life is too precious to entrust to any kind of third party service, especially in light of the collapse of discipline that prevents firms from enshittifying. Remember, the enshittifier's motto is "If your customer is too happy, you're leaving money on the table." My digital, networked online notebook makes me very happy indeed, which means that if it were under the control of an enshittotropic colony organism like Google or Apple or Microsoft or Meta, it would only be a matter of time until some dominant faction decided to see how much they could extract from me by holding it to ransom or making it worse.

It's not practical for everyone to self-host everything. I'm blessed with a lot of technical knowledge and the incredible talents and generosity of a brilliant sysadmin, the wonderful Ken Snider, who makes it all go for me. I've known Ken for 20+ years and the man is no enshittifier. But most of us don't have a Ken in our lives, and even fewer of us are Ken, and so perforce, most of us end up externalizing large parts of our brains to networked services run by companies that would enshittify you without a second thought.

Trusting these companies with so much of your life can be catastrophic, because they are manifestly too big to care, which is why you can't get a customer service rep to save your life (and why they're turning over their vestigial customer service functions to chatbots, AKA "the Idgaf Gambit").

Take the case of "Mike," a software developer whose infant son developed a UTI during the covid lockdowns. On advice from his pediatrician, Mike took a picture of his son's infected penis with his Android phone and sent it to the doctor using a secure telemedicine app, forgetting that his Android device would also automatically sync all his photos to Google's cloud. Google automatically scans all these photos, and it flagged this one as child sexual abuse material (AKA "child pornography"), which resulted in the termination of all of Mike's Google services.

In an instant, Mike lost every family photo he'd taken since his son's birth, every saved email, all of his business and tax records in his Google Drive, his phone number (he was a Google Fi subscriber), his authenticator app, and his email address itself. Google handed his search history and many other sensitive records they held on him to the San Francisco Police Department, who concluded that everything was fine. But the cops couldn't tell Mike any of this because he had no phone and no email, and, lacking these, could not recover any of his online accounts. Eventually, an SFPD detective had to ring Mike's doorbell to tell him he was cleared of any wrongdoing. Despite this, Mike never got his accounts or data back:

https://locusmag.com/2024/07/cory-doctorow-unpersoned/

This is an accidental lobotimization of your outboard brain – it's what happens when a company that's too big to care drops one of its procedures on your head and crushes it like a grape. But there is an important sense in which these companies do care: they care whether you hate them more than you value the data and connections and utility they control. They care about this because if you're too happy, they're leaving money on the table.

That's where Ardoline and Lenzo's work comes in. They both document the ways in which we turn these online services into cognitive prostheses, and then investigate how the enshittification of these services ends up making us stupider, by taking away the stuff that helps us think. They're drawing a line between platform decay and cognitive decay.

The authors look at examples like the enshittification of Google Search, a product that Google has deliberately and irretrievably enshittified:

https://pluralistic.net/2024/04/24/naming-names/#prabhakar-raghavan

The web is a giant cognitive prosthesis, and early web tools put a lot of emphasis on things like bookmark management and local caching, so that the knowledge and cognition you externalized to the web were under your control. But Google Search was so goddamned magic – before they cynically destroyed it – that a lot of us switched from "not remembering things because you have a bookmark that takes you to a website that remembers it for you" to "not remembering things and not remembering where to find them, and just typing queries into Google." The collapse of Google into a giant pile of shit is like giving every web user a traumatic brain injury.

It's a good paper, but I think the situation is actually more dire than the paper makes it out to be, thanks to the AI bubble –

Wait! I'm not actually going to talk about what AI can do (which is a combination of a small set of boring useful things, a bunch of novelties, and a long list of things that AI can't do but is being used to do anyway). I'm talking about the financial fraud that AI serves.

Tech companies must be perceived as growing, because when a company is growing, it is valued far more highly than a company is once it has "matured." This is called the "price to earnings ratio" – the number of dollars investors are willing to pay for the company compared to the number of dollars a company is bringing in. So long as a company is growing, the PE ratio is very high, and this helps the company to actually grow. That's because the shares in growing companies are highly liquid, and can be traded for equity in other companies and/or the labor of key employees, meaning that growth companies can almost always outbid their mature counterparts when it comes to expanding through acquisition and hiring. That means that while a company is growing, its PE ratio can help it keep growing.

But here's the corollary: when a growth company stops growing, its shares are suddenly and violently revalued as though they were shares in a mature company, which tanks the personal net worth of the company's top managers and key employees (whose portfolios are stuffed with their employer's now-plummeting stock). Worse: in order to retain those employees and hire more (or to acquire key companies), the no-longer-growing company has to pay with cash, which is much harder to get than its own shares. Even worse: they have to bid against growing companies.

A growth company is like an airplane that has two modes: climbing and nose-diving, and while it's easy to go from climbing to crashing, it's much harder to go the other way. Ironically, the moment at which a company's growth is most likely to stall is right after its greatest triumph: after a company conquers its market, it has nowhere else to go. Google's got a 90% Search market-share – how can it possibly grow Search?

It can't (just like Meta can't really grow social, and Microsoft can't grow office suites, etc), so it has to convince Wall Street that it has a shot at conquering some other market that the street perceives as unimaginably vast and thus capable of keeping the growth engine going. Tech has pulled a lot of sweaty tricks to create this impression, inflating bubbles like "pivot to video" and "metaverse" and "cryptocurrency," and now it's AI.

The problem is that AI just isn't very popular. People go out of their way to avoid AI products:

https://www.tandfonline.com/doi/full/10.1080/19368623.2024.2368040

For an AI-driven growth story to work, tech companies have to produce a stream of charts depicting lines that go up and to the right, reflecting some carefully chosen set of metrics demonstrating AI's increasing popularity. One way to produce these increasing trend-lines on demand is to replace all the most commonly used parts of a service that you love and rely on with buttons that summon an AI. This is the "fatfinger AI economy," a set of trendlines produced by bombarding people who graze their screens with a stray fingertip with a bunch of AI bullshit, so you can claim that your users are "engaging" with AI:

https://pluralistic.net/2025/05/02/kpis-off/#principal-agentic-ai-problem

It's a form of "twiddling" – changing how a service works on a per-user, per-interaction basis in order to shift value from the user to the company:

https://pluralistic.net/2023/02/19/twiddler/

Twiddling represents the big cognitive hazard from enshittification during the AI bubble: the parts of your UI that matter most to you are the parts that you use as vital cognitive prostheses. A product team whose KPI is "get users to tap on an AI button" is going to use the fine-grained data they have on your technological activities to preferentially target these UI elements that you rely on with AI boobytraps. You are too happy, so they are leaving money on the table, and they're coming for it.

This is a form of "attention rent": the companies are taxing your muscle-memory, forcing you to produce deceptive usage statistics at the price of either diverting your cognition from completing a task to hunt around for the button that banishes the AI and lets you get back to what you were doing; or to simply abandon that cognitive prosthesis:

https://pluralistic.net/2023/11/03/subprime-attention-rent-crisis/#euthanize-rentiers

It's true "engagement-hacking": not performing acts of dopamine manipulation; but rather, spying on your habitual usage of a digital tool in order to swap buttons around in order to get you to make a number go up. It's exploiting the fact that you engage with something useful and good to make it less useful and worse, because if you're too happy, some enshittifier is leaving money on the table.

Support me this summer in the Clarion Write-A-Thon and help raise money for the Clarion Science Fiction and Fantasy Writers' Workshop! This summer, I'm writing The Reverse-Centaur's Guide to AI, a short book for Farrar, Straus and Giroux that explains how to be an effective AI critic.

If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:

https://pluralistic.net/2025/07/28/twiddlehazard/#outboard-brains-considered-harmful

Image: Stephen Drake (modified) https://commons.wikimedia.org/wiki/File:Analog_Test_Array_modular_synth_by_sduck409.jpg

CC BY 2.0 https://creativecommons.org/licenses/by/2.0/deed.en

138 notes · View notes