#botshit

[mostlysignssomeportents]

When Google's slop meets webslop, search stops

Support me this summer in the Clarion Write-A-Thon and help raise money for the Clarion Science Fiction and Fantasy Writers' Workshop! This summer, I'm writing The Reverse-Centaur's Guide to AI, a short book for Farrar, Straus and Giroux that explains how to be an effective AI critic.

It's been more than a year since I gave up on Google Search (I switched to Kagi.com and never looked back). I don't miss it. It had gotten terrible. It's gotten worse since, thanks to AI (of course):

https://pluralistic.net/2024/04/04/teach-me-how-to-shruggie/#kagi

Google's a very bad company, of course. I mean, the company has lost three federal antitrust trials in the past 18 months. But that's not why I quit Google Search: I stopped searching with Google because Google Search suuuucked.

In the spring of 2024, it was clear that Google had lost the spam wars. Its search results were full of spammy garbage content whose creators' SEO was a million times better than their content. Every kind of Google Search result was bad, and results that contained the names of products were the worst, an endless cesspit of affiliate link-strewn puffery and scam sites.

It's not that the internet lacks for high-quality, reliable reviews. There are plenty of experts out there who subject a wide range of products to careful assessment, laboratory tests, and extensive comparisons. The sites where these reviews appear are instantly recognizable, and it's a great relief to find them.

One such site is Housefresh.com, whose proprietor, Giselle Navarro, runs a team that produces extremely detailed, objective, high-quality reviews of air purifiers. This is an important product category: if you're someone with bad allergies or an immunocompromising condition, finding the right air purifier can exert enormous influence on your health outcomes.

As good as Housefresh are at reviewing air purifiers, they are far less skilled at tricking Google. The world champions of this are spammers, content farms that produce garbage summaries of Amazon reviews and shovel them into massive, hidden sections of once-reputable websites like Forbes.com and Better Homes and Gardens, and thus dominate the Google results for product review searches:

https://pluralistic.net/2024/05/03/keyword-swarming/#site-reputation-abuse

Google calls this "site reputation abuse" and has repeatedly vowed to put a stop to it, and has repeatedly, totally failed to do so. What's more, Google has laid off more than 10,000 workers, including "core teams," even while spending tens of billions of dollars on stock manipulation through "buyback" schemes:

https://qz.com/google-is-laying-off-hundreds-as-it-moves-core-jobs-abr-1851449528

Of course, the Housefresh team are smart cookies – hence the high caliber of their air purifier reviews – and they could apply that intelligence to figuring out how to use SEO to trick Google's algorithm. Rather than doing so, they took the high road: they applied all that prodigious analytical talent to researching and publishing on Google's systematic failures – and even collusion – with the spammers who are destroying the web.

This month, Housefresh released its latest report on Google's enshittification, this time with an emphasis on the "AI Overviews" that now surmount every search results page. Google has widely touted these as the future of search, a way to bypass the ad-strewn, popup-obscured, AI-sloppified (!) pages that it is seemingly powerless to filter out of its search corpus:

https://housefresh.com/beware-of-the-google-ai-salesman/

Rather than hunting through these SEO-winning garbage pages, you can simply refer to Google's AI Overview, which will summarize the best the internet has to offer, in hyperlegibile black sans-serif type on a white background, with key phrases helpfully highlighted in bold.

Most critiques of AI Overview have focused on how these AI Overviews are a betrayal of the underlying bargain between the web and its monopoly search engine, whereby we all write the web and let Google index it for free, and in exchange, Google will send us traffic in proportion to the quality of our work:

https://www.bbc.com/future/article/20250611-ai-mode-is-google-about-to-change-the-internet-forever

This is true, as far as it goes, but it doesn't go far enough. Google is a platform, which is to say, a two-sided marketplace that brings together readers and publishers (along with advertisers). The bargain with publishers is that Google will send them traffic in exchange for access to their content. But the deal with readers is that Google will help them answer their questions quickly and accurately.

If Google's marketing pitch for AI Overviews is to be believed, then Google is only shafting publishers in order to double down on its bargain with readers: to give us faster, better access to high-quality information (recall Google's mission statement, "To organize the world's information and make it useful"). If that's true, then Google is the champion of readers in their long battle with publishers, a battle in which they are nearly helpless before publishers' abusive excesses.

This is a very canny move on Google's part. Publishers and advertisers have more concentrated money than readers, but the dominant theory of antitrust since the Reagan administration is something called "consumer welfare," which holds that monopolistic conduct is only to be condemned if it makes consumers worse off. If a company screws its workers or suppliers in order to deliver better products and/or better prices, then "consumer welfare" holds that the government should celebrate and protect the monopolist for improving "efficiency."

But all that is true only if Google AI Overviews are good. And they are very, very bad.

In the Housefresh report, titled "Beware of the Google AI salesman and its cronies," Navarro documents how Google's AI Overview is wildly bad at surfacing high-quality information. Indeed, Google's Gemini chatbot seems to prefer the lowest-quality sources of information on the web, and to actively suppress negative information about products, even when that negative information comes from its favorite information source.

Indeed, Navarro identifies a kind of madlibs template that Gemini uses to assemble an AI overview in response to the query "Is the [name of air purifier] worth it?"

The [model] air purifier is [a worthwhile investment/generally considered a good value for its price/a worthwhile purchase]. It's [praised/well-regarded] for its ability to [clean the air/remove particles/clean large rooms]. Whether the [product] is worth it depends on individual needs and priorities.

This is the shape of the response that Google's AI Overview shits out when you ask about any air purifier, including a model that Wirecutter called "the worst air purifier ever tested":

https://www.nytimes.com/wirecutter/blog/worst-air-purifier-we-ever-tested/

What's more, AI Overview will produce a response like this one even when you ask it about air purifiers that don't exist, like the "Levoit Core 5510," the "Winnix Airmega" and the "Coy Mega 700."

It gets worse, though. Even when you ask Google "What are the cons of [model of air purifier]?" AI Overview simply ignores them. If you persist, AI Overview will give you a result couched in sleazy sales patter, like "While it excels at removing viruses and bacteria, it is not as effective with dust, pet hair, pollen or other common allergens." Sometimes, AI Overview "hallucinates" imaginary cons that don't appear on the pages it cites, like warnings about the dangers of UV lights in purifiers that don't actually have UV lights.

Google argues that AI Overview won't displace traffic to the sites it summarizes. The company points to the fact that the statements in an AI Overview are each linked to the web-page they come from. This is a dubious proposition, predicated on the idea that people looking up a quick answer on a search engine will go on to follow all the footnotes and compare them to the results (this is something that peer reviewers for major scientific journals often fail at, after all).

But the existence of these citations allowed Navarro to compile statistics about the sources that Google relies on most heavily for information about product quality:

43.1% of these statements come from product manufacturers' marketing materials;

19.5% of these statements are sourced from pages that contain no information about the product.

Much of the remainder comes from the same "site reputation abuse" that Google said it would stop prioritizing two years ago. An alarming amount of this material is also AI generated: this is the "coprophagic AI" problem in which an AI ingests another AI's output, producing ever-more nonsensical results:

https://pluralistic.net/2024/03/14/inhuman-centipede/#enshittibottification

The balance is primarily drawn from Reddit, who announced a major partnership with Google as part of the company's IPO:

https://blog.google/inside-google/company-announcements/expanded-reddit-partnership/

Adding "reddit" to a Google query is a well-known and still-useful way to get higher quality results out of Google. Redditors is full of real people giving their real opinions about products and services. No wonder that Reddit appears in 97.5% of product review queries:

https://detailed.com/forum-serps/

Obviously, the same SEO scumbags who have been running circles around Google for years are perfecctly capable of colonizing and compromising Reddit, which has been rocked by a series of payola scandals in which the volunteer moderators of huge, reputable subreddit were caught taking bribes to allow SEO scumbags to spam their forums and steal their valor:

https://web.archive.org/web/20250607050622/https://www.reddit.com/r/TheseFuckingAccounts/comments/1kzzsuv/update_reddit_admins_have_escalated_the_paradise/

When it comes to product reviews, Google's AI Overviews consist of irrelevancies, PR nonsense, and affiliate spammer hype – all at the expense of genuine, high-quality information, which is still out there, on the web, waiting for you to find it.

Google CEO Sundar Pichai is unapologetic about the way that AI Overviews blurs the line between commercial pitches and neutral information, telling Bloomberg, "commercial information is information, too":

https://www.bloomberg.com/news/features/2025-03-24/google-s-ai-search-overhaul-racing-chatgpt-for-the-web-s-future

Which raises the question: why is Pichai so eager to enshittify his own service? After all, AI isn't a revenue center for Google – it's a cost center. Every day, Google's AI division takes a blowtorch to the company's balance sheet, incinerating mountains of money while bringing in nothing (less than nothing, if you count all the users who are finding ways to de-Google their lives to escape the endless AI slop):

https://pluralistic.net/2025/06/30/accounting-gaffs/#artificial-income

It's true that AI loses money for Google, but AI earns something far more important (at least from Pichai's perspective): a story about how Google can continue to grow.

Google's current price-to-earnings (PE) ratio is 20:1. That means that for every dollar Google brings in, investors are willing to spend $20 on Google's stock. This is a very high PE ratio, characteristic of "growth stocks" (companies that are growing every year). A high PE ratio tells you that investors anticipate that the company will get (much) bigger in the foreseeable future, and they are "pricing in" that future growth when they trade the company's shares.

Companies with high PE ratios can use their stock in place of money – for example, they can acquire other companies with stock, or with a mix of cash and stock. This lets high PE companies outbid mature companies – companies whose growth phase has ended – because stock is endogeous (it is produced within the company, by typing zeroes into a spreadsheet) and therefore abundant, while dollars are exogenous (produced by the central bank – again, by typing zeroes into a spreadsheet! – and then traded to the company by its customers) and thus scarce.

Google's status as a growth stock has allowed it to buy its way to dominance. After all, Google has repeatedly, continuously failed to create new products in-house, relying on acquisitions of other people's companies for its mobile technology, ad-tech, server management, maps, document collaboration…virtually every successful product the company has (except Search).

For so long as investors believe Google is growing, it can buy other companies with its abundant stock rather than its scarce dollars. It can also use that stock to hire key personnel, which especially important for AI teams, where compensation has blasted through the stratosphere:

https://www.reuters.com/business/zuckerbergs-meta-superintelligence-labs-poaches-top-ai-talent-silicon-valley-2025-07-08/

But that just brings us back to the original question: why build an AI division at all?

Because Google needs to keep up the story that it is growing. Once Google stops growing, it becames a "mature" company and its PE ratio will fall from 20:1 to something more like 4:1, meaning an 80% collapse in the company's share price. This would be very bad news for Googlers (whose personal wealth is disproportionately tied up in Google stock) and for Google itself (because many of its key personnel will depart when the shares they've banked for retirement collapse, and new hires will expect to be paid in scarce dollars, not abundant stock). For a company like Google, "maturity" is unlikely to be a steady state – rather, it's likely to be a prelude to collapse.

Which is why Google is so desperately sweaty to maintain the narrative about its growth. That's a difficult narrative to maintain, though. Google has 90% Search market-share, and nothing short of raising a billion humans to maturity and training them to be Google users (AKA "Google Classroom") will produce any growth in its Search market-share. Google is so desperate to juice its search revenue that it actually made search worse on purpose so that you would have to run multiple searches (and see multiple rounds of ads) before you got the information you were seeking:

https://pluralistic.net/2024/04/24/naming-names/#prabhakar-raghavan

Investors have metabolized the story that AI will be a gigantic growth area, and so all the tech giants are in a battle to prove to investors that they will dominate AI as they dominated their own niches. You aren't the target for AI, investors are: if they can be convinced that Google's 90% Search market share will soon be joined by a 90% AI market share, they will continue to treat this decidedly tired and run-down company like a prize racehorse at the starting-gate.

This is why you are so often tricked into using AI, by accidentally grazing a part of your screen with a fingertip, summoning up a pestersome chatbot that requires six taps and ten seconds to banish: companies like Google have made their product teams' bonuses contingent on getting normies to "use" AI and "use" is defined as "interact with AI for at least ten seconds." Goodhart's Law ("any metric becomes a target") has turned every product you use into a trap for the unwary:

https://pluralistic.net/2025/05/02/kpis-off/#principal-agentic-ai-problem

There's a cringe army of AI bros who are seemingly convinced that AI is going to become superintelligent and save us from ourselves – they think that AI companies are creating god. But the hundreds of billions being pumped into AI are not driven by this bizarre ideology. Rather, they are the product of material conditions, a system that sends high-flying companies into a nosedive the instant they stop climbing. AI's merits and demerits are irrelevant to this: they pump AI because they must pump. It's why they pumped metaverse and cryptocurrency and every other absurd fad.

None of that changes the fact that Google Search has been terminally enshittified and it is misleading billions of people in service to this perverse narrative adventure. Google Search isn't fit for purpose, and it's hard to see how it ever will be again.

If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:

https://pluralistic.net/2025/07/14/pole-star/#gnus-not-utilitarian

Image: Cryteria (modified) https://commons.wikimedia.org/wiki/File:HAL9000.svg

CC BY 3.0 https://creativecommons.org/licenses/by/3.0/deed.en

--

Radek Kołakowski (modified) https://commons.wikimedia.org/wiki/File:Warszawski_smog_(22798350941).jpg

CC BY 2.0 https://creativecommons.org/licenses/by/2.0/deed.en

[mckitterick]

And for what?

soulless botshit that moves no one.

source: X

more on Wendig's blog here: X

[azspot]

Then there is the growing use of generative A.I. to scale the creation of cheap synthetic videos for children on YouTube. Some example outputs are Lovecraftian horrors, like music videos about parrots where the birds have eyes within eyes, beaks within beaks, morphing unfathomably while singing in an artificial voice “The parrot in the tree says hello, hello!” The narratives make no sense, characters appear and disappear randomly, basic facts like the names of shapes are wrong. After I identified a number of such suspicious channels on my newsletter, The Intrinsic Perspective, Wired found evidence of generative A.I. use in the production pipelines of some accounts with hundreds of thousands or even millions of subscribers.

AI Garbage Is Already Polluting the Internet

[mckitterick]

honestly, my darling would probably love this

[mostlysignssomeportents]

Google is (still) losing the spam wars to zombie news-brands

I'm touring my new, nationally bestselling novel The Bezzle! Catch me TONIGHT (May 3) in CALGARY, then TOMORROW (May 4) in VANCOUVER, then onto Tartu, Estonia, and beyond!

Even Google admits – grudgingly – that it is losing the spam wars. The explosive proliferation of botshit has supercharged the sleazy "search engine optimization" business, such that results to common queries are 50% Google ads to spam sites, and 50% links to spam sites that tricked Google into a high rank (without paying for an ad):

https://developers.google.com/search/blog/2024/03/core-update-spam-policies#site-reputation

It's nice that Google has finally stopped gaslighting the rest of us with claims that its search was still the same bedrock utility that so many of us relied upon as a key piece of internet infrastructure. This not only feels wildly wrong, it is empirically, provably false:

https://downloads.webis.de/publications/papers/bevendorff_2024a.pdf

Not only that, but we know why Google search sucks. Memos released as part of the DOJ's antitrust case against Google reveal that the company deliberately chose to worsen search quality to increase the number of queries you'd have to make (and the number of ads you'd have to see) to find a decent result:

https://pluralistic.net/2024/04/24/naming-names/#prabhakar-raghavan

Google's antitrust case turns on the idea that the company bought its way to dominance, spending the some of the billions it extracted from advertisers and publishers to buy the default position on every platform, so that no one ever tried another search engine, which meant that no one would invest in another search engine, either.

Google's tacit defense is that its monopoly billions only incidentally fund these kind of anticompetitive deals. Mostly, Google says, it uses its billions to build the greatest search engine, ad platform, mobile OS, etc that the public could dream of. Only a company as big as Google (says Google) can afford to fund the R&D and security to keep its platform useful for the rest of us.

That's the "monopolistic bargain" – let the monopolist become a dictator, and they will be a benevolent dictator. Shriven of "wasteful competition," the monopolist can split their profits with the public by funding public goods and the public interest.

Google has clearly reneged on that bargain. A company experiencing the dramatic security failures and declining quality should be pouring everything it has to righting the ship. Instead, Google repeatedly blew tens of billions of dollars on stock buybacks while doing mass layoffs:

https://pluralistic.net/2024/02/21/im-feeling-unlucky/#not-up-to-the-task

Those layoffs have now reached the company's "core" teams, even as its core services continue to decay:

https://qz.com/google-is-laying-off-hundreds-as-it-moves-core-jobs-abr-1851449528

(Google's antitrust trial was shrouded in secrecy, thanks to the judge's deference to the company's insistence on confidentiality. The case is moving along though, and warrants your continued attention:)

https://www.thebignewsletter.com/p/the-2-trillion-secret-trial-against

Google wormed its way into so many corners of our lives that its enshittification keeps erupting in odd places, like ordering takeout food:

https://pluralistic.net/2023/02/24/passive-income/#swiss-cheese-security

Back in February, Housefresh – a rigorous review site for home air purifiers – published a viral, damning account of how Google had allowed itself to be overrun by spammers who purport to provide reviews of air purifiers, but who do little to no testing and often employ AI chatbots to write automated garbage:

https://housefresh.com/david-vs-digital-goliaths/

In the months since, Housefresh's Gisele Navarro has continued to fight for the survival of her high-quality air purifier review site, and has received many tips from insiders at the spam-farms and Google, all of which she recounts in a followup essay:

https://housefresh.com/how-google-decimated-housefresh/

One of the worst offenders in spam wars is Dotdash Meredith, a content-farm that "publishes" multiple websites that recycle parts of each others' content in order to climb to the top search slots for lucrative product review spots, which can be monetized via affiliate links.

A Dotdash Meredith insider told Navarro that the company uses a tactic called "keyword swarming" to push high-quality independent sites off the top of Google and replace them with its own garbage reviews. When Dotdash Meredith finds an independent site that occupies the top results for a lucrative Google result, they "swarm a smaller site’s foothold on one or two articles by essentially publishing 10 articles [on the topic] and beefing up [Dotdash Meredith sites’] authority."

Dotdash Meredith has keyword swarmed a large number of topics. from air purifiers to slow cookers to posture correctors for back-pain:

https://housefresh.com/wp-content/uploads/2024/05/keyword-swarming-dotdash.jpg

The company isn't shy about this. Its own shareholder communications boast about it. What's more, it has competition.

Take Forbes, an actual news-site, which has a whole shadow-empire of web-pages reviewing products for puppies, dogs, kittens and cats, all of which link to high affiliate-fee-generating pet insurance products. These reviews are not good, but they are treasured by Google's algorithm, which views them as a part of Forbes's legitimate news-publishing operation and lets them draft on Forbes's authority.

This side-hustle for Forbes comes at a cost for the rest of us, though. The reviewers who actually put in the hard work to figure out which pet products are worth your money (and which ones are bad, defective or dangerous) are crowded off the front page of Google and eventually disappear, leaving behind nothing but semi-automated SEO garbage from Forbes:

https://twitter.com/ichbinGisele/status/1642481590524583936

There's a name for this: "site reputation abuse." That's when a site perverts its current – or past – practice of publishing high-quality materials to trick Google into giving the site a high ranking. Think of how Deadspin's private equity grifter owners turned it into a site full of casino affiliate spam:

https://www.404media.co/who-owns-deadspin-now-lineup-publishing/

The same thing happened to the venerable Money magazine:

https://moneygroup.pr/

Money is one of the many sites whose air purifier reviews Google gives preference to, despite the fact that they do no testing. According to Google, Money is also a reliable source of information on reprogramming your garage-door opener, buying a paint-sprayer, etc:

https://money.com/best-paint-sprayer/

All of this is made ten million times worse by AI, which can spray out superficially plausible botshit in superhuman quantities, letting spammers produce thousands of variations on their shitty reviews, flooding the zone with bullshit in classic Steve Bannon style:

https://escapecollective.com/commerce-content-is-breaking-product-reviews/

As Gizmodo, Sports Illustrated and USA Today have learned the hard way, AI can't write factual news pieces. But it can pump out bullshit written for the express purpose of drafting on the good work human journalists have done and tricking Google – the search engine 90% of us rely on – into upranking bullshit at the expense of high-quality information.

A variety of AI service bureaux have popped up to provide AI botshit as a service to news brands. While Navarro doesn't say so, I'm willing to bet that for news bosses, outsourcing your botshit scams to a third party is considered an excellent way of avoiding your journalists' wrath. The biggest botshit-as-a-service company is ASR Group (which also uses the alias Advon Commerce).

Advon claims that its botshit is, in fact, written by humans. But Advon's employees' Linkedin profiles tell a different story, boasting of their mastery of AI tools in the industrial-scale production of botshit:

https://housefresh.com/wp-content/uploads/2024/05/Advon-AI-LinkedIn.jpg

Now, none of this is particularly sophisticated. It doesn't take much discernment to spot when a site is engaged in "site reputation abuse." Presumably, the 12,000 googlers the company fired last year could have been employed to check the top review keyword results manually every couple of days and permaban any site caught cheating this way.

Instead, Google is has announced a change in policy: starting May 5, the company will downrank any site caught engaged in site reputation abuse. However, the company takes a very narrow view of site reputation abuse, limiting punishments to sites that employ third parties to generate or uprank their botshit. Companies that produce their botshit in-house are seemingly not covered by this policy.

As Navarro writes, some sites – like Forbes – have prepared for May 5 by blocking their botshit sections from Google's crawler. This can't be their permanent strategy, though – either they'll have to kill the section or bring it in-house to comply with Google's rules. Bringing things in house isn't that hard: US News and World Report is advertising for an SEO editor who will publish 70-80 posts per month, doubtless each one a masterpiece of high-quality, carefully researched material of great value to Google's users:

https://twitter.com/dannyashton/status/1777408051357585425

As Navarro points out, Google is palpably reluctant to target the largest, best-funded spammers. Its March 2024 update kicked many garbage AI sites out of the index – but only small bottom-feeders, not large, once-respected publications that have been colonized by private equity spam-farmers.

All of this comes at a price, and it's only incidentally paid by legitimate sites like Housefresh. The real price is borne by all of us, who are funneled by the 90%-market-share search engine into "review" sites that push low quality, high-price products. Housefresh's top budget air purifier costs $79. That's hundreds of dollars cheaper than the "budget" pick at other sites, who largely perform no original research.

Google search has a problem. AI botshit is dominating Google's search results, and it's not just in product reviews. Searches for infrastructure code samples are dominated by botshit code generated by Pulumi AI, whose chatbot hallucinates nonexistence AWS features:

https://www.theregister.com/2024/05/01/pulumi_ai_pollution_of_search/

This is hugely consequential: when these "hallucinations" slip through into production code, they create huge vulnerabilities for widespread malicious exploitation:

https://www.theregister.com/2024/03/28/ai_bots_hallucinate_software_packages/

We've put all our eggs in Google's basket, and Google's dropped the basket – but it doesn't matter because they can spend $20b/year bribing Apple to make sure no one ever tries a rival search engine on Ios or Safari:

https://finance.yahoo.com/news/google-payments-apple-reached-20-220947331.html

Google's response – laying off core developers, outsourcing to low-waged territories with weak labor protections and spending billions on stock buybacks – presents a picture of a company that is too big to care:

https://pluralistic.net/2024/04/04/teach-me-how-to-shruggie/#kagi

Google promised us a quid-pro-quo: let them be the single, authoritative portal ("organize the world’s information and make it universally accessible and useful"), and they will earn that spot by being the best search there is:

https://www.ft.com/content/b9eb3180-2a6e-41eb-91fe-2ab5942d4150

But – like the spammers at the top of its search result pages – Google didn't earn its spot at the center of our digital lives.

It cheated.

If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:

https://pluralistic.net/2024/05/03/keyword-swarming/#site-reputation-abuse

Image: freezelight (modified) https://commons.wikimedia.org/wiki/File:Spam_wall_-_Flickr_-_freezelight.jpg

CC BY-SA 2.0 https://creativecommons.org/licenses/by-sa/2.0/deed.en

[mckitterick]

Problems with "AI" generated "art," Exhibit #47,200

image by Contingencies.org - not botshit, I'm pretty sure

one of the worst things about the onslaught of procedurally generated botshit these days (for consumers of art / writing / etc) is that - unless you follow the creators themselves and trust them to be actual creators and not purveyors of botshit - we lose the joy of just enjoying shared art

now, whenever we see / read / etc a piece of art, our first response is no longer "ooooh cool," it's to zoom in and inspect it for fakeness artifacts. kinda starts to feel like work

I guess the positive aspect of this is that, rather than following art-aggregator blogs, more people are now following real artists directly?

[mckitterick]

remember, folks, what the billionaire techbros are calling "AI" is nothing of the sort. one day we might very well enjoy (or suffer under) true AI, but this decade is not that time

the correct word for what these LLMs and art-theft algorithms produce is best called -

botshit

Vent art

[azspot]

Ironically, AI companies are setting themselves up for this problem. Google and Microsoft's full-court press for "AI powered search" imagines a future for the web in which search-engines stop returning links to web-pages, and instead summarize their content. The question is, why the fuck would anyone write the web if the only "person" who can find what they write is an AI's crawler, which ingests the writing for its own training, but has no interest in steering readers to see what you've written? If AI search ever becomes a thing, the open web will become an AI CAFO and search crawlers will increasingly end up imbibing the contents of its manure lagoon.

The Coprophagic AI crisis

[younes-ben-amara]

إذًا كيف تجعل الذكاء الاصطناعي يقرأ عنك الأخبار العالمية دون أن تكتئب وتضرب الناس بالسكاكين؟

ما هذه المجموعة من المختارات تسألني؟ إنّها عددٌ من أعداد نشرة “صيد الشابكة” اِعرف أكثر عن النشرة هنا: ما هي نشرة “صيد الشابكة” ما مصادرها، وما غرضها؛ وما معنى الشابكة أصلًا؟! 🎣🌐 🎣🌐 صيد الشابكة العدد #69 أهلًا ومرحبًا بكم؛ 🎣🌐 صيد الشابكة العدد #69🛡 كيفية استخدام نيوز مينيمالست (News Minimalist) وإضافة نيوز غارد (NewsGuard)🤔 الآن علامَ حصلت بقراءة كل هذا؟👶🏻 أحسن مهدٍ ذكي للأطفال😵 غريب أنه لا…

[mostlysignssomeportents]

The Coprophagic AI crisis

I'm on tour with my new, nationally bestselling novel The Bezzle! Catch me in TORONTO on Mar 22, then with LAURA POITRAS in NYC on Mar 24, then Anaheim, and more!

A key requirement for being a science fiction writer without losing your mind is the ability to distinguish between science fiction (futuristic thought experiments) and predictions. SF writers who lack this trait come to fancy themselves fortune-tellers who SEE! THE! FUTURE!

The thing is, sf writers cheat. We palm cards in order to set up pulp adventure stories that let us indulge our thought experiments. These palmed cards – say, faster-than-light drives or time-machines – are narrative devices, not scientifically grounded proposals.

Historically, the fact that some people – both writers and readers – couldn't tell the difference wasn't all that important, because people who fell prey to the sf-as-prophecy delusion didn't have the power to re-orient our society around their mistaken beliefs. But with the rise and rise of sf-obsessed tech billionaires who keep trying to invent the torment nexus, sf writers are starting to be more vocal about distinguishing between our made-up funny stories and predictions (AKA "cyberpunk is a warning, not a suggestion"):

https://www.antipope.org/charlie/blog-static/2023/11/dont-create-the-torment-nexus.html

In that spirit, I'd like to point to how one of sf's most frequently palmed cards has become a commonplace of the AI crowd. That sleight of hand is: "add enough compute and the computer will wake up." This is a shopworn cliche of sf, the idea that once a computer matches the human brain for "complexity" or "power" (or some other simple-seeming but profoundly nebulous metric), the computer will become conscious. Think of "Mike" in Heinlein's *The Moon Is a Harsh Mistress":

https://en.wikipedia.org/wiki/The_Moon_Is_a_Harsh_Mistress#Plot

For people inflating the current AI hype bubble, this idea that making the AI "more powerful" will correct its defects is key. Whenever an AI "hallucinates" in a way that seems to disqualify it from the high-value applications that justify the torrent of investment in the field, boosters say, "Sure, the AI isn't good enough…yet. But once we shovel an order of magnitude more training data into the hopper, we'll solve that, because (as everyone knows) making the computer 'more powerful' solves the AI problem":

https://locusmag.com/2023/12/commentary-cory-doctorow-what-kind-of-bubble-is-ai/

As the lawyers say, this "cites facts not in evidence." But let's stipulate that it's true for a moment. If all we need to make the AI better is more training data, is that something we can count on? Consider the problem of "botshit," Andre Spicer and co's very useful coinage describing "inaccurate or fabricated content" shat out at scale by AIs:

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4678265

"Botshit" was coined last December, but the internet is already drowning in it. Desperate people, confronted with an economy modeled on a high-speed game of musical chairs in which the opportunities for a decent livelihood grow ever scarcer, are being scammed into generating mountains of botshit in the hopes of securing the elusive "passive income":

https://pluralistic.net/2024/01/15/passive-income-brainworms/#four-hour-work-week

Botshit can be produced at a scale and velocity that beggars the imagination. Consider that Amazon has had to cap the number of self-published "books" an author can submit to a mere three books per day:

https://www.theguardian.com/books/2023/sep/20/amazon-restricts-authors-from-self-publishing-more-than-three-books-a-day-after-ai-concerns

As the web becomes an anaerobic lagoon for botshit, the quantum of human-generated "content" in any internet core sample is dwindling to homeopathic levels. Even sources considered to be nominally high-quality, from Cnet articles to legal briefs, are contaminated with botshit:

https://theconversation.com/ai-is-creating-fake-legal-cases-and-making-its-way-into-real-courtrooms-with-disastrous-results-225080

Ironically, AI companies are setting themselves up for this problem. Google and Microsoft's full-court press for "AI powered search" imagines a future for the web in which search-engines stop returning links to web-pages, and instead summarize their content. The question is, why the fuck would anyone write the web if the only "person" who can find what they write is an AI's crawler, which ingests the writing for its own training, but has no interest in steering readers to see what you've written? If AI search ever becomes a thing, the open web will become an AI CAFO and search crawlers will increasingly end up imbibing the contents of its manure lagoon.

This problem has been a long time coming. Just over a year ago, Jathan Sadowski coined the term "Habsburg AI" to describe a model trained on the output of another model:

https://twitter.com/jathansadowski/status/1625245803211272194

There's a certain intuitive case for this being a bad idea, akin to feeding cows a slurry made of the diseased brains of other cows:

https://www.cdc.gov/prions/bse/index.html

But "The Curse of Recursion: Training on Generated Data Makes Models Forget," a recent paper, goes beyond the ick factor of AI that is fed on botshit and delves into the mathematical consequences of AI coprophagia:

https://arxiv.org/abs/2305.17493

Co-author Ross Anderson summarizes the finding neatly: "using model-generated content in training causes irreversible defects":

https://www.lightbluetouchpaper.org/2023/06/06/will-gpt-models-choke-on-their-own-exhaust/

Which is all to say: even if you accept the mystical proposition that more training data "solves" the AI problems that constitute total unsuitability for high-value applications that justify the trillions in valuation analysts are touting, that training data is going to be ever-more elusive.

What's more, while the proposition that "more training data will linearly improve the quality of AI predictions" is a mere article of faith, "training an AI on the output of another AI makes it exponentially worse" is a matter of fact.

Name your price for 18 of my DRM-free ebooks and support the Electronic Frontier Foundation with the Humble Cory Doctorow Bundle.

If you'd like an essay-formatted version of this post to read or share, here's a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:

https://pluralistic.net/2024/03/14/14/inhuman-centipede#enshittibottification

Image: Plamenart (modified) https://commons.wikimedia.org/wiki/File:Double_Mobius_Strip.JPG

CC BY-SA 4.0 https://creativecommons.org/licenses/by-sa/4.0/deed.en

[mckitterick]

I teach writing workshops because I believe that anyone willing to learn the tools and techniques - and to put in the time and effort to master them - can become a skilled author whose work finds publication and makes readers happy

no publisher is willing to buy botshit, it's not copyrightable, and inputting prompts to receive amalgamations of other writers' work will not teach you how to write

if you hope to become a professional author, using generative algorithms won't do you any good, and in fact could ruin your chances of ever getting published

[mckitterick]

New clown award at the Webbys

this year, The Webby Awards got overrun by - er, "teamed up with" - Adobe to give a "Special Achievement in Creative AI" award for a set of "reimagined" MOMA artifacts

not surprising that a corporation that's trying to force AI into people's creative work would influence the Webbys to "honor" people using plagiarism machines to replace art

so here's a glimpse into how "AI" shillers are trying to get their botshit generators accepted by mainstream users:

if something wins an award, it's valid art, right? right?

[azspot]

The rise and rise of botshit cannot be separated from this phenomenon. The botshit in our search-results, our social media feeds, and our in-boxes isn’t making money for the enshittifiers who send it — rather, they are being hustled by someone who’s selling them the “picks and shovels” for the AI gold rush…

Cory Doctorow

[librarianrafia]

"These "hallucinations" are a stubbornly persistent feature of large language models, because these models only give the illusion of understanding; in reality, they are just sophisticated forms of autocomplete, drawing on huge databases to make shrewd (but reliably fallible) guesses about which word comes next:

Guessing the next word without understanding the meaning of the resulting sentence makes unsupervised LLMs unsuitable for high-stakes tasks. The whole AI bubble is based on convincing investors that one or more of the following is true:

I. There are low-stakes, high-value tasks that will recoup the massive costs of AI training and operation;

II. There are high-stakes, high-value tasks that can be made cheaper by adding an AI to a human operator;

III. Adding more training data to an AI will make it stop hallucinating, so that it can take over high-stakes, high-value tasks without a "human in the loop."

These are dubious propositions. There's a universe of low-stakes, low-value tasks – political disinformation, spam, fraud, academic cheating, nonconsensual porn, dialog for video-game NPCs – but none of them seem likely to generate enough revenue for AI companies to justify the billions spent on models, nor the trillions in valuation attributed to AI companies:

https://locusmag.com/2023/12/commentary-cory-doctorow-what-kind-of-bubble-is-ai/

The proposition that increasing training data will decrease hallucinations is hotly contested among AI practitioners. I confess that I don't know enough about AI to evaluate opposing sides' claims, but even if you stipulate that adding lots of human-generated training data will make the software a better guesser, there's a serious problem. All those low-value, low-stakes applications are flooding the internet with botshit. After all, the one thing AI is unarguably very good at is producing bullshit at scale. As the web becomes an anaerobic lagoon for botshit, the quantum of human-generated "content" in any internet core sample is dwindling to homeopathic levels:

This means that adding another order of magnitude more training data to AI won't just add massive computational expense – the data will be many orders of magnitude more expensive to acquire, even without factoring in the additional liability arising from new legal theories about scraping:

[grumpy-old-patriot]

Word of the Day: BOTSHIT (neologism) - nonsensical AI-generated content that’s trusted and used by humans