#the map is not the territory

[mostlysignssomeportents]

The seductive, science fictional power of spreadsheets

Tomorrow (Apr 30) at 2PM, I’ll be at the San Francisco Public Library with my new book, Red Team Blues, hosted by Annalee Newitz.

This week, John Scalzi was kind enough to let me write a guest-editorial for his Whatever blog about the themes in my new crime technothriller, Red Team Blues; specifically, about the ways that spreadsheets embody the power and the pitfalls of science fiction at its best and worst:

https://whatever.scalzi.com/2023/04/26/the-big-idea-cory-doctorow-2/

If you’d like an essay-formatted version of this post to read or share, here’s a link to it on pluralistic.net, my surveillance-free, ad-free, tracker-free blog:

https://pluralistic.net/2023/04/29/gedankenexperimentwahn/#high-on-your-own-supply

Yes, spreadsheets. Marty Hench (the protagonist of Red Team Blues) is a 67-year-old forensic accountant who specializes in unwinding Silicon Valley financial frauds, a field he basically invented 40 years ago, when, as a PC-struck MIT dropout, he moved from Cambridge to San Francisco to recover the stolen millions hidden in spreadsheets.

Working through this book — and its two sequels, which travel back in time to the 1980s and Marty’s first encounters with VisiCalc and Lotus 1–2–3 — I was struck by the similarities between spreadsheets and science fiction.

While many people use spreadsheets as an overgrown calculator, adding up long columns of numbers, the rise and rise of spreadsheets comes from their use in modeling. Using a spreadsheet, a complex process can be expressed as a series of mathematical operations: we put these inputs into the factory and we get these finished goods. Once the model is built, we can easily test out contrafactuals: what if I add a third shift? What if I bargain harder for discounts on a key component? If I give my workers a productivity-increasing raise, will the profits make up for the costs?

These are the questions that anyone managing a complex system asks themselves all the time. Historically, the answers have sprung from intuition, from fingerspitzengefühl — the “fingertip feeling” of how a system’s components work and what their potential and limitations are. But intuition can calcify, become a rigid set of rules that increasingly diverge from the best strategy.

By contrast, spreadsheets yield a set of crisp, instantly tallied answers to any question you put to them. Change the input and watch as that change ripples through the whole system in an eyeblink. If you’re adding three more people to your camping trip, will the amount of additional water require renting another vehicle? No need to guess: just check and see.

This has a lot in common with science fiction, a genre full of thought experiments that ask Heinlein’s famous three questions:

What if?

If only, and

If this goes on…

These contrafactuals are incredibly useful and important. As critical tools, science fiction’s parables about the future are the best chance we have for resisting the inevitabilism that insists that technology must be used in a certain way, or must exist at all. Science fiction doesn’t just interrogate what the gadget does, but who it does it for and who it does it to:

https://pluralistic.net/2023/03/20/love-the-machine/#hate-the-factory

One of science fiction’s key methods comes from sf grandmaster Theodore Sturgeon: “ask the next question.” Ask a question, then ask “what happens next?” Do it again, and again, and again:

https://christopher-mckitterick.com/Sturgeon-Campbell/Sturgeon-Q.htm

This technique produces excellent, critical ways of interrogating technological narratives — check out this delightful example of the possible pipeline from self-driving cars to ransomware gangs to mutual aid societies to the reinvention of the train:

https://dduane.tumblr.com/post/715940904747352064/you-can-make-your-mercedes-ev-go-faster-for-60-a

The commonalities between sf and spreadsheets don’t stop there — sf and spreadsheets share pitfalls, too. A spreadsheet is a model and a model is not the thing it models. The map is not the territory. Every time a messy, real-world process is converted to a crisp, mathematical operation, some important qualitative element is lost.

Modeling is an intrinsically lossy operation. That’s why “all models are wrong, but some models are useful.” There is no process so simple that it can be losslessly converted to a model. Even the actions of the nanoscale transistors in a microchip, which toggle between “0” and “1,” are rarely in a state of “no voltage” and “voltage.” That clean, square-wave line that’s used to describe what happens in a chip is a lie — that is to say, it is a model.

The wave isn’t square, it’s a squiggly line that hovers around zero and around one. Under normal circumstances, “zero” and “zero-ish” is a distinction without a difference. But when computers go wrong, it’s sometimes because a sufficiently ambiguous “zero-ish” acts like a “one.” That’s true all the way up the stack. On engineering diagrams, the nanoscale lines that electrons travel along inside a chip are represented as sharp paths, the kind of thing a Tron-cycle would lay down. But in the real world, we get all kinds of weird effects at that scale — electrons sometimes tunnel through those lines, performing a spooky quantum trick that reminds us that Newtononian physics are also just a model.

Every real-world phenomenon contains qualitative and quantitative elements, but computers can only do math on the quantitative parts. This creates a powerful temptation to incinerate the qualitative and perform operations on whatever dubious quantitative residue is left in the crucible, often with disastrous results.

Remember during lockdown, when a pair of University of Illinois at Urbana-Champaign physicists produced a model of covid spread that predicted that the campus could safely reopen, predicting no more than 500 cases over the entire semester and no more than 100 cases at any one time? The physicists were openly contemptuous of their epidemiologist peers, saying that this kind of model making lacked the “intellectual thrill” of real science.

UI was so swayed by the crisp, precise model that they invited students back to campus — only to shut down again in a matter of weeks, with 780 active cases on campus and more rolling in every day.

The model reduced qualitative factors — like the propensity of undergrads to get drunk, take off their masks, and lick each others’ eyeballs — to a quantitative probability, using the highly precise, scientific technique of taking a wild-ass guess. That guess was wrong. The campus reopening was a super-spreader event.

Any model runs the risk of hiding the irreducible complexity of qualitative factors behind a formula, turning uncertainty into certainty and humility into arrogance.

Think of how we replaced contact tracing with exposure notification. Contact tracing has a qualitative foundation: public health workers establish rapport with infected people, win their trust, and get them to fully enumerate the places they’ve been and the activities they participated in.

By contrast, exposure notification measures whether two Bluetooth radios were within range of each other for a predetermined interval. It substitutes signal strength for a person’s own understanding of their experience. Now, people can be wrong about their own experience — we lose track of time, we misremember emotionally charged events, and so on — but that doesn’t mean we can substitute Bluetooth measurements for personal experience.

That’s why, despite all the clever privacy-preserving math and interesting analysis, exposure notification was a bust, something between a distraction and a false-confidence-generating disaster. Contact tracing ended the 2014 ebola outbreak. Exposure notification just wasted a lot of time:

https://locusmag.com/2021/05/cory-doctorow-qualia/

It’s just too easy to forget which parts of a model are based on guesses and which parts are based on ground truth. And even if you can keep track of those differences, it’s even harder to re-check the model’s ground truth to determine whether the underlying factors have changed. That’s how we got into so much trouble with collateralized debt obligations, which were supposed to be “risk-free” mortgage derivatives that could be safely insured and invested in.

The formulas behind CDO hedging were designed by some of the world’s smartest mathematicians and physicists, who simply assumed that market actors — from loan-originating bank officers to insurance underwriters — would act in reliable, predictable ways. They were so very wrong that they brought the world economy to the brink of ruin:

https://www.wired.com/2009/02/wp-quant/

This is also science fiction’s failure-mode: any science fictional “ask-the-next-question” exercise represents a series of guesses or speculations or maybe possibilities — but when you combine that guesswork with the deceptive certainty that comes from inhabiting a cracking story, it’s easy to mistake “guessing” for “prediction.”

Prediction is hard, especially about the future. The assumptions that go into a prediction are always incomplete, not least because human beings have free will and agency and can change the circumstances that go into the assumptions. The very best science fiction embodies this principle. I’m thinking here of the likes of Ada Palmer, an historian and sf writer whose deep historical knowledge informs her sf and her pedagogy at the University of Chicago:

https://pluralistic.net/2022/02/10/monopoly-begets-monopoly/#terra-ignota

Palmer is famous — even notorious — for her annual four-week undergraduate LARP in which students re-enact the election of the Medicis’ Pope. It’s four weeks of alliances, betrayal and skullduggery by the students, each of whom is enacting the agenda of a real-world Cardinal or other power-broker.

The final investiture is done in full costume at the university’s massive faux-gothic cathedral, and going into that climax, of the four candidates, two are always the same, because the great forces of history are bearing down on that moment to ensure that the champions of the two dominant power-blocs are in the running. But the other two? They’re never the same — because the agency of the actors jockeying for power change the outcome, every single time, in absolutely unpredictable ways.

Like any other model, sf is wrong, but sometimes useful. Thinking about jetpacks and flying cars is “useful” insofar as it gets us to interrogate how we think about cities, about mobility, about privilege and geography. But it’s not a prediction. Worse, the endless tales in which flying cars are presented a fait accompli is a gift to grifters raising money for the objectively stupid idea of flying cars. After all, we all know flying cars are inevitable, so it’s basically a risk-free investment, right? With flying cars just around the corner, wouldn’t it be irresponsible to build a city with mass-transit instead of helipads?

There’s a whole range of thought-experiments that got transformed into predictions and then certainties: self-driving cars, “general artificial intelligence,” infinite life-extension, space colonization, faster-than-light travel, cryptocurrency, etc etc.

Spreadsheets don’t just lead their users astray — they also trick their creators. The very same people who transform wild-assed guesses about hairy, unknowable outcomes into neat mathematical relationships are perfectly capable of acting as if those relationships are based on fact, rather than supposition. The Great Financial Crisis wasn’t just about people who didn’t understand the uncertainty in the hedging algorithm going all-in — the people who made those models were also fooled by them.

It’s very easy to get high on your own supply. I’ll never forget the sf convention panel I was on with Robert Silverberg about sf’s supposed predictive value, where the subject of Robert A Heinlein came up, and Silverberg sniffed, and, in that trademark bone-dry way of his, said, “Ah yes, ‘Robert A Timeline.’”

Sf isn’t just full of writers who mistake their suppositions for predictions — the canon is full of tales in which brilliant people can and do predict the future, with near-perfection. Think of Hari Seldon, the hero of Asimov’s Foundation series, who is able to forecast the future several millennia out. Or Heinlein’s first-ever story, “Life-Line,” in which a genius inventor destroys the insurance industry by creating a computer that can predict your exact date of death using statistical methods.

There’s something wild about this phenomenon, in which writers make stuff up and then assume that anything that cool must also be accurate. One tantalizing explanation for this comes from EL Doctorow’s (no relation) essay “Genesis,” from his 2007 collection “The Creationists”:

https://www.penguinrandomhouse.com/books/41520/creationists-by-e-l-doctorow/

Doctorow tells the history of the Genesis story, which the Hebrews plagiarized from the Babylonians. In Doctorow’s telling, the Babylonian mystics who made up the Genesis story assumed that it had to be true, because they considered themselves to be nowhere near imaginative enough to have come up with something as great as Genesis. An idea that amazing had to be divinely inspired.

I like this because it’s a story of being led astray by humility, rather than hubris.

Imaginative exercises — whether or not they are assisted by mathematical models and self-updating digital spreadsheets — are powerful tools for thinking about the future we want, and to guide our attempts to make that future come true. All models are wrong but some models are useful, of course!

I’m on tour with Red Team Blues right now — I’m writing this post while waiting for my flight to San Francisco, where I’m appearing at the public library with Annalee Newitz tomorrow (4/30) at 2PM:

https://sfpl.org/events/2023/04/30/author-cory-doctorow-and-annalee-newitz-conversation-red-team-blues

One especially fun stop on this tour will be on May 5, at the Books, Inc in Mountain View, where I’ll be talking about the book with Mitch Kapor, the creator of Lotus 1–2–3, who knows a thing or two about spreadsheets:

https://www.booksinc.net/event/cory-doctorow-books-inc-mountain-view

The tour is bringing me to Berkeley, Vancouver, Calgary, DC, Gaithersburg, Toronto, PDX, Nottingham, Hay, London, Manchester, Edinburgh and Berlin — I hope to see you!

https://craphound.com/novels/redteamblues/2023/04/26/the-red-team-blues-tour-burbank-sf-pdx-berkeley-yvr-edmonton-gaithersburg-dc-toronto-hay-oxford-nottingham-manchester-london-edinburgh-london-berlin/

Catch me on tour with Red Team Blues in Mountain View, Berkeley, San Francisco, Portland, Vancouver, Calgary, Toronto, DC, Gaithersburg, Oxford, Hay, Manchester, Nottingham, London, and Berlin!

[Image ID: A Lotus 1-2-3 spreadsheet with green-on-black, low-res type; its center has an irregular vignette revealing a space station.]

[noster-tempus]

Something about the concept of time blindness has never quite made sense to me, so these were a gratifying listen.

[spaceintruderdetector]

https://archive.org/details/map-and-territory-eliezer-yudkowsky

The philosopher Alfred Korzybski once wrote: “A map is not the territory it represents, but, if correct, it has a similar structure to the territory, which accounts for its usefulness.” And what can be said of maps here, as Korzybski noted, can also be said of beliefs, and assertions, and words.

“The map is not the territory.” This deceptively simple claim is the organizing idea behind this book, and behind the four sequences of essays collected here: Predictably Wrong, which concerns the systematic ways our beliefs fail to map the real world; Fake Beliefs, on what makes a belief a “map” in the first place; Noticing Confusion, on how this world-mapping thing our brains do actually works; and Mysterious Answers, which collides these points together. The book then concludes with “The Simple Truth,” a stand-alone dialogue on the idea of truth itself.

[glitch-slap]

I’ve thought about changing my name because it doesn’t describe me. But then I think about people who drive into lakes because GPS told them to. When you give humans maps, they stop looking at the territory. Maybe if my map is completely off-base, people will actually pay attention to the scenery?

[small-names-big-ideas]

the idea:

bonini's paradox is most succinctly explained as "the map is not the territory", meaning the representation of the thing is never the same as the actual thing. this is related to rené magritte's the treachery of images, but Bonini's paradox's extension is that a model (or map)'s generalizability is inversely proportional to its level of precision. bonini's paradox is particularly relevant in scientific and mathematical models, where model designers must try to strike a balance between a model that describes their data accurately and one that describes other, similar, data accurately.

my original sources:

"on exactitude in science" by jorge luis borges, collected in collected fictions (translated by andrew hurley).

"the man who collected the first of september, 1973" by tor åge bringsvaerd, collected in the book of fantasy (edited by jorge luis borges, silvina ocampo, and a. bioy casares).

[hellobeautifuldigitalworld]

The map is not the territory.

The traffic jam is not the traffic.

The code is not the truth.

Art: the beautiful lie.

[lilithsaintcrow]

“In the world of digital mapping and cartography, snares to catch unwary plagiarists take the form of fake roads or places, known as “trap streets” or “paper towns”.”

[mapsontheweb]

Changing borders of Israel / Palestine, 1888-2023.

[revive-the-fandom]

Map of Berk

Credit to DanGref on DA for the larger map of Berk, I believe it was taken from one of the games - I usually say that those are Dubious Canon but since we have nothing else and it looks pretty close I'm including it on my map.

I made a better map of Berk, hopefully it's understandable? There's a couple of problems with it:

for one, Gobber's house in "Legend of the Boneknapper" is the same house as Snotlout & Spitelout's in "Race to Fireworm Island" ???? - so I left it as Gobber's since it's close to his Smithy and I'm putting the Shorts in a higher category of Canon than the shows I guess?

Astrid's house in "Team Astrid" is the same as the Armoury in "In Dragons We Trust" ???? - so I left it as the Armoury since it's seen in multiple episodes as "The Armoury" and Astrid's house is burned down.

Aaand I couldn't find the path to the Academy/Arena so I left it blank.

I would've liked to put things like The Cove, Bucket & Mulch's farm, the Path to Gothi's Hut, and the Beaches that they keep visiting onto the map, but there never seems to be a solid spot of any of them, so I'm presuming they're too far away to be on this map.

Blank Version:

[tales-of-snaktooth]

So, who are y'all exactly?

The main gals (and Ehsel)! A bit more information about them and a few other grumpuses that'll show up can be found on the blog's Characters page!

[noster-tempus]

This is just me trying to get my thoughts together, not trying to educate or convince anyone, because I'm WAY outside my lane here.

I'm struggling to reconcile everything Mel Baggs ever said about functioning labels with everything Level 2 and 3 autists are saying now.

One possible explanation is that psychiatry has gotten better at sorting people into levels based on how disabled they are rather than how well they fit stereotypes. I know there was a time when people's idea of "high-functioning autism" was Rain Man. The idea that the differences between "low" and "high" functioning autism are often entirely about optics was probably more true back then, and people who are considered Level 1 today are now parroting that idea which was never about them.

Admittedly, part of me is still a little uneasy with the way the current discourse never really acknowledges the possibility of someone being mislabeled with lower support needs than they actually have, and how that could be a very dangerous situation. I don't know, maybe that doesn't happen nearly as much as it used to. And anyone it's currently happening to is probably thinking "this is just what being Level 1 is like" because they're surrounded by Level 1's saying functioning labels aren't real. So that doesn't change the fact that people who fit neatly into Level 1 need to stop dominating the conversation.

@chavisory I'm curious as to whether you have any thoughts, or whether you could point me towards any current writers who are considered Level 2 or 3 by today's standards, who've been around for both stages of the discourse and might be able to illuminate the source of this disconnect?

[harriertail]

cartographer allegations etc etc // maps for @felisfern

[coyoteclan]

There are 330 following this blog now HI

Thought I'd share some territory concepts I've been working on! Both for Coyoteclan and a second clangen blog I've been debating, which is set in the wild west :)

I'll write a little blurb for each little area under the cut for those curious about these locations (and separate images)

Coyoteclan

Fallen-Tree Coyoteclan's Leader den and clan meeting place. The tree is ancient and hollow, the entrance coated in the scratches of long-gone cats. The Leader calls meetings from atop the tree, while the deputy sits on the rocks below.

Thunder River A small collection of waterfalls that eventually lead to the sea. From here, you can hear an ominous, melodic sound coming from the beach.

Graves of the Fallen The resting place for Coyoteclan cats. Coyoteclan prefer to commune with their dead here, but still go with the other healers during halfmoon.

Sweetwater Wilds

Trail of Blood Untamed land filled with bandits and danger. Not only are there plenty of predatory animals living here, but it's a popular hiding place for many outlaws.

Canyon of Stars A place for communing with the stars. Sheriffs and Healers visit often for guidance, though some bandits are known to come here in secret.

Hareshade's Dawn Not really a place but this is the Sheriff of Sweetwater Wilds lol I really like her but she's SO TRANSPHOBIC she is BULLYING the only trans cat in the clan and I cannot stop her

[holosaur]

Exploring Pleasant Valley and looking for the signal hill

[loriache]

Notes on the World Map: Melini

I'm kind of annoyed the complete World Bible doesn't have a version of the World Map with Melini filled in... unless it does and I missed it, but I literally bought it and looked at the relevant pages for this post! Anyway, I think this is the closest we have. I'm looking at this and -- Kahka Brud literally doesn't have a coast anymore. No wonder they're pissed!! Any industry involving the sea, trade, fishing, etc. is basically impossible now - you'd have to get to the coastline on the west side of the continent. That's probably miles!

Speaking of miles... how big is Melini?

This is where I got... really unwell. So first off, let's assume the official Dunmeshi World Map is a Mercator projection. I think the stage cartography in the Dunmeshi universe is at, it's going to be most important that maps are useful for marine navigation. So we can't just drop an approximate scale onto the grid and leave it at that; we have to work out where the equator is.

The way I've done that is via two places that seem like they have a close(ish) real world equivalent - in terms of climate and culture - Utaya and the Island of Wa (Toshiro's home). The Island of Wa is obviously Japan, and I've seen plausible arguments that Utaya is probably a climate/location similar to Nepal.

So, here's the (approximate) location and latitude of those places on a Mercator map of Earth:

If we arbitarily superimpose the two maps, asssuming they're the same size, we can draw a theoretical equator line that puts the two places at the same latitude, though on a different side of the equator. Putting them on the same side as Earth made the equator really low. I feel like that's ... unlikely, so we're going with this.

That puts Melini at a rough latitude of .. ~55? which would, using the same scale as the previous map, make two squares (at that latitude) about 600 miles. Very roughly calculating the size as given on the map based on that, that gives a width of (approximately) 250 miles and a length of 400, coming in at a bit less than 100,000 miles² due to the shape. Probably it's between the sizes of the UK and Aotearoa.

That really helps me visualise what Laios is working with here when I think about post-canon. Though I reckon what I've done here is "Just enough research to really annoy an expert". Sorry if you're that expert. I really don't know much about cartography <3 It's like that xkcd comic:

On distances, as someone who lives in the UK; with a car, you can get almost anywhere in a day if you're prepared to drive until late. But they don't have cars, so getting from one end to the other would take multiple days of travel. It'd take about 7 days to walk the approx "length", 400~ miles (from the top of Scotland to Liverpool) (according to Google Maps) and 2 days cycling, so probably 3-5 days with horses, depending on how you pushed them. That'd vary a bit depending on terrain, of course.

[zumurruds]

The number of times that they had made love were still finite enough that Damen could remember each one of them: at Ravenel, the unspoken full of painful secrets; in Karthas, losing themselves in each other; aching sweetness by firelight at a roadside inn at Mellos; the desperation of their first lovemaking after Damen's recovery. None of them had been like this, half sprawled on the bed looking up at Laurent.

LOVEMAKING LOCATIONS