vaniver

[vaniver]

So I am in this category (at LessOnline, believe not many years left) but not in the literal category of "talked with AW about roguelites".

For me a big part of it is--I came out to Berkeley in 2016 and worked for many years on trying to avert the end of the world, and eventually came to believe that if I'm going to do something helpful, it's thru narrow channels or weird things and not via thrashing or asceticism or whatever. Like, I kind of feel like I earned playing video games now by trying much harder when things looked much more hopeful, and I 'earned' not being super optimistic about future efforts by trying and failing.

(Like I am trying to instead get the wisdom of 'which things to try next' but, like, I definitely am making both updates.)

The strangest thing about being at LessOnline is that there are a lot of people here who think that we're only two or so years away from the world ending. As someone who does not believe that, it always remains a bit stunning to be in a conversation with someone about their favorite roguelites and then they're like "yeah, hope we get some good stuff before The End".

And I don't think it's absolutely ridiculous. I've looked at the analyses, I've run the numbers, I think there are things you could believe, which are reasonable to believe, that could lead you down that path. I don't think that LLMs are everything they're promised to be, but we're clearly in a time of change, with an uncertain future. There's still plenty of low-hanging fruit, and some fundamental problems with current approaches that I expect to be solved.

But what I do find puzzling is that they're just at this event with me, living a similar life to the life that I'm living.

[vaniver]

Husband points out that cane sugar and sugar beets are a very high percentage of sucrose/glucose, and the original quote is about the _average_ plant but we don't care very much about that. I think an acre of sugar beets gets you about 20 tons at 15% sugar -> about 6,000 pounds of sugar?

JBS Haldane - Daedelus (1924):

Everyone knows that food is ultimately produced by plants, though we may get it at second or third hand if we eat animals or their products. But the average plant turns most of its sugar not into starch which is digestible, but into cellulose which is not, but forms its woody skeleton. The hoofed animals have dealt with this problem in their own way, by turning their bellies into vast hives of bacteria that attack cellulose, and on whose by-products they live. We have got to do the same, but outside our bodies. It may be done on chemical lines. Irvine has obtained a 95% yield of sugar from cellulose, but at a prohibitive cost. Or we may use micro-organisms, but in any case within the next century sugar and starch will be about as cheap as sawdust.

I still don't really get why this hasn't happened. It wouldn't exactly be sugar as we're used to it, it would be glucose rather than sucrose, but that's still sweet.

[vaniver]

(At time of writing, sugar is about $620 a ton and sawdust is about $50 a ton.) Also how hard is it to make a plant that's just, like, constantly leaking syrup?

JBS Haldane - Daedelus (1924):

Everyone knows that food is ultimately produced by plants, though we may get it at second or third hand if we eat animals or their products. But the average plant turns most of its sugar not into starch which is digestible, but into cellulose which is not, but forms its woody skeleton. The hoofed animals have dealt with this problem in their own way, by turning their bellies into vast hives of bacteria that attack cellulose, and on whose by-products they live. We have got to do the same, but outside our bodies. It may be done on chemical lines. Irvine has obtained a 95% yield of sugar from cellulose, but at a prohibitive cost. Or we may use micro-organisms, but in any case within the next century sugar and starch will be about as cheap as sawdust.

I still don't really get why this hasn't happened. It wouldn't exactly be sugar as we're used to it, it would be glucose rather than sucrose, but that's still sweet.

[vaniver]

bringing napoleon to the modern world and showing him all the powerful states and liberal republics like that doctor who van gogh scene

[vaniver]

two chapters in to greer's "Inside a Magical Lodge" (thnx @fruityyamenrunner for the rec). A lot of it so far is just about the practicalities of administering any small group where everyone has day jobs and meets infrequently. Sometimes I was reminded of a Dungeons and Dragons group, sometimes I thought LessWrong meetups could use some of these rules and procedures, to deal with various kinds of annoying person and situation. Although LessWrong is like, pseudoacademic in some ways, and a better source of meetup structure might be academic conferences or lab meetings. Aside from that though a lot of it seems strange and foreign and I'm struggling to understand it as anything but words on a page. Trying to take some vocabulary and relate it to stuff I'm familiar with.

The recognition sign. This is I like your shoelaces, thanks I stole them from the president. Doesn't seem to make any sense to me in modern times. Yes, I meet certain kinds of "insiders" when I travel (tumblr usrs, lesswrongers, academics), but this is because I talk to them online and arrange these meetings in advance.

Those interested in magical work vs posers interested in make believe and dressup. This was confusing to me, because I thought all magical rituals were make believe and dress up. But I realize it's analogous to something I understand, which is discussion. Like, there's certain people in a conversation just for the excitement of being in a conversation like that, getting to use some of their technical knowledge. I think some of these people are trying to enter a community and should be welcomed, and I have not always been kind enough to them. But there is a reason for that, which is I'm trying to get something out of the conversation, something intangible and hard to explain. You could call it "insight". This is true with LessWrong stuff and academic stuff. In both cases there is potential for some "tangible" result if some insight from a conversation leads to a paper or a post, but mostly that is not the case and there's still a difference between a "real" and "fake" conversation. Steven Pinker's description of Jeffrey Epstein is a good example: "He likes schmoozing with smart and intellectual people, but he couldn’t really or had very little interest in exploring an issue. He’d wisecrack, change subjects, or get bored after a few seconds. He’s a kibitzer more than a serious intellectual." SO while LessWrong meetup groups are centered around discussion rather than ritual, the same problem the magicians have, of an intangible goal that not everyone there is actually interested in, emerges.

The mundane vs the magical. Ayn Rand wants to make industrial society seem exciting, Greer takes for granted that it isn't and people need some escape. So there's "two worlds", the mundane and the magical, and the lodge is supposed to have the feeling of being between worlds. Greer explains a lot about lodges according to this requirement. This is why the meetings are indoors (LessWrong meetups, on the other hand, are often outdoors, like in public parks). This is why they have opening and closing rituals. Interestingly, the secrecy contributes to this. I've wondered about the consequences of MIRI's imitation of the secrecy of nuclear research, but I've never heard any emotional effect like this mentioned.

Lineage, Grand Lodges, charters, secret chiefs. This got me thinking of a silly thought experiment: what if Ziz started a LessWrong meetup group? Presumably she'd be condemned, there'd be some warning from more "central" rationalists not to attend. So there's nothing official, but I'm pretty sure she's implicitly excommunicated. I, on the other hand, could start one, as long as I don't conflict with one already existing in the city. But there's nothing official like a charter from a grand lodge. Or is there? If I posted the announcement on LessWrong people could see my karma and past posts; this is effectively central recordkeeping, just efficiently automated. DEoes that make Lightcone the Grand Lodge? But the New York LessWRong meetup group is so old it's an "Overcoming Bias" meetup group, and I don't think their legitimacy derives from the california people. So it's all informal but I feel like I see the structure of social relations that the lodges formalized.

[vaniver]

Thank you Vasili Arkhipov, who 60 years ago today averted nuclear war by disagreeing with 2 other people and stopped the launch of nuclear weapons from their submarine during the cuban missile crisis.

May we all have such clear decision making through uncertainty and the strength to hold up to peer pressure.

[vaniver]

Hmm I don't think I heard “it” first from Yudkowsky; I think I heard it first from a lot of the technical original sources. I read Epistemology and the Psychology of Human Judgment back in 2009 because it was linked on the xkcd forums, I took decision analysis classes in graduate school around 2011, my master's degree  in 2012 was basically statistics plus industrial rationality, and around this point I found LW thru the HPMOR author notes (and HPMOR of course was linked on the xkcd forums).

I did have much more in common with the rationalists than the OR folks; I never really tried to get any of my classmates/labmates into HPMOR or LW, mostly because I didn’t expect it to succeed. (I did once happen to introduce a former labmate to Michael Vassar at a SENS life extension conference, now that I think about it, but that didn’t go anywhere.) I wasn’t aware of any of the OR folks that were interested in the breadth of technical topics that I was or the psychology angle (I do think I got introduced to Kahneman, Gendlin, and Korzybski specifically from LW; like definitely being around LW deepened my thinking and polished my cognitive style for a lot of this stuff).

I feel like I can't explain the significance Yudkowsky had to me to young people because you grew up with bookstores stocking "Thinking, Fast and Slow" and "Superforecasting". When I was your age there was the Sequences and that was it. Aside from reading the more technical original sources but how would you even know to do that if nobody told you about it in a readable way? This whole "judgments are made in some way and we can treat the judgment-making process from a scientific or engineering perspective and measure how well it does" perspective had a wave of popularization, which I think Nate Silver was also a part of, but this was primarily in the 2010s. Any of us who got in on it early, which is me, @youzicha, @nostalgebraist, @vaniver, etc, you know us old folk, we heard about it first from Yudkowsky. Not necessarily all Yudkowsky fans but I think even @nostalgebraist read stuff in Yudkowsky, read Tetlock later, and was like, huh, it's a reasonable version of some of the stuff Yudkowsky fans are on about, so he heard it from Yudkowsky first (even though the reason it was already familiar was that a bunch of Sequences posts are summaries of Tetlock). It changed my life by motivating me to study statistics, which everyone thinks is dry, but if you think of it as the scientific study of processes by which guesses are made, well I think it's very exciting.

[vaniver]

Thales Was Right

It's interesting to me how much people struggle to intuit differences of scale. Like, years of geology training thinking about very large subjects, and I'm only barely managing it around the edges.

The classic one is, of course, the mantle- everybody has this image of the mantle as a sort of molten magma lake that the Earth's crust is floating on. Which is a pedagogically useful thing! Because the intuitions about how liquids work- forming internal currents, hot sections rising, cool sections sinking, all that- are all dynamics native to the Earth's mantle. We mostly talk about the mantle in the context of those currents, and how they drive things like continental drift, and so we tend to have this metaphor in mind of the mantle as a big magma lake.

The catch, of course, is that the mantle is a solid, not magma. It's just that at very large scales, the distinction between solids and liquids is... squirrely.

When cornered on this, a geologist will tell you that the mantle is 'ductile'. But that's a lie of omission. Because it's not that the mantle is a metal like gold or iron, what we usually think of when we talk about ductility. You couldn't hammer mantle-matter in to horseshoes or nails on an anvil. It's just a rock, really. Peridotite. Chemically it's got a lot of metal atoms in it, which helps, but if you whack a chunk of it with a hammer you can expect about the same thing to happen as if you whacked a chunk of concrete. Really, it's just that any and every rock is made of tons and tons of microcrystal structures all bound together, and the boundaries between these microcrystals can shift under enormous pressure on very slow timescales; when the scope of your question gets big enough, those bonds become weak in a relative sense, and a rock starts to become more like a pile of gravel where the pebbles can shift and flow around one another.

The blunt fact is, on very large scales of space and of time, almost everything other than perfect crystals start to act kind of like a liquid- and a lot of those do as well. When I made a study of very old Martian craters, I got used to 'eyeballing' the age based on how much the crater had subsided, almost exactly like the ways that ripples in the surface of water gradually subside over time when you throw a rock in to a lake. Just, you know. Slower.

But at the same time, these things are more fragile than you'd believe, and can shatter like glass. The surface of the Earth is like this, too. Absent the kind of overpressures that make the mantle flow like it does, Earth's crust is still tremendously weak relative to many of the planet-scale forces to which it is subject- I was surprised, once, when a professor offhandedly described the crust as having a tensile strength of 'basically zero;' they really thought of the surface as a delicate filigreed bubble of glass that formed like a thin shell, almost too thin to mention, on the outside of a water droplet. On human scales, liquid is the thing that flows, and solid is the thing that breaks. But once stuff gets big or slow or both, the distinction between a solid and a liquid is more that a liquid is the thing that doesn't shatter when it flows. And it all gets really, really vague, which I suppose you'd expect when you get this far outside the contexts in which our languages were crafted.

[vaniver]

The PS5 scalper isn't actually providing any value to society.

They’re moving a PS5 from someone who wants it less to someone who wants it more, which is providing value.

In response to brazenautomaton:

I agree that the scalper is taking gains-from-trade that the creator is leaving on the ground (and the creator should do it instead). 

It’s less obvious that the landlord is innocent in a world where lots of housing construction trades are blocked (or taxed) by regulations, as the landlord who owns housing supply benefits from the restriction of housing supply and might thus advocate for that restriction. (It’s complicated because every landlord wants to be able to build themselves while their competitors don’t build, and not everyone votes in favor of their class interest, and all that; it’s easy to have a city where the landlords are pro-building and the non-landlord owners are anti-building and the renters focus on how the landlords are bleeding them, and not on the underlying causes of the situation.) 

Scalpers: There is a low supply and high demand for PS5s right now. I will buy up a bunch of them at market price just to resell at my new inflated price for personal profit

Average Person: Dude, you’re scum. This should be illegal

Scalpers: There is a low supply and high need for affordable housing right now. I will buy up a bunch of houses at their affordable price just to charge people my new inflated monthly price to live in them, without actually owning them. For personal profit.

Average Person: You may not like it, but this is a valid business practice and a necessary part of adult life. This is our free market at work, and-

[vaniver]

I think it’s mostly because the revealed capabilities of AI systems were impressive enough that it no longer seemed like a necessary part of the argument.

In ~2000, it didn’t look like you would be able to build Skynet (or even a Terminator) without major advances in software engineering, and so positing a computer system that makes advances in software engineering and then builds Skynet looked possible. In ~2016, it looked like you might be able to build a relatively small number of modules that would fit together to make Skynet, at which point you no longer need to posit recursive self-improvement; you could imagine Cyberdyne making it.

why did EY stop talking about self improving AI? It used to be his whole thing. Not just in his pre-"friendly AI" days when he was getting funding to write a "seed AI", but in his AI safety days when it seemed like he was basically trying to extend formal verification of programs to self-modifying programs. But I haven't heard him mention it in years? Like he didn't talk about it in the "list of lethalities" did he? Although I read that a long time ago, now. You'd think he'd be saying something there like "if it's a little superhuman it will get very superhuman" as an item, I mean he argued for that extensively in "Intelligence Explosion Microeconomics". What happened to this whole side of the EY thesis?

I suspect it was a "don't want to give the deep learning people any ideas" thing. The whole "infohazard" side of the MIRI people is so silly and toxic. But idk whether this was an instance. And maybe I'm just overlooking recent mentions.

[vaniver]

Ok that was my joke answer; my serious answer is that his primary focuses are politics and sports. I think lots of rationalists view his models of politics as a sensible baseline but don’t think about politics much, and think about sports even less.

I really feel like Nate Silver should be more of a rationalist icon than he is.

He's an outspoken Bayesian who believes in probabilistic reasoning and does explicit calibration tests of himself. He's incredibly good at this, producing consistently well-calibrated forecasts across multiple domains. (And he's winning!)

He even exemplifies the actual primary rationalist virtue, which is pissing people off by being contrarian on the internet.

I suspect he loses points for forecasting with actual math, rather than putting a number on his intuition and muttering something about priors. Doing math in your Bayesian updates is cheating!

[vaniver]

Tho to be fair this is like Eugene Goostman passing ‘the Turing test’.

[vaniver]

Dwarf Fortress is now available on Steam! It’s got a graphical tileset and a mouse UI, so it’s somewhat more accessible than it used to be.

Dwarf Fortress

I’m playing it again after many years, and noticing some of the new features.

But this bit of a dwarf’s personality spoke to me:

He is moved by art and natural beauty, but he is troubled by this since he dislikes the natural world.

[vaniver]

Reblog if you think that the Lizardmen are a cool Warhammer Fantasy faction!

[vaniver]

I was looking forward to a Disco Elysium sequel if ever they made one, but... how are we sure this isn’t just “turns out Marxist-Leninists are bad at not splintering, especially if they get a bunch of money”?

The ZA/UM debacle is so so heart wrenching man it's tragic, it's poetic, it's literally more of a thematic sequel to Disco Elysium than anything the remaining shell of the company will ever be able to churn out.

So so so fucking bleak.

[vaniver]

So it’s true that stealing trade secrets is illegal, but the thing that’s protected is the method of acquisition rather than the idea. If I patent “pasta sauce with both garam masala and five spice in it”, and you come up with the same brilliant idea a year later, I can sue you for infringement, even without proving (or there being) a causal connection between my marketing the sauce and you making the sauce.

But if the ingredients of my sauce are just a trade secret, there needs to be a person that leaked it (or you needed to have reverse engineered my sauce, which is itself a crime) for me to sue.

A neat thing about trade secrets is that they better assess “how hard was this to come up with?” since they allow for independent rediscovery. [If you can keep your recipe secret for a century, then you can still have your trade secret a century later!]

trade secrets make sense to me as legally protected property, patents theoretically but maybe not the real life patent system. Copyright im not sure; it has the same kind of appeal as trade secrets in that it’s an original creation but it’s not “naturally protectable” like a trade secret

[vaniver]

(buddhist youth pastor voice) i see you're refreshing tumblr. do you know what else is a painful and unending cycle of content