#building walls to keep the waves back (an ode to the archivists)

[zoanzon]

If you've seen the post circulating around that HR9495 - the bill to strip nonprofits of their tax-exemption - is a credible threat to AO3 and to download your fic, you may or may not be going 'oh fuck'. This might be because 9495 is a fucking abominable document...or it might be because you're like me and you have far too many fics to download one at a time.

Well, this is a bit quicker-and-dirtier than the post I'd planned on making next month, but as it's needed now:

Bulk Downloading from Ao3

Nianeyna has a program up on Github that can download Ao3 fics in bulk. You give it a starting URL (such as the start of your Bookmarks, or someone else's Works page) and tell it to either stop at a certain page or to keep going until it downloads the whole range. If you don't have code experience, that's fine, the linked page has a basic walkthrough on how to download and initialize the program.

Now, if you're also worried about fiction on other sites too...

Downloading from Fanfiction.net, SpaceBattles, SufficientVelocity, QuestionableQuesting, & More [WITHOUT Embedded Pictures]

Have you heard of FicHub? You have now.

FicHub only goes on fic at a time, and if you try and feed it too many URLs too quick it'll time out and you have to wait a bit to let it 'cool down', but it was the first program I encountered that could not only download fics from FF.Net, but also the forums where the Worm fandom is found.

Plus, that's not all it download from. To quote FicHub's site:

Supported Sites SpaceBattles, SufficientVelocity, QuestionableQuesting (XenForo) FanFiction.net, FictionPress Archive Of Our Own Harry Potter Fanfic Archive Sink Into Your Eyes AdultFanfiction.org Worm, Ward Partial support (or not tested recently): XenForo based sites (Bulbagarden Forums, The Fanfiction Forum, Fanfic Paradise) Fiction Alley The Sugar Quill (largely untested) FanficAuthors (minimal) Harry Potter Fanfiction (archive from pre-revival)

One thing I really like about FicHub is that it doesn't default to EPUBs. Like how Ao3's built-in downloader gives you file options, FicHub lets you choose between EPUB, a zipped HTML file, MOBI, PDF, and a 'sharable link to this export' which I've never used but sounds nice.

The downside of FicHub is that if a fic has embedded media - pictures, for example - you don't want to be using FicHub as it won't embed the picture. All it'll do is embed a link to were the picture was supposed to be drawn from. However, this isn't a concern for most-all FF fics as well as a good chunk of stuff from other sites. Just be warned that if you want media embedded, you want the next tool.

(You'll also want the next program if downloading from a forum and the OP who's fic you're downloading has a gigantic signature box: for some reason, FicHub adds the signature page to the bottom of each chapter in the document you download. Fine enough if it's a line or two, annoying if it takes up a page or more.)

Downloading from AlternateHistory.com, SpaceBattles, SufficientVelocity, QuestionableQuesting, & More [WITH Embedded Pictures]

This time we're looking at a Firefox addon: WebtoEpub.

According to its page, its download options include the following:

Baka-Tsuki.org (obviously) ArchiveOfOurOwn.org Blogspot (some) FanFiction.net. gravitytales.com hellping.org krytykal.org moonbunnycafe.com mugglenet.com nanodesu (some of the *thetranslation.wordpress.com sites) readlightnovel.com royalroad.com shikkakutranslations.org http://sonako.wikia.com And many other web sites.

It only downloads EPUBs, but if you're trying to grab a fic that has facecasts for characters, or maps, or anything like that...well, this actually grabs the pictures.

However, brief tip: I don't know how it is for any of the pages it actually lists, but if you use this for a fic from AH/SB/SV/QQ - the forums listed in the header - you want to open the page and then click the button that opens the whole index of threadmarks, and then call the function. If you don't do it that way, it won't know the full range of posts.

So, yeah...

While I hope HR9495 dies and isn't passed, it's better to prepare than to not. Even if it doesn't pass, I lived through the NC-17 purges on Fanfiction.net and saw thousands of fics vanish into the abyss; and I've seen authors delete fics or their whole collections off of Ao3 too. All that's taught me that I can't rely on just going to a bookmarked fic and expecting it to be there; if I want to be able to find and reread anything into perpetuity, I need to have a copy downloaded.

Hopefully this helps some of you download up the fics you want to have on hand, too.

[zoanzon]

This doesn't stop all the rest of the concerns of us speedrunning our way into fascism, but many groups were aware the website/article purge was coming and were working to get stuff backed up the moment they saw the election results.

The End of Life Web Archive already backed up all FedGov websites; Harvard's Library Innovation Lab had at least 116,000 datasets (16ish terabytes) as of today; and they aren't the only ones. There's a compilation post regarding all the groups trying to keep back up Federal data: this post over in r/DataHoarder lists 3 versions of the same thing (a live GDoc version; a PDF version; and a Bluesky upload).

Again, I'm not saying this to go 'there is nothing to worry about'. That's out the goddamn window. However, in comparsion to 'oh god we're speedrunning Weimar' or 'speedrunning Putin's rise', we're able to coordinate and organize in ways that people couldn't in the 1930s or even the 1990s.

Btw, this is how conservatives keep getting to claim that trans people are a new thing no one has ever heard, because our history and existences have continually been erased or obscured systematically through out history.

The most famous example was 92 years when the Nazis raided the library of the Institut für Sexualwissenschaft, the medical practice where the term transsexual was first coined and the first gender affirming surgery was performed in in 1931.

What did the Nazis do after raiding the library on May 6th, 1933? You may be familiar with these images

It is happening again.

[zoanzon]

Yeah, so, for the last most-of-a-year I've had a particular project I've been working on to back up all my electronic writing - from the half-dozen places it's been scattered, content going back to early 2016 - into a single fucking repository.

I'm not done yet - still got over 500 sticky notes of stuff to import from my phone - but as of today...

I broke the million-word mark.

It wasn't a New Year's resolution last year to do anything like this, but yeah if you want to set yourself a resolution for this year to do something like, oh I don't know, 'back up a copy of everything 'I' (ie you) have written so it doesn't get lost to the void one day'...

Consider this you sign it's a goal you can pull off.

[zoanzon]

Extremely fucking terrified at how I've apparently managed to back up a bit over a million and a half words of personal writing in a year? Like, I went through my diary and confirmed I first started deep-dive archiving my old stuff (and cross-archiving anything written after that point) last March.

And I still have more to go???

[zoanzon]

Just was shown the latest Defunctland - about the question of who wrote the Disney Channel tune? - and by god this video came into my house and shot me dead.

[zoanzon]

Physical media can last for longer if maintained right, but there's generally fewer copies. Electronic media is stored on digital drives that degrade sooner than other mediums, but can be copy-pasted to new drives in a way that other mediums can't be as easily copied.

I get (and mostly agree with) the posts on here talking about the importance of physical media vs digital but sometimes the tone of like 'if it's physical media you'll always have it unlike digital media' you get in these posts is deeply silly like this is in fact something that's notably famously not true of physical media actually. you can quite literally lose everything by only having physical media. both is good etc etc.

[zoanzon]

Maybe you've got someone from Yemen or Dijibouti that can tell you exactly what the fuck happened there instead of my layman's piss-poor guesses from the deck of a cruise ship. I was sloshed, trying to have a vacation and forget my mess of a life. But just in case I'm one of the only ones left that knows, I thought I'd write you and make sure you have at least this fragment. I figure, what the hell, at least it will level me up in my History-Keeping stats.

“Sleep Over: An Oral History of the Apocalypse” by H.G. Bells, page 187

[zoanzon]

I had a longass reply written out and then my phone fucking ate it, so let's try this again...

For anyone concerned about all the other federal sites, go look at what r/datahoarding is up to! Members have been backing up the entirety of data.cdc.gov or the HOAA + NIM + EPA + Deprtment of Education websites.

Outside of Reddit, there's groups like the End of Life Web Archive, the Harvard Law Library Innovation Lab, the Environmental Data and Governance Initiative, and other groups. They've been backing stuff up since last year to get ahead of the writing on the wall.

For people wanting to try and contribute their own efforts, what OP posted right above is excellent for that! ArchiveTeam Warrior helps Internet Archive get stuff backed up using your own devices to help split up the work. The setup has a lot of steps, but this post gives a good breakdown as to how to go about it. It'll take up some network bandwidth and storage space on your device, but it looks to be pretty automated once it's all set up.

But yeah, as it was decades ago we say it again:

I haven’t slept in three days.

I’ve been working around the clock to back up over a petabyte of research data and records and articles from the National Library of Medicine before it gets attacked by that elongated dipstick and his gestapo squad. Trying to get copies of every federal research program’s data before it gets destroyed in the crossfire. I hope to God that it won’t, but I have no confidence that it will be safe. Lots of copies keep stuff safe.

[zoanzon]

If you can determine what the URL of the fic was - say, if you have a bookmark to where it used to be - try and plug the URL into FicHub: it’s a ‘backs up fanfiction so you can download it later’ site and works on FF.net, AO3, and more. If the fic is old enough, it very well might be stored somewhere in FicHub.

there's a difference between love and obsession by flower in the snow where to find this fic?It was deleted some time ago from ao3

Anyone have this fic downloaded?

- Mod C

[zoanzon]

Looked around a bit and think I found a copy on openarchive and over on archive.org: dunno if its a whole copy or how hard it is to access off the site, but its something?

@grammarpedant says

oh hell yeah. please tell us about the kensho series so we can live vicariously through you; also have you considered participating in archival efforts for it?

The Kensho series, also called the Way-Farer series by Dennis Schmidt is about an abandoned colony of humans on an alien world, having to live with a population of higher dimensional aliens that feed off of strong emotions. They feed by escalating strong emotions such as anger into a rabid frenzy, generally resulting in lots of death. The prologue is about how the entire colony nearly died the second they touched down, and the books are about the different methods used through the generations to survive. And eventually, thrive.

It's based very heavily around Buddism as a concept, and I image could be a rough read if you go in expecting hard sci fi. I first read them very young, so I didn't exactly carry a bunch of expectations.

The author has written a little other sci fi that's really not as good, and besides that is a total unknown. We have a birth date, death date, and a list of his works. That's it. I'm fascinated

[zoanzon]

For once, it looks like the darklordpotter forum is coming in clutch: found this, looks like someone uploaded a .zip copy of the fic - or the series? - to the forum

Lunch break is ending rn so can't download the fic to check it, but it's got the right name at least, so hope this is it

I wonder where the fic Amazing Agent Loki is. I wanna reread it. I should have downloaded that sucker when I had the chance