#ao3downloader does

[muffinlance]

In light of the recent FanFiction.net scare, for anyone looking to do bulk downloads of their favorites/bookmarks/fics in general, two excellent resources:

ao3downloader: works with Python. Does what it says on the tin.

FanFicFare: works with Calibre. Can download from many sites, including AO3 and FF. My bad, FF is NOT supported. Lots of other sites are, though. Anyone got one that DOES work on FF?

Edit: I am again emphasizing the BULK in bulk downloads. AO3 obviously has an excellent single-fic download option. This is for if you want to automate that. For all your fics. ...Bulk download.

[nianeyna]

*steps up to podium, taps mic*

i think. authors on ao3 should not re-use the same title for different fics. this is because. calibre does not understand that they are different fics. and it fucks up my ao3downloader process. thanks for coming to my ted talk.

[nianeyna]

girl help I figured out that I can dump all my unrealized ao3downloader improvement ideas into github issues and now I can't stop adding shit to it

[nianeyna]

Well folks I've been sitting on this little script for ages and finally decided to just go ahead and publish it. What does it do?

you can enter any ao3 link - for example, to your bookmarks or an author's works page - and automatically download all the works and series that are linked from that page in the format of your choice. if there are multiple pages, the script will page through results for you until there are no more fics to download

you can check your fanfic-savin' folder for unfinished fics and automatically update them if there are new chapters. you can also check for new fics in series!

if you're a dinosaur who uses Pinboard, you can back up all the Pinboard bookmarks you have that link to ao3

don't worry about crashing ao3 with this! this baby takes forever to run, guaranteed. anyway ao3 won't let me make more than one request per second even if I wanted to so it's quite safe

I've been working on this for about two years and it's finally in a state where it does everything I want and isn't breaking every two seconds, so I thought it was time to share! I hope y'all get some use out of it.

note: this is a standalone desktop app that DOES NOT DO ANYTHING aside from automate clicking on buttons on the ao3 website. Everything this script does, can be done by hand using ao3's regular features. It is just a utility to facilitate personal backups for offline reading - there's no website or server, I have no access to or indeed interest in the fics other people download using this. No plagiarism is happening here, please don't come after me.

[nianeyna]

Hi :) I recently saw your awesome ao3downloader on mojos blog - first, and most importantly, I can’t believe how much love and effort is poured into fandom but the work you do will be so fricken appreciated.

Second, I was wondering if there were instructions on how to implement the program for technologically incompetent people 😅😅 I’m trying to figure it out by following the notes you’ve left on GitHub, but I’m truly just. Not good at computers haha. I got all the way up to unzipping the files and hitting ao3downloader.cmd. Black pop-up screen flickers to life for 2 seconds and then closes out again and I don’t know what’s wrong (if there is something wrong).

Hi, thanks for contacting me! That does sound problematic and I'm happy to help if I can. First, pardon me for asking but have you tried all the troubleshooting steps listed at the bottom of the readme? In particular, make absolutely sure you have the right version of python installed. Sounds like you're on Windows, so in order to check this

hit the windows key + r

type in "cmd"

hit enter

type in "python --version" (without the quotes)

hit enter

you should see a message that says "Python 3.9.9"

if it doesn't say exactly "Python 3.9.9", go to the start menu, type in "programs", and click on "Add or remove programs". Then find anything that says "Python" in it and uninstall it. Restart your computer, then go to the link at the top of the readme to install python version 3.9.9. make sure you choose the right installer for your operating system!

after this, delete your current copy of the ao3downloader repository, redownload it from github, unzip it as before, and try again to double-click on ao3downloader.cmd

Finally, if all of that still doesn't work... consider sending me an email! Troubleshooting via anon message isn't the easiest. :)

[nianeyna]

so now that I have a fairly large number of fanfictions backed up to calibre in orderly fashion... a Problem has occurred to me. which is that many of these fanfictions are Unfinished. so I think to myself, there must be a way to periodically check these Unfinished fics to see if they have been updated, right?

not really.

it was suggested that FanFicFare might help me with this but as it turns out it doesn't really work that way, it's more of a bulk downloader similar to mine. which makes it more or less useless to me unless I want to download a fic from ff.net or something. I mean it has a "check for updates" option but I'm frankly not sure what it does other than show you a progress bar for a couple hours.

anyway.

so in true nia fashion I thought to myself, fanficfare may not be able to check your library for unfinished fics but surely I can! so I started reading the documentation for how to write a calibre plugin and then I decided not to do that.

instead I made a branch in the ao3downloader repository and added yet another menu item to it and started throwing stuff at the wall to see what sticks. so first we need to find all of the .epub files in my calibre library, that's os.path stuff, fairly trivial. then I need to see what's inside those .epub files. slightly more complicated but there's a python package for that. okay now I have the first xhtml file in the epub file which I'm going to assume has the front matter in it, and I can use etree to parse through and finds where it says "Chapters: 10/12 or whatever. this seems to be successful which is cool. some gnarly string parsing later I have the numbers before and after the slash, and I assume if they don't match the fic is incomplete. yay! now I have a solid list of incomplete fics.

what I am going to do with them is a problem for tomorrow nia.