#kitkatt0430 tutorials

[kitkatt0430]

Welp, I've been using external methods of auto-backing up my tumblr but it seems like it doesn't do static pages, only posts.

So I guess I'll have some manual backing up to do later

Still, it's better than nothing and I'm using the official tumblr backup process for my smaller blogs so hopefully that'll net the static pages and direct messages too. But. My main - starstruckpurpledragon - 'backed up' officially but was undownloadable; either it failed or it'd download a broken, unusable, 'empty' zip. So *shrugs* I'm sure I'm not the only one who is trying to back up everything at once. Wouldn't be shocked if the rest of the backups are borked too when I try to download their zips.

There are two diff ways I've been externally backing up my tumblr.

TumblThree - This one is relatively straight forward in that you can download it and start backing up immediately. It's not pretty, but it gets the job done. Does not get static pages or your direct message conversations, but your posts, gifs, jpegs, etc are all there. You can back up more than just your own blog(s) if you want to as well.

That said, it dumps all your posts into one of three text files which makes them hard to find. That's why I say it's 'not pretty'. It does have a lot of options in there that are useful for tweaking your download experience and it's not bad for if you're unfamiliar with command line solutions and don't have an interest in learning them. (Which is fair, command line can be annoying if you're not used to it.) There are options for converting the output into nicer html files for each post but I haven't tried them and I suspect they require command line anyway.

I got my blogs backed up using this method as of yesterday but wasn't thrilled with the output. Decided that hey, I'm a software engineer, command line doesn't scare me, I'll try this back up thing another way. Leading to today's successful adventures with:

TumblrUtils - This one does take more work to set up but once it's working it'll back up all your posts in pretty html files by default. It does take some additional doing for video/audio but so does TumblThree so I'll probably look into it more later.

First, you have to download and install python. I promise, the code snake isn't dangerous, it's an incredibly useful scripting language. If you have an interest in learning computer languages, it's not a bad one to know. Installing python should go pretty fast and when it's completed, you'll now be able to run python scripts from the command line/terminal.

Next, you'll want to actually download the TumblrUtils zip file and unzip that somewhere. I stuck mine on an external drive, but basically put it where you've got space and can access it easily.

You'll want to open up the tumblr_backup.py file with a text editor and find line 105, which should look like: ''' API_KEY = '' '''

So here's the hard part. Getting a key to stick in there. Go to the tumblr apps page to 'register' an application - which is the fancy way of saying request an API. Hit the register an application button and, oh joy. A form. With required fields. *sigh* All the url fields can be the same url. It just needs to be a valid one. Ostensibly something that interfaces with tumblr fairly nicely. I have an old wordpress blog, so I used it. The rest of the fields should be pretty self explanatory. Only fill in the required ones. It should be approved instantly if everything is filled in right.

And maybe I'll start figuring out wordpress integration if tumblr doesn't die this year, that'd be interesting. *shrug* I've got too many projects to start a new one now, but I like learning things for the sake of learning them sometimes. So it's on my maybe to do list now.

Anywho, all goes well, you should now have an 'OAuth Consumer Key' which is the API key you want. Copy that, put in between the empty single quotes in the python script, and hit save.

Command line time!

It's fairly simple to do. Open your command line (or terminal), navigate to where the script lives, and then run: ''' tumblr_backup.py <blog_name_here> '''

You can also include options before the blog name but after the script filename if you want to get fancy about things. But just let it sit there running until it backs the whole blog up. It can also handle multiple blogs at once if you want. Big blogs will take hours, small blogs will take a few minutes. Which is about on par with TumblThree too, tbh.

The final result is pretty. Individual html files for every post (backdated to the original post date) and anything you reblogged, theme information, a shiny index file organizing everything. It's really quite nice to dig through. Much like TumbleThree, it does not seem to grab direct message conversations or static pages (non-posts) but again it's better than nothing.

And you can back up other blogs too, so if there are fandom blogs you follow and don't want to lose or friends whose blogs you'd like to hang on to for your own re-reading purposes, that's doable with either of these backup options.

I've backed up basically everything all over again today using this method (my main is still backing up, slow going) and it does appear to take less memory than official backups do. So that's a plus.

Anyway, this was me tossing my hat into the 'how to back up your tumblr' ring. Hope it's useful. :D

[kitkatt0430]

Instead of doing a Six Sentence Sunday today, I think I'll do a short tutorial on copying over fanfic from FFnet to Ao3.

So you've got some old fics on FFnet and you'd like to back them up to Ao3, given the instability of FFnet. And for whatever reason you don't have the original files for the fics, or maybe you have edits to the FFnet versions that you don't want to lose that the OG files don't have. Whatever the reason, you're looking to directly copy over your fic from FFnet to Ao3. And you're looking for a relatively easy way to do so, but Ao3's import functionality doesn't work with FFnet web pages.

Never fear! It's actually a fairly easy process to get your fic copied over from FFnet.

First, head over to FFnet and open up the fic you want to port over to Ao3. You don't need to log in if you don't want to, just so long as the fic in question is yours and you can access the page, then you're good.

In a separate tab, open Ao3 and login, then choose the option for posting a new work.

Now back on the FFnet tab, you should be able to directly copy over the title, summary, fandom, and what little tagging was available on that site onto the relevant Ao3 fields in the tab you have for a new fic. You'll also want to take note of the published date on FFnet and back date the new work in the Ao3 tab.

FFnet may not have a lot of useful tag data, but it's pretty easy to replicate and build off that in Ao3.

Now for the hard part. Which is still pretty easy. Getting the fic body, plus any notes in the fic itself, copied over to FFnet.

While getting around FFnet's lockdown on the text of the fics they host is fairly simple - I'm pretty sure it's entirely css based - you don't really need to do that in order to get the body of your fic copied. And, honestly, even if you do have a work around in place to allow copying of the fic's text... you will probably find the following method a lot easier still.

In the body of the fic, right click the first line of the fic, which should bring up a menu with a bunch of options. On Firefox or Chrome you want the inspect option.

This'll bring up the dev tools with the html inspection tab open and, if you give it a few seconds to load, the specific line you right clicked to inspect should become the visibly selected section of the html.

The selected section of the html should be a paragraph (or <p>) element. You're going to want to right click the div (<div>) element that encapsulates that paragraph and the rest of the paragraphs in the fic body. This'll bring up another browser menu with the option to copy, which will bring up a flyout menu when you select it. From that flyout menu, you want the select the option for Inner HTML.

You have officially copied the html for the fic body. And you can dump that entirely in html format straight into Ao3's html work text editor. Then switch it to rich text for easier editing if you want to fix any spelling, grammar, formatting, or aesthetic issues. I typically try to fix at least the line breaks since it took a long while before FFnet adopted real line breaks and so there are a lot of fics where I have various combinations of dashes, em-dashes, equals signs, and other characters as line breaks. I figure, if I'm bringing the fic to Ao3 then I can try to make it more screen reader friendly in the process.

You can also move fic notes around in order to move pre/post fic notes out of the fic body or basically whatever you want to the fic. Maybe re-read it to determine any additional tagging you want to add now that your fic has access to Ao3's much more robust tagging system.

But that's it. You can hit post and have your fic with all it's original notes, and a back dated post date to reflect when it was actually written, all available on Ao3 now.

It's a pretty quick process, all told, and the only real bottleneck you might encounter is any time spent in re-editing the fic between migrating and posting. Even chaptered fics are fairly easy to migrate with this process, since the bulk of the work in publishing a new chapter is just copying the inner html and then moving any notes to the appropriate location before hitting post.

Anyway, for my fellow fic writers looking to move your old FFnet fics to a more stable archive, I hope this process helps a lot.

[kitkatt0430]

Manipulating Metadata with FanFicFare - Part 2

So last time I covered how to manipulate fic rating tags, fandom tags, and character tags in order to standardize these tags in your Calibre fanfic library. This time lets look at relationship tags. There's a lot going on with those, so we'll stick to just the one tag type for this post.

Standardizing Relationship Tags:

Similar to Character tags, you can wind up with multiple tags in different archives, or even the same archive, that represent the same relationship. And if you're picky about name order being alphabetized…

Thankfully relationship tags are pretty straightforward to manipulate.

For Ao3, or Ao3 clones like SquidgeWorld, you have a few different relationship types to consider. Ao3 only officially supports romantic - / - and platonic - & - relationship markers, but it allows various unofficial queerplatonic - ~, +, etc - and antagonistic - vs, etc - indicators as well. SquidgeWorld officially supports more relationship types, standardizing queerplatonic relationships with ~ in it's site faqs since there was some consensus towards that particular marker, but like Ao3 the site has been known to accept unofficial relationship indicators as well (though the site lead is also pretty quick about learning the reasons for these alternate indicators and updating to use them too if the relationship they represent doesn't already have an indicator).

Other sites are more likely to only tag for romantic relationships or even constrain tagging in a way you may still need to update manually. Ffnet, for example, only allows a max of two relationships to be tagged on a fic because the max number of characters that can be tagged is four and the relationship tagging is basically a sub tagging of the character tags. And while you can tag more than two people in one relationship, if the polycule is bigger than four characters then the writer has to choose which character(s) to leave out. Meaning that for sites like FFnet, you have to be aware of what the tagging limitations are in order to decide when you still need to go back later to manually update the metadata after downloading a fic and when the site tagging will be adequate after it's been run through the personal.ini file rules on download.

To keep the examples easy for me to keep track of in the next sections I'm going to use some Flash characters here. Barry Allen, Iris West, Eddie Thawne, and Eobard Thawne. And also break this down by indicators, using the four main ones from SquidgeWorld - '/', '&', '~', and 'vs' - since you'll see these on Ao3 too.

Romance indicators:

On various fics, you might see a relationship between Iris West and Eddie Thawne tagged as Iris West/Eddie Thawne and Eddie Thawne/Iris West. SquidgeWorld and Ao3 both recognize these as being the same tag, but Calibre isn't going to be able to do that. So you're going to have to pick one and alter the other to match with a 'replace_metadata:' rule. Assuming alphabetical name order, Eddie Thawne will come before Iris West. So you'll want a rule that looks like:

''' ships=>Iris West/Eddie Thawne=>Eddie Thawne/Iris West '''

For the Barry Allen and Iris West ships, however, you might see 'The Flash' get thrown in there. Barry/Iris, Iris/Barry, Barry Allen/Iris West, Iris West/Barry Allen, The Flash/Iris West, Barry Allen | The Flash/Iris West, Iris West/The Flash, Iris West/Barry Allen | The Flash, The Flash | Barry Allen/Iris West, and Iris West/The Flash | Barry Allen. You might even see 'NewsFlash' or 'WestAllen' in place of the traditional 'Character A/Character B' tagging indicators.

That's… a lot of potential tags all for one ship, am I right? To be honest, I don't think the Flash fandom actually uses all those potential tags but this gives you an idea of how many permutations of one relationship tag you might find yourself dealing with elsewhere. And if you want to consolidate all these into one tag with the names in alphabetical order, you'd want a set of tags something like these.

First, you'd want a few tags to consolidate Barry's name into just Barry Allen ''' ships=>Barry Allen | The Flash=>Barry Allen ships=>The Flash | Barry Allen=>Barry Allen ships=>The Flash=>Barry Allen ''' This turns all 'Barry Allen | The Flash' and 'The Flash | Barry Allen' instances into 'Barry Allen' and then converts and remaining standalone 'The Flash' instances into 'Barry Allen' too.

Next you'd want these rules to further consolidate the tags: ''' ships=>NewsFlash=>Barry Allen/Iris West ships=>WestAllen=>Barry Allen/Iris West ships=>Iris West/Barry Allen=>Barry Allen/Iris West ''' This should fix everything except for the 'Barry/Iris' and 'Iris/Barry' tags. And before fixing those, consider that there could also be Barry/Iris West or Iris/Barry Allen. Yup, throwing two more tag permutations at you. Handle the tagging wrong and you could wind up with Barry Allen/Iris West Allen or Barry Allen/Iris West West. ''' ships=>Barry/Iris=>Barry Allen/Iris West ships=>Iris/Barry=>Barry Allen/Iris West ships=>West Allen=>West ships=>West West=>West ''' This, however, should smooth out any issues remaining with the tags I listed and hopefully prepare you for other issues in tagging that could show up down the line.

But what about a polycule? Westhawne is one of my favorite polycules for The Flash, so let's use that as an example. It could be tagged as Barry Allen/Eddie Thawne/Iris West, Barry Allen/Iris West/Eddie Thawne, Iris West/Eddie Thawne/Barry Allen, Iris West/Barry Allen/Eddie Thawne, Eddie Thawne/Barry Allen/Iris West, or Eddie Thawne/Iris West/Barry Allen.

If all the rules above for Eddie/Iris and Barry/Iris are already in place then some of these will have already been manipulated a bit without adding any new rules at all. 'Barry Allen/Iris West/Eddie Thawne' will already be changed to the alphabetized 'Barry Allen/Eddie Thawne/Iris West' thanks to the rule turning 'Iris West/Eddie Thawne' into 'Eddie Thawne/Iris West'.

'Iris West/Eddie Thawne/Barry Allen' will get manipulated twice. The first time it will hit the Eddie/Iris rule that flips the order of their names. Then it'll hit the Iris/Barry rules that flips the order of those two names as well, leaving the final result as 'Eddie Thawne/Barry Allen/Iris West'.

To clean that up and also alphabetize any Barry Allen/Eddie Thawne fics you might download, you can add this rule: ''' ships=>Eddie Thawne/Barry Allen=>Barry Allen/Eddie Thawne '''

Because the 'replace_metadata:' rules are applied in the order they're written in, this will make the final output of a fic with the tag 'Iris West/Eddie Thawne/Barry Allen' become 'Barry Allen/Eddie Thawne/Iris West'.

'Iris West/Barry Allen/Eddie Thawne' however, will only be affected by the 'Iris West/Barry Allen' rule, since Iris's name won't be moved to form 'Iris West/Eddie Thawne' until after that rule has already been checked and found not to apply. So with the current ruleset, it'll become 'Barry Allen/Iris West/Eddie Thawne'.

At this point you'll want to add one of the two following rules: ''' ships=>Barry Allen/Iris West/Eddie Thawne=>Barry Allen/Eddie Thawne/Iris West ''' or ''' ships=>/Iris West/Eddie Thawne=>/Eddie Thawne/Iris West ''' There are advantages to each option, so it's really down to personal preference and how common it is for you to read polycules that involve Iris and Eddie, but not Barry.

Let's finish addressing the rest of the tag permutations. 'Eddie Thawne/Barry Allen/Iris West' would be altered by the Barry/Eddie rule to become 'Barry Allen/Eddie Thawne/Iris West'. And 'Eddie Thawne/Iris West/Barry Allen' will first be affected by one of the Iris/Barry rules to become 'Eddie Thawne/Barry Allen/Iris West', then the Eddie/Barry rule will come into play, leaving the ship in an alphabetized state. Thanks to the existing rules and the order they're applied, no new rules are needed in either case.

If you have ship rules in place for a subset - or subsets - of a polycule already, it's a good idea to go ahead and download a fic to see what the metadata is currently being massaged into. You can then add a rule and re-download just the metadata for the fic to verify the new rule works as expected.

Platonic ships:

There's not a lot here that is different from the romantic ships. The biggest, and really only difference, in how to handle these tags is the ampersand ('&') is handled a bit differently in Regular Expressions (RegExp) than most characters. You can't just use the & symbol on it's own and it's one that can't be handled using a preceding backslash to denote usage of the character and not the special meaning RegExp gives it.

In this particular case, the & symbol has to be represented by a collection of characters -> '&amp;' (in case tumblr tries to eat that it's '& a m p ;' minus the spaces and I'm going to be quietly swearing over my examples for a while as I try to figure out how to fix them)

A couple of examples of manipulating platonic ship tags: ''' ships=>Iris West &amp; Barry Allen=>Barry Allen &amp; Iris West ships=>Eddie Thawne &amp; Barry Allen=>Barry Allen &amp; Eddie Thawne '''

You might also want to break up platonic tags into groups of two, if you want Barry & Eddie & Iris to show up with your other Barry & Eddie, Eddie & Iris, and Barry & Iris fics. (Or you might feel like the tags convey different meanings entirely, but lets have a quick lesson in splitting one tag into multiple tags anyway, okay?)

To split one tag into multiple tags, you can do something like this: ''' ships=>Barry Allen &amp; Eddie Thawne &amp; Iris West=>Barry Allen &amp; Eddie Thawne, Barry Allen &amp; Iris West, Eddie Thawne &amp; Iris West ''' the comma denotes the end of one tag and the start of a new tag once it's input into Calibre's columns, but while FanFicFare is doing it's thing this is still a simple string manipulation. So as long as you don't try to start a new line for each tag you want to add in place of the existing tag, then all the tags you want to replace it with will show up in the final output.

Queerplatonic Ships:

Again, the way to handle QPR ships is pretty similar to how the romance ships are handled. In this case, however you'll need to use the backslash before the tilde symbol so it looks like '\~' (and if tumblr eats my backslashes again '\ ~' with no space in there)

''' ships=>Iris West ~ Barry Allen=>Barry Allen ~ Iris West ships=>Eddie Thawne ~ Barry Allen=>Barry Allen ~ Eddie Thawne '''

Bit of a warning however, if you are importing fics that mix the romance and QPR indicators in one tag. Running those mixed tags through these rules could wind up radically changing what's being indicated by the tag.

For Example, 'Iris West/Eddie Thawne ~ Barry Allen' would indicate a romantic relationship between Eddie and Iris that includes Barry as a queerplatonic partner. But run it through all those rules above and it's going to mutate a bit. It'll hit the rule for 'Iris West/Eddie Thawne' first and change the tag to 'Eddie Thawne/Iris West ~ Barry Allen' which isn't so bad. But then it reaches our new 'Iris West ~ Barry Allen' tag and flips those two names around, turning the tag into 'Eddie Thawne/Barry Allen ~ Iris West' which would mean Eddie and Barry have the romantic relationship and Iris is the queerplatonic partner. Which is… not correct.

For this case you may want to add a rule before all the existing 'ships' rules that would put the tag in the expected order: ''' ships=>Iris West/Eddie Thawne \~ Barry Allen=>Barry Allen \~ Eddie Thawne/Iris West '''

But this gets more complicated if Barry's name actually came after Eddie's but before Iris's alphabetically since it would still wind up mutating thanks to the rules that would come after it. In that case you may prefer adding rules to split mixed ship tags like this into separate tags, similar to the example above for platonic tags. In this case you'd still want to add this rule before the rules that would mutate the tag and do something like this: ''' ships=>Iris West/Eddie Thawne \~ Barry Allen=>Eddie Thawne/Iris West, Barry Allen \~ Eddie Thawne, Barry Allen \~ Iris West ''' This will mutate the original tag into a tag for the romantic relationship between Eddie and Iris with additional tags to indicate that Barry is queerplatonic partners with both Eddie and Iris. It may not be as succinct as the original tag, but the three tags together convey the same information and will make the fic available when searching for any three of those tags or any combination of those tags.

Antagonistic ships:

I do love that SquidgeWorld officially supports these relationship indicators. Sometimes I just want Eobard Thawne trying to fight Barry like Wile E. Coyote tries, and fails, to defeat the Roadrunner.

Again, not a lot of differences between how to handle these tags and the romance tags. However, you're probably going to see things like 'Barry Allen vs Eobard Thawne', 'Eobard Thawne vs. Barry Allen', 'Eobard Thawne | Harrison Wells VS Barry Allen', etc, so you'll need to decide first whether you want 'vs' or 'vs.' or 'VS' or 'VS.' as your antagonistic relationship indicator. Once you do, add these rules before the rest of your antagonistic relationship rules: ''' ships=> vs.=> vs ships=> VS=> vs ships=> VS.=> vs ''' You'll need to use the preceding space to help identify an actual antagonistic relationship indicator and you'll want to preserve that space in the modified tag. You'll also need a backslash before the period to indicate the character and not the RegExp operator.

From there you can do the usual name alphabetizing tricks to get the names in the right order so that all the tag variations will be consolidated.

Well, that takes care of Ao3 and sites that make use of it's open source UI code. But what about FFnet?

Barry Allen/Iris West would be represented as Barry A./Iris W. or Iris W./Barry A. depending on the order the characters are added to the fic.

Add these rules to the 'replace_metadata:' section for 'fanfiction.net' ''' ships=>Barry A.=>Barry Allen ships=>Iris W.=>Iris West ships=>Iris West/Barry Allen=>Barry Allen/Iris West ''' This will change the character names to their full names and then switch them into alphabetical order if they aren't already.

Finally…

If at any time you want to lock a rule to a specific fandom, be it for a FFnet tag or Ao3 tag or whatever, you can always do the following: ''' ships=>Iris West/Barry Allen=>Barry Allen/Iris West&&category=>The Flash ''' This rule would only change 'Iris West/Barry Allen' into the alphabetized 'Barry Allen/Iris West' for the Flash fandom, meaning if you have a Barry/Iris ship for a different fandom where there are entirely different and unrelated characters with the same name (unlikely but not entirely impossible), then you could use the alphabetized version for one fandom and the reverse alphabetized name order for the other fandom. Or you could also do something like this:

''' ships=>Iris West/Barry Allen=>Barry Allen/Iris West (The Flash)&&category=>The Flash ships=>Iris West/Barry Allen=>Barry Allen/Iris West (Mystery Fandom)&&category=>Mystery Fandom '''

That's it for now, but there's still more to talk about when it comes to manipulating tags using FanFicFare's personal.ini file.

[kitkatt0430]

Manipulating Metadata with FanFicFare - Part 3

One of the many things I like about Ao3 is how it can so very easily group fics in a series. And one fic can even be in multiple series which is awesome for things like subseries or prompt challenges. But when you're downloading fanfics you probably noticed that for fics attached to multiple series, only the first series has it's information added to the downloaded fanfic's metadata. Or that if you download a series as both separate fics and as an anthology, the anthology isn't being marked as part of the series.

These are all fairly easy to fix, but lets start with the easiest to fix first. Which is the anthology issue.

Head into the FanFicFare menu and open the Configure FanFicFare model. Then choose the "Standard Columns" tab and take a look at the table of options for each standard column. Specifically the row of options for the 'Series' column. There should be three checkbox items: "New Only", "Set Calibre Series URL", and "Set 'Series [0]' for New Anthologies".

Make sure that last one is checked and the first one isn't. Then go to the OK button and click that, which will save and close the model.

Now go find either a new series to download as an anthology or one that's already downloaded (should have 'Anthology' in the title). For a new series use the "Get Story URLs from Web Page" option, choose the option for making an anthology, approve the URLs it finds, and let it download everything. For an existing anthology, select the fic in Calibre's book list and then use the FanFicFare option "Update Anthology Epub" in the FanFicFare menu (in the "Anthology Options" flyout) and walk through the update flow (basically the same as the add new anthology flow).

When it's done adding or updating the anthology, you should see it's added as "0 of <Series Name>".

Okay, that's the easy part. Now for adding some custom columns for those extra series listings.

First, lets take a look in the plugin-defaults.ini file. Well, I'm taking a look and pasting my findings here so you don't have to go digging, but it's pretty interesting to take a look through what's in the default settings for FanFicFare. (This file is found in the Configure FanFicFare modal in the "personal.ini" tab.)

Ao3 (and thus any supported Ao3 clones like SquidgeWorld) are going to have the following "extra_valid_entries:" options set in the plugin-defaults.ini file.

extra_valid_entries:fandoms, freeformtags, freefromtags, ao3categories, comments, chapterslashtotal, chapterstotal, kudos, hits, bookmarks, collections, byline, bookmarked, bookmarktags, bookmarksummary, bookmarkprivate, bookmarkrec, subscribed, markedforlater, restricted, series00, series01, series02, series03, series00Url, series01Url, series02Url, series03Url, series00HTML, series01HTML, series02HTML, series03HTML

That's a lot of metadata that FanFicFare scrapes. Some of it's transformed into standardized metadata information already. 'byline' presumably becomes 'author', 'fandoms' gets funneled into 'category', and so on. Of course, what we care about for the sake of this tutorial is the series data.

Specifically, lets take a look at series00, series01, series02, and series03.

Series00 is going to be the same information already being populated into the existing column for Series. Which means that extra series data for a fic in more than one series will be in the series01, series02, and series03 entries that aren't currently being saved anywhere.

Time to make some custom columns.

Head into the Calibre Preferences and then choose "Add your own columns". Click the button to "Add custom column" and fill out the following info:

Lookup name: series01 Column heading: Series 01 (Note: this can be whatever you want to name it) Column type: Text column for keeping series-like information

That should be all you really need. Hit OK and then add two more columns for series02 and series03.

It is important that the Lookup name matches the entry name for the metadata. For whatever reason, linking non matchy Lookup names to metadata entry names is a pain in the butt that I'm still working on figuring out. (It shouldn't be this hard and yet… sigh)

Once you've got your new columns added, hit Apply and restart Calibre. Now we can map the metadata entries to the new columns, we're officially halfway done. (It's all easy going again from here too.)

Time to go open up the FanFicFare personal.ini file to make a few minor changes. Under the "[archiveofourown.org]" section, before any metadata manipulation begins - so probably right beneath your username and password if you've set them - add the following four lines:

''' custom_columns_settings: series01=>#series01 series02=>#series02 series03=>#series03 '''

If you have a "[www.squidgeworld.org]" section you'll want to past those four lines there too.

Click the OK button to save and close the personal.ini file. Now go download a new fic associated with multiple series or find one you've already downloaded and update it's metadata only (unless you know it has new chapters). When the download/update process is done, you should see that the missing series data has been populated

Wait, however, there's one more thing you'll want to do. Because fic 1 in Series A might have Series A listed in the Series column but fic 2 might have it in the Series 01 column, searching just one column for all the fics in a series is no longer going to cut it. Searching 'series:"=Series A' won't return fic 2, but '#series01:"=Series A" will leave out fic 1. And if you want to combine the two search terms, it can be tedius to have to do that every single time.

The answer to this problem is to create a custom search term so that you can do 'allseries:"=Series A"' and have the search return Fic 1 and Fic 2. To do this, head back into Calibre's Preferences. In the top section labeled "Interface", the last option on the right should be "Searching" with a blue magnifying glass icon. Click that and then head to the "Grouped searches" tab.

In the Names input, type 'allseries'. Then head to the Value input. This one will offer auto-complete options as you type which is pretty useful - basically you want to type in the Lookup names for the columns you want the search to check and separate those Lookup names with commas. You'll want it to look like 'series, #series01, #series02, #series03,' when you're done filling it in. Note that the standard column series doesn't have a hashtag before it's Lookup name, but the custom columns do.

With the Names and Value inputs filled, the Save button should be available now. Click that to save the new search term and then click OK to close the modal. Exit from the Calibre Preferences and test out your new allseries search option.

You may need to go back and update metadata for a bunch of fics you've previously downloaded now, but it's definitely useful data to be able to keep track of.

[kitkatt0430]

Don't know how interested anyone is, but I'm finally putting together a how to document for creating an epub in Calibre for backing up fanfiction that your typical fanfic scrapper can't grab.

I'll be using a couple of my tumblr posts to demo the process. I'll be creating a blank epub with Calibre and then using the ebook editor bundled with Calibre to add chapter html files that I can then copy/paste the html from my tumblr posts into the chapter files. I'll also demonstrate how to include image files for any fanart that may accompany the fic.

This is obviously not a technique for backing up fanfics anyone would need to consider for any of the popular fic archives, FanFicFare and several other options cover those quite well. But old fics on Live Journal, fics that exist across multiple tumblr posts, fics that only exist on the Internet Archive now... those are all things that this is useful for.

[kitkatt0430]

With the latest news about Tumblr going into extended maintenance mode, I think we're all pretty worried about the continued health of Tumblr and what'll happen to our fandom spaces if Tumblr winds up going away. While I'm not super worried about losing Tumblr at the moment, this has reminded me that Tumblr isn't always the easiest to navigate of blog spaces and I've been wanting to - trying and kinda failing at - backing up my Tumblr posts elsewhere for a while.

I'm not going anywhere - Tumblr is gonna have to die for me to leave - but backing things up elsewhere is always a good practice and something I've been meaning to do for a long while.

I'm still working out what I want to back up where - some will definitely be on Pillowfort, so look for me to start being more active there as starstruckpurpledragon but I'm also revamping my old Wordpress blogs and deciding what to do with my Dreamwidth account.

It's looking like my Kitkatt0430 Wordpress is going to be where my Calibre + FanFicFare tutorials are going to wind up, as will the Fic Request tracking and guidelines, though Tumblr Asks/Submissions are still the only places I'll be accepting requests (I may back this information up to Pillowfort as well, or at least link the wordpress blog pages there). I'm also going to create on the wordpress blog a Masterpost tracking all my fics - where to find them across different archives and what formats (text/podfic) are they available in. Which will be a massive undertaking, but be super helpful for me when it's done since I'll be able to tell more easily what's been crossposted where and what fics haven't been podficced yet. I've placed the old blog under construction for now, so nothing is visible yet, but I'll start linking it here when it's at that ready-enough stage of things that I don't mind people seeing it.

I'll also be digging through all my old posts which may help with the slump I've been in writing wise this last month or so. Though that's at least partially attributable to the effort I've been making to get everything on Ao3 crossposted to SquidgeWorld, since it's been taking up a lot of my time lately.

[kitkatt0430]

The State of the KitKatt0430

I'll be keeping this pinned post updated with my current statuses and any important page links.

If you want to find me in places other than tumblr, check out my new Where to Find Me page

To specifically check if I've seen and accepted a fic request or headcanon ask, check the Fic And Headcanon Acceptance Status page. I'll be keeping that updated (relatively) regularly with any new accepted asks and a general status showing which have been accepted and which are in progress.

Fic Requests:

Current Status - Not currently open for asks/submissions for fic requests

Notes on the request closure: I'll be leaving asks/submissions open for the ask games I like to play - which may include prompt games like the kiss fic meme - but any further fic requests will be rejected at this time until I'm ready to start accepting them again. I just have quite a backlog of requests at the moment and would like to spend time focusing on the requests I've already got before adding any more to the todo list.

More info - Check out my Fic Requests Faq for more info on how to request a fic and what my boundaries regarding fic requests look like. It's a living document, so if you haven't checked it out recently and want to request a fic then it might be a good idea to do so before sending your request as it may have changed since you last saw it.

Headcanons:

Current Status - Open for asks/submissions for headcanon discussions

More info - I love talking about my headcanons for fandoms I enjoy, even if they're not fandoms I'm actively writing for. Feel free to send questions about my headcanons via ask/submission and I'll usually respond pretty quickly. Though you might just wind up sparking an entire fic. ;)

Fic Archiving Tutorials:

Ever wondered how to download and maintain your own local archive of fanfictions? That way if an archive goes down or someone takes down one of their works that you've bookmarked, you'll still have a copies of the now lost fics for private use. If that's something you're interested in then perhaps my fanfic archiving series is for you.

Looking to migrate your fics from Fanfiction.net to Ao3 quickly? I've got a tutorial for that too.

[kitkatt0430]

Just to add on about TumblThree, it is really finicky about certain things.

It initially downloaded half my main and then got interrupted. When I re-ran it, it would add new posts but not the half of the blog missed on the original pass. I had to remove and re-add my blog and run it all over from start to get the whole thing.

So that also played into me deciding to try a different backup method.

Welp, I've been using external methods of auto-backing up my tumblr but it seems like it doesn't do static pages, only posts.

So I guess I'll have some manual backing up to do later

Still, it's better than nothing and I'm using the official tumblr backup process for my smaller blogs so hopefully that'll net the static pages and direct messages too. But. My main - starstruckpurpledragon - 'backed up' officially but was undownloadable; either it failed or it'd download a broken, unusable, 'empty' zip. So *shrugs* I'm sure I'm not the only one who is trying to back up everything at once. Wouldn't be shocked if the rest of the backups are borked too when I try to download their zips.

There are two diff ways I've been externally backing up my tumblr.

TumblThree - This one is relatively straight forward in that you can download it and start backing up immediately. It's not pretty, but it gets the job done. Does not get static pages or your direct message conversations, but your posts, gifs, jpegs, etc are all there. You can back up more than just your own blog(s) if you want to as well.

That said, it dumps all your posts into one of three text files which makes them hard to find. That's why I say it's 'not pretty'. It does have a lot of options in there that are useful for tweaking your download experience and it's not bad for if you're unfamiliar with command line solutions and don't have an interest in learning them. (Which is fair, command line can be annoying if you're not used to it.) There are options for converting the output into nicer html files for each post but I haven't tried them and I suspect they require command line anyway.

I got my blogs backed up using this method as of yesterday but wasn't thrilled with the output. Decided that hey, I'm a software engineer, command line doesn't scare me, I'll try this back up thing another way. Leading to today's successful adventures with:

TumblrUtils - This one does take more work to set up but once it's working it'll back up all your posts in pretty html files by default. It does take some additional doing for video/audio but so does TumblThree so I'll probably look into it more later.

First, you have to download and install python. I promise, the code snake isn't dangerous, it's an incredibly useful scripting language. If you have an interest in learning computer languages, it's not a bad one to know. Installing python should go pretty fast and when it's completed, you'll now be able to run python scripts from the command line/terminal.

Next, you'll want to actually download the TumblrUtils zip file and unzip that somewhere. I stuck mine on an external drive, but basically put it where you've got space and can access it easily.

You'll want to open up the tumblr_backup.py file with a text editor and find line 105, which should look like: ''' API_KEY = '' '''

So here's the hard part. Getting a key to stick in there. Go to the tumblr apps page to 'register' an application - which is the fancy way of saying request an API. Hit the register an application button and, oh joy. A form. With required fields. *sigh* All the url fields can be the same url. It just needs to be a valid one. Ostensibly something that interfaces with tumblr fairly nicely. I have an old wordpress blog, so I used it. The rest of the fields should be pretty self explanatory. Only fill in the required ones. It should be approved instantly if everything is filled in right.

And maybe I'll start figuring out wordpress integration if tumblr doesn't die this year, that'd be interesting. *shrug* I've got too many projects to start a new one now, but I like learning things for the sake of learning them sometimes. So it's on my maybe to do list now.

Anywho, all goes well, you should now have an 'OAuth Consumer Key' which is the API key you want. Copy that, put in between the empty single quotes in the python script, and hit save.

Command line time!

It's fairly simple to do. Open your command line (or terminal), navigate to where the script lives, and then run: ''' tumblr_backup.py <blog_name_here> '''

You can also include options before the blog name but after the script filename if you want to get fancy about things. But just let it sit there running until it backs the whole blog up. It can also handle multiple blogs at once if you want. Big blogs will take hours, small blogs will take a few minutes. Which is about on par with TumblThree too, tbh.

The final result is pretty. Individual html files for every post (backdated to the original post date) and anything you reblogged, theme information, a shiny index file organizing everything. It's really quite nice to dig through. Much like TumbleThree, it does not seem to grab direct message conversations or static pages (non-posts) but again it's better than nothing.

And you can back up other blogs too, so if there are fandom blogs you follow and don't want to lose or friends whose blogs you'd like to hang on to for your own re-reading purposes, that's doable with either of these backup options.

I've backed up basically everything all over again today using this method (my main is still backing up, slow going) and it does appear to take less memory than official backups do. So that's a plus.

Anyway, this was me tossing my hat into the 'how to back up your tumblr' ring. Hope it's useful. :D