tumblr-backup and datasette
I've been using tumblr_backup, a script that replicates the old Tumblr backup format, for a while. I use it both to back up my main blog and the likes I've accumulated; they outnumber posts over two to one, it turns out.
Sadly, there isn't an 'archive' view of likes, so I have no idea what's there from way back in 2010, when I first really heavily used Tumblr. Heck, even getting back to 2021 is hard. Pulling that data to manipulate it locally seems wise.
I was never quite sure it'd backed up all of my likes, and it turns out that a change to the API was in fact limiting it to the most recent 1,000 entries. Luckily, someone else noticed this well before I did, and a new version, tumblr-backup, not only exists, but is a Python package, which made it easy to install and run. (You do need an API key.)
I ran it using this invocation, which saved likes (-l), didn't download images (-k), skipped the first 1,000 entries (-s 1000), and output to the directory 'likes/full' (-O):
tumblr-backup -j -k -l -s 1000 blech -O likes/full
This gave me over 12,000 files in likes/full/json, one per like. This is great, but a database is nice for querying. Luckily, jq exists:
jq -s 'map(.)' likes/full/json/*.json > likes/full/likes.json
This slurps (-s) in every JSON file, iterates over them to make a list, and then saves it in a new JSON file, likes.json. There was a follow-up I did to get it into the right format for sqlite3:
jq -c '.[]' likes/full/likes.json > likes/full/likes-nl.json
A smart reader can probably combine those into a single operator.
Using Simon Willison's sqlite-utils package, I could then load all of them into a database (with --alter because the keys of each JSON file vary, so the initial column setup is incomplete):
sqlite-utils insert likes/full/likes.db lines likes/full/likes-nl.json --nl --alter
This can then be fed into Willison's Datasette for a nice web UI to query it:
datasette serve --port 8002 likes/full/likes.d
There are a lot of columns there that clutter up the view: I'd suggest this is a good subset (it also shows the post with most notes (likes, reblogs, and comments combined) at the top):
select rowid, id, short_url, slug, blog_name, date, timestamp,
liked_timestamp, caption, format, note_count, state, summary,
tags, type
from lines order by note_count desc limit 101
Happy excavating!
2 notes
·
View notes
LibreVastServitor computing stack designs 1/?
Just a quick reminder here, this is a customized computing stack manifestation game where I simply write what I desire and then let the wider universe manifest it for me whatever way that means, including personal efforts of mine. Boils down from customizing and adapting research material into a actionable series of items to manifest.
Ashur dream specifications
(mid-tower personal workstation computer)
2+ 2560x1440p monitors (one vertical, one or more horizontal)
Intel Core i5-4690 @ 3.5 GHz with its 4-cores (hoping forward to upgrade the RISC-V + OpenPOWER like processor for something decent with ~12-cores & much more open design) central processor unit
32GB of RAM
Some recent mid-range AMD GPU
64 GB Linux swap partition (mostly for virtual machines and RAMdisk partitions)
4TB+ SSD storage
Bluray burner
Floppy drive
Cassette / datasette drive
Themed GRUB bootloader
S6 init system
Arch-like package manager and software ecosystem
Customized alternative Linux kernel between Linux Libre & Zen kernel ( XanMod + Liquorix )
ZealOS, Parade, OpenBSD, OpenIndiana...
CLADO, DIS, Venera, Perseus, Maskoch, Synod, Monad, Valenz, Constans?
KDE Plasma with Liquid shell as desktop environment, complete with custom ricing, dot files & all the KDE desktop environment utilities;
Bash + Fish, Tmux, Astro-Neo-Vim with LSP, Emacs, LibreOffice Suite, Calligra, Bottles, Wine, WineTricks, QGIS, Firefox, LibreWolf, Dolphin, Konsole, Inkscape, Karbon, OpenStreetMap, GPlates, GProjector, Itch, Steam, GOG Galaxy, Lutris, Cyberpunk 2077, Ken Silverman's, FreeBASIC, Common Lisp, Godot + Qodot, VLC, MPV, .ogg / .ogv media player, musical tracker, 'Landchad.net', Brasero, K3B, FloppyFormatter, LibreCAD, AutoCAD, Blender, Kate, Qt, Nim, MUSL, C compiler, assembly monitor, HxD debugger, Rust, Swift, Kotlin, F#, C#, GNU make, NASM, Sweet Home 3D, some digital audio workstation software, Audacious, FFMPEG, Wayland, Morevna OpenToonz, some HTTP(S) web server suite, MongoDB, Hexo, Netlify CMS, RSS feed reader + generator, Pomodoro, Calendar, timely Tracker, Notion-like service, Tape, Gollum, some level editors, FreeCiv, The Sims 2, SimCity 4, Quake 1, Doom 1 & Doom 2, Markdown / Argdown, Konqueror, some WYSISYG rich media editor, some Raycaster engine, Daggerfall Unity, Portal 2, Source (1 & 2) Engine modding, some VirtualTableTop software, some remote desktop control software like VNC, OpenSSH, some distributed share storage software, Trenchbroom, StableDiffusionXL, ChatGPT open source alternative, DAO, Krita, GIMP, G'MIC & its plugins, PaintDotNet, CataclysmDDA, CataclysmBDA, Evennia, Python 3, Firefox for KDE (Developer Edition), Perl, PHP, MariaDB, lighttpd, Apache, Nginx, Themix Oomox GTK+ theme editor, Falkon, ...
Custom shell scripts, interactive REPL programming languages, some GUI programs, command aliases and dot file configurations;
?
Venera (computation "deque" project)
Original components:
RISC-V + OpenPOWER = LibreVast (tribble word-based open hardware architecture designed for daily use & tinkering developer purposes)
Tropix + OGAS = Nucleus (optimized distributed processing micro-kernel, like 'Inferno' & 'Plan9')
RedSeaFS + Parade = CLADOgram (direct-access rich media agentive filesystem & file server suite)
KDE + POSIX-compilant CDE = VUE (lightweight desktop environment with profound customization options)
CommonLisp w/ CLOS + Nim = Pan-Lisp (both low-level and high-level REPL programming language)
Existing components:
Fish, Tmux, Vim, Konsole, Flatpak, Git
KDE Plasma w/ Liquid shell alternative
Konqueror, LibreWolf
GIMP w/ G'MIC & Krita w/ G'MIC
Hexo (flat blog self-hosting web server), MariaDB, "Landchad.net" stuff
QEMU, Wine, Wine-tricks, Proton, Bottles, Lutris
Trenchbroom, Godot w/ Qodot
Kate, KDevelop, Okteta, Mousepad, Notepadqq
[...]
3 notes
·
View notes
Open Source Javascript Tables
Recently I came across a bunch of expensive plugins for Wordpress. They were focused on tables. I get it, it takes time to build plugins, but they were expensive.
So I wondered whether there are Javascript tools for tables. You know, to display on my website.
And guess what, I found a few that I'd like to try out someday.
ISOTOPE by MetaFizzy
While checking out websites that were using WPDataTables, I came across City Colleges of Chicago website. They were using this for their course search page. You can have build filters to easily identify the relevant data from the non-relevant.
DataTables.net
This tool is free to use, and converts HTML Tables to searchable ones. However, the HTML table must have the tags, <thead> and <tbody>. The <table> must also be identified as having an "id", and a class of "display". I think this has a lot of potential, but you need to convert data to HTML format. That would mean CSV to HTML, and ensuring that the HTML has the proper tagging.
GRID.JS
This is a really nice Javascript library that I'd like to use, if I could. It's free and open source. I think there are some premium tutorials written by an Indian guy, on how to incorporate it into a Wordpress site. The data should be in JSON format. It would load faster then JQuery DataTables. I think I like this a lot.
Tabulator
An open source project with many plugins, it seems that it's great for use in a Wordpress site. But I'm not sure how many rows it can handle.
FooTable
A useful Javascript table framework built for Bootstrap. It's not the only one, but it was a good effort. My concern for this is how to input the data and how many rows it can handle.
Bootstrap Table
This is the other one that was built with Bootstrap in mind. Now that Elon Musk has taken over Twitter, I'm not sure about the future of Twitter Bootstrap. But it's a good project that has given a lot of help to web developers. I hope that Bootstrap survives Elon's reign.
Datasette
Not sure if Datasette is a Javascript framework that I could use easily, but it powers certain high profile websites, like the LAION database website. That is the data set that powers Stable Diffusion, so it's a pretty big deal. Might be the most complex, yet high impact, tool of the lot.
Conclusion
These are just a few of the Javascript libraries that I could find for tables. Ideally, I would one day like to run my own instance of an online spreadsheet software. And use that to embed data into my website. But it may be just a dream. Meanwhile do check out my main site.
0 notes