digitillogic-blog-blog - Tumblr blog

[digitillogic-blog-blog]

Blog URL migration with GAE redirect with nginx rewrite

I am in the process of migrating a number of applications I used to run on my VPS onto the Google Apps framework, mostly because I am lazy. One such application was this blog. Creating and copying the blog entries across was simple enough, but I also wanted to incorporate my static pages. This would require configuring www.die.net.au and die.net.au to make to my home page on the blog. As an extra step I wanted my VPS server to have as little to do as possible. My plan was to get www.die.net.au hosted in the Google Application Engine service then simple do a rewrite for die.net.au on my VPS. I created a very simple Go app for GAE mapped to www.die.net.au that redirected to my blogs home page.

package page_redirect import ( "http" ) func init() { http.HandleFunc("/", root) } func root(w http.ResponseWriter, r *http.Request) { http.Redirect(w, r, "http://blog.die.net.au/p/home.html", 302) }

All my VPS needed to do now was redirect web requests for die.net.au to www.die.net.au. This was easily achieved by added a simple server configuration to my nginx front end.

server { listen 80; server_name die.net.au; rewrite ^/(.*) http://www.die.net.au permanent; }

And that's it. My blog has now been fully integrated into Google for them to spy on with my blessings.

[digitillogic-blog-blog]

YUM: was supposed to be removed but is not!

I was trying to update a Fedora15 server and got the dreaded error. The top recommended suggesting was the remove, rebuild and clean your yum database. But this wasn't working for me. A verbose run of the yum update didn't reveal much more of interest, except.

Warning: scriptlet or other non-fatal errors occurred during transaction.

Nothing in the logs either. My next course of action was to manually install the RPM files. These are usually located under /var/cache/yum/ but there was nothing there. Why weren't the RPM files getting stored? Then it hit me. The presto plugin uses Delta-RPMs to only download the sections of an RPM that differ from your installed ones. It's quite a nifty feature. But to get the RPM files I needed to disable it. I think you can see where this is going. Disabling presto and delta-rpms caused yum update to work again. This may have been due to RPM itself being part of the updates. I'm going to re-enable presto again and see what happens next update.

[digitillogic-blog-blog]

Dialog Windows in XMonad

The current implementation for dialog detection only treats transient windows of the parent window as dialogs, and ignores EWMH compliant applications that set the window type to _NET_WM_WINDOW_TYPE_DIALOG The following code detects EMWH dialog windows:

import XMonad import Graphics.X11.Xlib.Extras -- | Float all dialog windows manageDialogs :: ManageHook manageDialogs = checkDialog --> doFloat -- | Check if window is DIALOG window checkDialog :: Query Bool checkDialog = ask >>= \w -> liftX $ do a <- getAtom "_NET_WM_WINDOW_TYPE" dialog <- getAtom "_NET_WM_WINDOW_TYPE_DIALOG" mbr <- getProp a w case mbr of Just [r] -> return $ elem (fromIntegral r) [dialog] _ -> return False -- | Helper to read a property getProp :: Atom -> Window -> X (Maybe [CLong]) getProp a w = withDisplay $ \dpy -> io $ getWindowProperty32 dpy a w

It's then just a matter of adding the manageDialogs hook to your manageHooks, here is my manageHooks:

myManageHook :: ManageHook myManageHook = composeAll [ manageDocks , manageDialogs , manageHook defaultConfig ]

[digitillogic-blog-blog]

Introducing the Yocto Package Manager (yPkg)

Many times I've needed a newer version of a library or a tool not provided on the computer labs at university. Rather than pester the admin, I would generally install software into $HOME/.local and go on my merry way. This was fine until I started hitting my quota and had to manually clean out my installed software. So I decided to write a simple package manager with a minimal set of features to allow me to more easily install and remove software, a micro package manager. I was going to call it µPkg, but the aim it to have the smallest package manager possible and the smallest official SI prefix is y (yocto). Hence the name yPkg Features it does not have:

Build scripts

Package upgrades

Regression tests

Checksums

You get the software yourself, you validate it yourself, you build it your self. All it does it create a simple environment to build in, creates a package and generates a footprint. The footprint can then be later used to remove the package. Here is an example of how to use it:

~ $ ypkg init hello (ypkg) ~/.ypkg/hello $ wget http://ftp.gnu.org/gnu/hello/hello-2.4.tar.gz (ypkg) ~/.ypkg/hello $ tar zxf hello-2.4.tar.gz (ypkg) ~/.ypkg/hello $ cd hello-2.4 (ypkg) ~/.ypkg/hello $ ./configure --prefix=$HOME/.local ... (ypkg) ~/.ypkg/hello $ make ... (ypkg) ~/.ypkg/hello $ make DESTDIR=$PWD/../pkg install ... (ypkg) ~/.ypkg/hello $ exit ~ $ ypkg build hello ~ $ ypkg gen_footprint hello ~ $ ypkg install hello ~ $ ypkg list /home/lucas/.ypkg/hello ~ $ cat ~/.ypkg/hello.footprint /home/lucas/.local/usr/bin/hello ... ~ $ ypkg uninstall hello

The source is available here http://git.die.net.au/cgit/ypkg

[digitillogic-blog-blog]

Upgrading to CRUX 2.6 on Slicehost

I'm one of those sadists that like to upgrade CRUX releases from ports rather than release packages. I've been testing the CRUX 2.6 branch on my home PC for some time now without any problems, so I've decided to upgrade my VPS as I am no longer maintaining my multilib branch of 2.5.

libarchive and pkgutils

First get yourself a copy of the 2.6 core repo, either from git or rsync. If you rebuilt libarchive while lzma was installed, you will need to remove lzma and rebuild/install libarchive manually. You will then need to install xz then upgrade libarchive. Follow this by upgrading pkgutils and ports. Now you can upgrade your ports and begin rebuilding the toolchain.

Rebuilding the Toolchain

The only issue I encountered here was that the Slicehost kernel is too old to be built with the new glibc. This is easily fixed by changing one line in core/glibc/Pkgfile, change

--enable-kernel=2.6.27

to,

--enable-kernel=2.6.24

e2fsprogs and util-linux-ng

libuuid and libblkid has moved from e2fsprogs to util-linux-ng. You will need to remove e2fsprogs, update util-linux-ng then reinstall e2fsprogs. Everything else should go fine after that. It's simply a matter of doing a sysup, checking for missing libraries via revdep and rebuilding where appropriate.

[digitillogic-blog-blog]

BOM Weather Observations Available in JSON

When I discovered that BOM weather observations were available in JSON, I had to have a play. Initially I tried doing something with jQuery, but cross site request are only supported through JSONP. Then I had a better idea, let my server collect and cache the observations, this then allowed me to process the observations on the server side and do cool things like make pretty graphs. After an hour or so of mucking around, I now have a weather page. The graph is generated using matplotlib and the data is smoothed using numpy. In the future I might play with this more, allowing more observation stations, and add support for other BOM products. But I suppose I better get back to studying for exams.

[digitillogic-blog-blog]

Doomsday Averted

Ok, first of all. Yes I know I should have had backups, but I didn't. My file server, hidden away in the shed, had been randomly rebooting lately. All hardware tests checked out OK, but it always recovered after a reboot and only happened every other week, so I just ignored it. Until FreeBSD-7.2 hit an official release, and it rebooted every time I ran csup. I pulled it out of the shed and hooked it up to the KVM switch in the study so I could investigate what was happening. It took some time to replicated, but when it did, it was immediately obvious what was wrong; "ufs_dirbad", followed by a panic. The file system had somehow become corrupted, so immediately I unmount the drive and naively run fsck -y and remount it. Disaster!!! The file system was empty, years of photos, backups of our DVD collection (some of which the original DVDs no longer exist, thanks to the kids), the entire music collection. In all 400Gb of data, gone. Needless to say, when the wife found out, she was a little freaked out. I tried to calm her down, explaining that the data was still there, but just not accessible under conventional means. It seemed to work, I was putting all bets on that I'd lost a few inode references, and praying that I hadn't managed scramble the data along with it. 4 years working as a technician has taught me how to pretend to know what you're doing, fake it till you make it and that if you google hard enough you'll eventually find someone with the same problem, or even just a clue to fix it yourself. After about an hour, I find the clue I was looking for, fsdb.

Fsdb opens fsname (usually a raw disk partition) and runs a command loop allowing manipulation of the file system's inode data.

I began manually inspecting each individual inode, and resigned to the fact that this was a 500Gb hard drive and was going to possibly take a very long time. Fortunately, when I hit inode 128, there was my filesystem. I linked back onto the root inode, remounted the filesystem and there it was, all my precious media, all my music, all my anime, all my DVDs, my entire Console and MAME ROM collection. I suppose the family photos and the wedding video were important too. I immediately scrounged together some spare hard drives, dumped the data and reformatted the disk. Now to convince the wife to let me by some more hard drives ;)

[digitillogic-blog-blog]

Why you shouldn't skate in shorts

The bottom rash is about 12cm in diameter. It's been a few months since I skated at my local skate park, preferring to go to Uralla or Walcha because they have better parks. As a result I wasn't used to the flow of the ramps anymore. On top of that I've bent my bearing spacers, so my skates are rolling really weirdly. Anyway, tried to do a basic 270 hip transfer, but instead I landed on my leg. Of course the wife doesn't care as it's a "self inflicted injury". This is probably the skating gods punishing me for neglecting my local skate park, even if it is awful.

[digitillogic-blog-blog]

Extending Admin Forms in Django

So I've decided to start a blog, because well it seems no one will take you seriously if you don't have one these days for some reason. I've been using the basic-apps blog engine as I've used this before to implement a simple news section for a client. I like it because it provides the basic functionality of a blog engine, and leaves you free to implement all the bells and whistles yourself. One feature I wanted to add was a WYSIWYG editor for the blog posts and flatpages. I found a neat little tutorial on using markItUp! for the textarea inputs. The problem is that this tutorial was written with the purpose of adding this as a core feature to an application, not extending an existing one. Thankfully, with newforms-admin, this is really easy in Django. First we need to define the markItUp! widget (as described in the tutorial):

from django import forms class MarkItUpWidget(forms.Textarea): class Media: js = ( 'js/jquery.js', 'js/markitup/jquery.markitup.pack.js', 'js/markitup/sets/markdown/set.js', 'js/init_markitup.js', ) css = { 'screen': ( 'js/markitup/skins/simple/style.css', 'js/markitup/sets/markdown/style.css', ) }

I use a package in my projects called "local", which I use for local modifications to 3rd part and contrib applications. I added this to local/admin.py

from django.contrib import admin from django.contrib.flatpages.admin import FlatpageForm, FlatPageAdmin from django.contrib.flatpages.models import FlatPage class LocalFlatPageForm(FlatpageForm): content = forms.CharField(widget=MarkItUpWidget) class LocalFlatPageAdmin(FlatPageAdmin): form = LocalFlatPageForm admin.site.unregister(FlatPage) admin.site.register(FlatPage, LocalFlatPageAdmin)

Then add this to the root urls.py

admin.autodiscover() import local.admin

It's that simple, my flatpages now use the markItUp! editor. NOTE: I am using markdown to render flatpages now, if you do the same you will need to rewrite your flatpage templates.

[digitillogic-blog-blog]

Postrocking

Whilst idling on #crux, someone posted a link to heavy metal audio stream. Not being that into metal these days, I decided to see if there were any post-rock based streams out there. Just my luck I stumbled across one that was launched just days ago. Postrocking was launched on the 23rd of April. It offers news about new post-rock bands, album releases and of course an audio stream. As someone who is always looking for new post-rock bands to enjoy, I dare say I will be keeping a close eye on this site.

mplayer http://89.149.242.30:8000/