davidschulz

[davidschulz]

A Journey from ADSL to NBN… and Back?

We got the second class tickets to the NBN with fibre only coming as far as the little green cabinet in our street.  Here’s what I’ve learnt from tracking our internet speed across a couple of years.

This graph shows the internet speed tests done with SpeedTest.  You’ll be able to pick where we switched across to the NBN connection.  

I suspect we were the first people in our street to move across.  I do quite a lot of online meetings and web based software demonstration from our home office.  So we initially signed up at the top tier plan to allegedly give us 100/40* Mbps (Down/Up).  OK, we never got that but with speeds peaking in the mid 70s I was a pretty happy camper.  

After a few months as more people switched across and we were sharing the fibre, I noticed speeds dropping below 50Mbps.  Why pay for 100 if you are only getting 50?  At that point I scaled back our plan to be a 50/20.

Over recent weeks the speed of the internet had slowed again noticeably which I was correlating with the frequency of seeing a ute parked at the cabinet switching across more people onto the shared fibre.  Sure enough the speed tests confirm that our internet speeds have dropped back below 25 Mbps.

I will need to again downgrade our plan to reflect the service that can actually be delivered.  Again, why pay for 50 when you are not even getting 25?

If I hear another politician - I’m looking at you Peter Dutton - talk about how “We didn't need fibre to the premise because no one wants the top tiers anyway.” I’ll…. Well words fail me.

If the trend continues, we’ll be looking at speeds on par with what we had on ADSL.  The way it is looking to me, we’ll have made an investment of $49B (and counting) for a zero net gain.  Oh the glory of Australian telecommunications policy!

* When checking the NBN site today I notice they no longer specify the target upload speeds so I am working from memory for the upload speeds quoted.

[davidschulz]

Daniel Ricciardo brilliantly won the race with a measured, classy and fast drive to once again be the man who took the crumbs from the Mercedes table. His team-mate Sebastian Vettel would have at least liked to have been in a position to run into him because that would have meant he was quick enough, yet still easy on his tyres like the amiable Australian.

[davidschulz]

Weather Nerd

Pretty sure this is official now.  As my sister observed... blood will run true.  I have just invested an inordinate amount of time in getting a wireless weather station hooked up to our home server and publishing data to the internet of things.

For those of you also with a QNAP and thinking it maybe can;t be done.  Here's how I went about it and what's been successful to date.

HARDWARE

I purchased a 'Wireless Weather Sation' from Jaycar (http://www.jaycar.com.au/productView.asp?ID=XC0348). This station is sold under a variety of names but is generally referred to in most cases as a Fine offset WH1080. The 1080 has a USB port that allows it to be connected to a computer to download the data from the station. It is supplied with some Windows software but I don't have a PC. The best solution for me was to connect it to my QNAP NAS. This acts as my home 'server' and is always on, making it the best place to connect the weather station to.

DEAD ENDS

The QNAP uses a highly customised version of Linux with a graphical user interface to configure and operate it. This is the main selling point for me for whom the command line is a foreign country. The QNAP system is extensible with its own package management system (QPKGs) which makes adding additional software literally as simple as a single click. This is my kind of Linux! So first port of call was to find weather station software that was available as a QPKG. I downloaded and installed wfrog which installed OK. It relied on another package called pywws to actually get the data with wfrog primarily acting as a presentation layer (from what I understand). Despite great support from the author of the packages I never could get pywws working.

SOFTWARE

The user community for both wfrog and pywws is quite small. A number of people pointed me at weewx which has a comparatively much larger support base and I knew people running this. I decided weewx was the best bet for the software part of my solution.

AND THE BAD NEWS...

The downside of this decision was there is no QPKG. The worse news was that a number of posts on the weewx user forums said that because the QNAP was such a specialised Linux environment it would simply not be possible to get weewx working on it. The 'smart money' seemed to be to grab a Raspberry Pi or a Beaglebone Black and run up a standard Linux system to act as a dedicated controller for the weather station. I've been itching for an excuse to buy one of these mini marvels for a while so I wasn't averse to that. But I really like the idea of getting some more value from the QNAP system. I had it, so what was the harm in having a go at getting it running?

THE JOURNEY

As Linux novice fearful of the command line, I found the weewx documentation to be really quite good. Rather than the usual instructions laden with assumed knowledge, the document contained clear commentary on what needed to be achieved in each step, along with the actual commands required with variants for a range of Linux distributions - but sadly not for the QNAP. But it did give me a really good idea of what I needed to achieve to get this going.

QNAP AND QPKGS

The first steps were to turn on embedded capabilities in the QNAP and to grab the things that were available as QPKGs. I opted to use the MySQL database for my weather data. The default is SQLite which doesn't require any setup. I kinda liked the idea of being able to use PHPMyAdmin to look at the data directly so opted for MySQL. The weewx system is a Python application so naturally enough Python is required. All these things are able to be installed and setup from the QNAP graphical interface. Yay!

MySQL

Enabling MySQL is as simple as logging into the QNAP QTS Desktop, opening the Control Panel, navigating to Applications and selecting MySQL Server. Click the checkbox to enable MySQL Server.

PHPMyAdmin

PHPMyAdmin gives you a graphical UI to manage your databases, permissions and you can execute queries if you can manage the odd “select * from...” In the QTS Desktop, open the App Centre and find the PHPMyAdmin QPKG and click “Add to QTS'. The package will download and install.

Python

Weewx requires Python. As at the time of writing, Python 3 is not supported. Similarly, the Python QPKG installs version 2.7 so this aligns nicely. Find the Python QPKG and click 'Add to QTS'. The package will download and install.

PRE-PREREQUISITES

The generally helpful weewx documentation lists the prerequisites including a number of Python modules. In practice I found a couple of these modules would not install without a compiler. So I first needed to install the gcc compiler before these would install successfully.

GCC Module

A compiler is needed to install the Cheetah and PIL modules (from memory). The way I installed he GCC compiler was from an IPKG. There is a PQKG that allows you to use IPKGs. The first step is to install the Optware QPKG. Open the App Centre on the QTS Desktop and find the Optware package and click 'Add to QTS'. It will download and install.

Click the Optware IPKG icon on the QTS Desktop. This will open the IPKG system. In the “Filter' box type 'gcc' and click Submit. To the left of the gcc IPKG you will see the option to 'Install' Click this to install the compiler.

WE"RE NOT IN KANSAS ANYMORE DOROTHY

So here ends the part where I was in my comfort zone. The next steps are executed from the command line. I used a terminal window on OS X to connect to the QNAP. When your terminal window opens, connect to the QNAP using the following command ssh admin@nnn.nnn.nnn.nnn where the last part is the IP address of the QNAP. If you don't know this you can find the IP address from the Network page in System Settings of the control Panel. You will be prompted for the admin password and then you will see the unnerving, vacuous blinking of the command line prompt “$”; telling you nothing, taking you nowhere. When you are done you can type 'exit' to disconnect.

PREREQUISITES

To PIP or not to PIP

Python has a number of package management system designed to make it simple to download and install Python modules. The most current method from what I read is PIP. I used these nice instructions to get PIP setup: http://rdkevinchien.blogspot.com.au/2013/08/how-to-setup-my-python-developing.html As it turned out I was never able to get modules installed using PIP. PIP enforces an SSL connection for downloading the package which is a good thing. The self issued certificate on the QNAP meant the download always failed. It is beyond me to sort out the certificate thing and I couldn't find a pip option or switch to override the secure download. There is a nice set of instructions for purchasing and installing a certificate for your QNAP. See my references at the end. This would probably make the PIP installs works and will be bit simper if you want to go down that path. My solution was to use the commands to download and install pip as a template to download the modules manually and build and install them manually. I used the admin home folder to store the downloads into. cd /share/homes/admin I used the Python module pages to grab the URL to download each module. The Python Package Index allows you to search by module name: https://pypi.python.org/pypi For example the configobj module page is: https://pypi.python.org/pypi/configobj/5.0.5 From this page you can find the download link and copy the link address. Paste the link address into the download commands below to make it easier and avoid messing about retyping the url without making an error. I downloaded, unzipped, built and installed each of the module dependencies in turn.

Configobj

First download $ wget https://pypi.python.org/packages/source/c/configobj/configobj-5.0.5.tar.gz#md5=00af246754b4bb41a4e9c625c60dca5c --no-check-certificate Extract the archive $ tar xvfz configobj-5.0.5.tar.gz Change to the folder where the file was extracted. cd configobj-5.0.5 Build the module $ python setup.py build Install the module $ python setup.py install Repeat for the remaining dependencies listed in the weewx documentation: http://www.weewx.com/docs/setup.htm I installed the following modules:

configobj

cheetah

pil

pyusb

pyephem

NEARLY THERE

I needed just a couple of last steps to finally get things running.

The MySQL Gotcha

Walking the path less travelled had one last little bump in the road that needed to be ironed out. I was following the pip install instructions in the weewx documentation. When I tried to run weewx for the first time it failed with an error message saying mysqldb module not found. No dramas I thought - feeling quite confident have smashed all the other module installs - once I finally sorted my workaround method out. After downloading and expanding the module as expected, the build command failed with an error saying a config was not found. I had horrible premonitions about needing to manually configure a build file of some sort. Fortunately MySQL Boy had solved this particular problem. Just a path problem: http://blog.mysqlboy.com/2010/08/installing-mysqldb-python-module.html This command let me build and install the mysqldb module successfully $ export PATH=$PATH:/usr/local/mysql/bin

DOWNLOAD AND INSTALL WEEWX

Now I was able to download and install weewx as per the standard documentation. http://www.weewx.com/docs/setup.htm I found the weewx documentation explaining the weewx.conf file clear and straight forward. Similarly the documentation covering configuring the weewx outputs is generally clearly explained. When I first saw the lines of data from the weather station scrolling up the terminal window I thought it was a problem and weewx was writing to screen instead of the database. A quick peek at the tables confirmed in MySQL using the PHPMyAdmin confirmed the data was writing there. So I concluded the writing to screen was normal behaviour. It will take a few minutes but you will be able to point your browser at http://nnn.nnn.nnn.nnn/weewx and see your weather data. The nnn.nnn.nnn.nnn part is the IP address of the QNAP. It was great to see this page populated with data from my weather station! I kept the terminal window with weewx running in it open on the Mac for a few days as I wasn't game to just close it in case it stopped the process. It eventually disconnected and the web pages keep updating so it seems OK to just close the terminal window once weewx is up and running.

Apache Config and Restart

The Weewx documentation provides instructions to get weewx talking to the web server (Apache) in the case it was not installed from a package. This was certainly the case for me. These instructions needed some slight modifications to reflect the location of apache on the QNAP and a different commend to restart apache. The command in the document: cp util/apache/conf.d/weewx.conf /etc/apache2/conf.d Needs to be: cp util/apache/conf.d/weewx.conf /etc/config/apache/conf.d The equivalent commas to restart apache on the QNAP is: /etc/init.d/Qthttpd.sh restart

OUTSTANDING TASKS

I have the data coming off the weather station onto the QNAP and an elementary web page displaying our weather data. There are still quite a few tasks still to sort out.

Autorun

So far I have just started weewx from the command line. If the QNAP is restarted I will manually need to restart it. Adding things to start on boot on the QNAP is not straightforward. It looks like I'll need to declare a dummy QPKG as the recommended solution form QNAP but that looks like another journey of discovery... http://wiki.qnap.com/wiki/Running_Your_Own_Application_at_Startup

Graphs

None of the graphs on the web page are showing. I thought this may be to do with a font specified in the skin.conf and not on the QNAP. I have downloaded the required font but still not getting any plots.

Publish to Web

So far the web page is generated to the QNAP. Weewx provides a report to publish to FTP to make our weather data available outside our network. Need to configure the ftp report options to make this happen.

Publish to Weather Underground and WOW

Weewx also allows data from my weather station to be published to crowd sourced weather data systems like Weather Underground and WOW. I've setup a Weather Underground account and station. Need to configure to publish to that account.

REFERENCES AND RECOGNITION

I am nowhere near enough knowledgeable enough to have solved any of this myself. All I have done is piece together a lot of disparate threads leveraging the knowledge and contributions of people who actually know what they are doing. The following sources were absolutely vital in me nutting this out. Thanks to all the authors! PIP Setup on QNAP http://rdkevinchien.blogspot.com.au/2013/08/how-to-setup-my-python-developing.html Workaround for PIP Download Problem http://gleenders.blogspot.com.au/2014/04/install-python-pip-on-qnap-nas.html MySQLDB Build Fail http://blog.mysqlboy.com/2010/08/installing-mysqldb-python-module.html QNAP Autorun http://wiki.qnap.com/wiki/Running_Your_Own_Application_at_Startup

[davidschulz]

Phil Anderson (Peugeot) - 1981 TdF, Stage 7: Nay - Pau. Aussie cycling legend Anderson was the first non-European to wear the yellow jersey.

[davidschulz]

Sunset @ Kyoto Japan by pasuay at gmail dot com on Flickr.

[davidschulz]

[davidschulz]

Charge point built into my new desk beats the hell out of crawling under the desk and is much prettier than the previous system of a power board and extensions cord cluttering up the desk.

[davidschulz]

Microsoft's Wile E. Coyote Windows Download

Life as a Mac user presents precious few compatibility challenges these days. The move of everything to the cloud makes the hardware and operating system you are using increasingly irrelevant. The iWork suite opens MS Office documents and can save back to that format. If you are one of those odd souls that thinks Word is actually a desirable place to draft content you can get MS Office for the Mac. So all in all it's never been easier to work on your Mac harmoniously with Windows based colleagues and customers.

In my world there are a few annoying reasons to still need a Windows PC. To cater for those circumstances I have relied for a number of years on running up a vestigial copy of Windows XP on VirtualBox. Over the weekend I thought I'd do the right thing and bring my virtual windows environment up to scratch.

The plan was to step up to VMWare's Fusion and do a clean install of Windows 8.1. Simple plan I thought. What a nightmare!

The first step went as smoothly as expected. The VMWare store happily took my money and allowed me to download Fusion which installed without fuss. I was lulled into a false sense of security.

VMWare was looking for an ISO image file of the Windows DVD. It seemed to identify that my MacBook Air doesn't have an optical drive. So I headed off to the Microsoft online store to purchase a copy of Windows. Let the pain begin!

I dutifully created a new Microsoft account. I tried using my Office 365 Microsoft account but that didn't seem to be the right sort of Microsoft account. After spending some time understanding the usual confusing range of product choices that Microsoft prefers, I finally elected for a full version of Windows 8.1 - Download Only. The store successfully processed the payment and I attempted to download a copy of Windows.

Apparently when you purchase a download copy of Windows, that doesn't allow you to actually download a copy of Windows.

The download instructions, which still referenced Windows 7, described a process for selecting an ISO image file but the actual download screen contains only a single option to download an .exe file. Clearly this isn't much use until I can get Windows up and running. After going around in circles on the Microsoft store and broader website and a couple of hours trolling forums and websites I gave up and called the Microsoft store support number.

A very helpful member of the Microsoft call centre staff answered my call close to midnight on Saturday. She confirmed for me the conclusion I had reached in my fruitless internet search. Microsoft aren't expecting any new Windows customers. You have to already be using Windows to be able to install Windows from the online store. She offered a refund which I gladly accepted.

Her advice to me was that I needed to contact technical support to be able to obtain a Windows ISO file. It seems strange to me to contact technical support to transact a sale but who am I to quibble. They are only open sensible hours so a wait for the work week was required. During this week I have made a couple of attempts to call technical support but each attempt fails in the transfer process from the greeters to the actual technical support people.

Failing to communicate by phone I tried the chat option. Lawrence was only able to repeat the advice I should seek a refund. The only options I could find to communicate with Microsoft by email required me to pay close to $400 for the privilege. I declined.

So after 4 days of diligently attempting to buy Windows 8.1 I have to admit defeat. Really wishing now I had taken up the 30 day free trial of Fusion before committing to the purchase! 

[davidschulz]

Progress on Office Relocation Finished soft wiring my new desk this weekend. All looks nice and clean and uncluttered. Won't last!

[davidschulz]

[davidschulz]

Now that looks like it could be fun!

Yes.

[davidschulz]

A tale of a little SOHO infrastructure upgrade

We use a little QNAP magic black box to backup our Macs over the network.  Over time it has also become a part of our internet connection, keeping copies of files we download so if we need them again we don't have to download them multiple times.  That's been quite useful with 4 iOS devices around the place and app updates arriving thick and fast.

I noticed, somewhat belatedly, the hard disk in the QNAP was almost full the other day.  I grabbed one of these Western Digital Red drives designed to be used for this sort of device to give us a bit more space.  The process seems to be to mirror the drive using Raid-1 and then transplant the new drive in to replace the old and hey presto, nothing up my sleeve, things should just pick up where you left off, just with more space to play with. 

It took a little while to format the 3Tb drive but that was nothing.  The process of Raid-1 synching over USB is on to its second day now and it is not quite 2/3 done.  

[davidschulz]

Upcoming iPhone-Compatible Sony Lens Attachment Detailed in Leaked Manual by Juli Clover, macrumors.com

Sony is craft­ing a new high-end lens sys­tem designed to work with smart­phones, accord­ing to doc­u­ments and pho­tos pub­lished by Sony Alpha Rumors. First unveiled in pho­tos last week, Sony’s lens acces­sories are actu­al­ly fully…

This is a very interesting idea...

[davidschulz]

Renai LeMay, delimiter.com.au

A senior columnist writing for Melbourne newspaper The Age has falsely claimed that the fibre technology used in Labor's NBN project had a life of only 25 years and that it could be made obsolete by "low earth orbiting satellites", in an article…

Low orbit satellites will make the NBN redundant!!!! The bullshit being reported about telecommunications policy makes me wonder whether these journos are just plain dopey (unlikely...), lazy (possible), are being asked to report on things they just plain don't understand or... (the option that worries me most) there are incentives to argue a case even if they have to lie to do so.

[davidschulz]

This might make a great style upgrade to the usual fondue pot burner. Might be a bit too hot...

[davidschulz]

Howard Davies, japantimes.co.jp

Americans have been surprised at the sharply divided views in Britain over the governance of Margaret Thatcher, a product with more appeal in export markets.

The neatness of Howard Davies' prose appeals in this Japan Times piece. "Thatcher has long been a British product with more appeal in export markets than at home."

[davidschulz]

For my bro...

Port a Bach