Updating my web presence

I made my first website on Geocities. I started my first blog when they were still called journals. I was working on social networking software before it was called that. It meant I was on Facebook, Twitter and other platforms long before most ‘normal’ people.

After being early to join the social web I was also early to leave, for the usual reasons (offline life is more fun than online life!). I hadn’t touched this blog in about 8 years, and have barely posted on Twitter for 10 years. When facebook introduced banner images, I used it to inform my facebook friends I was not actually logging in to facebook. If you wanted to reach me, you could choose only from phone, SMS, E-mail and (reluctantly) WhatsApp.

But I’m no longer so worried about internet addiction or getting misunderstood about my defenses against it. In 2020 it seems it has become normal to use social networking not at all or only in moderation, and many communities seem to (once again) welcome casual and intermittent participation. I might, just might, stop lurking around and try again to contribute something useful in the public sphere.

As a first step, I’ve cleaned up my websites and am making some hobby software projects public over on GitHub.

Image of my dipping my toes into the watter on a beach

Writing one line shell scripts with bash

If you are using ruby with bundler and Gemfiles properly, you probably know about running commands with bundle exec. However, sometimes this does not get you quite the right results, in particular if your Gemfile is not quite precise enough.

For example, I had an issue with cucumber + autotest + rails where I had both rails 3.1 and rails 3.2 apps using the same RVM. Since I was confused, and in a hurry, I figured one brute force option to unconfuse me would be to simply remove all old versions of all gems from the environment. I did just that, and thought I’d explain the process of incrementally coming up with the right shell script one liner.

First things first, let’s figure out what we have installed:

$ gem
    gem command [arguments...] [options...]
    gem list --local

Ok, I guess we want gem list:

$ gem list
*** LOCAL GEMS ***

actionmailer (3.2.2, 3.1.1)
ZenTest (4.7.0)

actionmailer is part of rails, and we can see there are two versions installed. Let’s figure out how to remove one of them…

$ gem help commands
GEM commands are:
    uninstall         Uninstall gems from the local repository
$ gem uninstall --help
Usage: gem uninstall GEMNAME [GEMNAME ...] [options]

    -I, --[no-]ignore-dependencies   Ignore dependency requirements while
    -v, --version VERSION            Specify version of gem to uninstall

Great. Let’s try it:

$ gem uninstall actionmailer -v 3.1.1

You have requested to uninstall the gem:
rails-3.1.1 depends on [actionmailer (= 3.1.1)]
If you remove this gems, one or more dependencies will not be met.
Continue with Uninstall? [Yn]  y
Successfully uninstalled actionmailer-3.1.1

Ok, so we need to have it not ask us that question. From studying the command line options, the magic switch is to add -I.

So once we have pinpointed a version to uninstall, our command becomes something like gem uninstall -I $gem_name -v $gem_version. Now we need the list of gems to do this on, so we can run that command a bunch of times.

We’ll now start building our big fancy one-line script. I tend to do this by typing the command, executing it, and then pressing the up arrow to go back in the bash history to re-edit the same command.

Looking at the gem list output again, we can see that any gem with multiple installed versions has a comma in the output, and gems with just one installed version do not. We can use grep to filter the list:

$ gem list | grep ','
actionmailer (3.2.2, 3.1.1)
sprockets (2.1.2, 2.0.3)

Great. Now we need to extract out of this just the name of the gem and the problematic version. One way of looking at the listing is as a space-seperated set of fields: gemname SPACE (version1, SPACE version2), so we can use cut to pick fields one and three:

$ gem list | grep ',' | cut -d ' ' -f 1,3
gherkin 2.5.4)
jquery-rails 2.0.1,

Wait, why does the jquery-rails line look different?

$ gem list | grep ',' | grep jquery-rails
jquery-rails (2.0.2, 2.0.1, 1.0.16)

Ok, so it has 3 versions. Really in this instance we need to pick out fields 3,4,5,… and loop over them, uninstalling all the old versions. But that’s a bit hard to do. The alternative is to just pick out field 3 anyway, and run the same command a few times. The first time will remove jquery-rails 2.0.1, and then the second time the output will become something like

jquery-rails (2.0.2, 1.0.16)

and we will remove jquery-=rails 1.0.16.

We’re almost there, but we still need to get rid of the ( and , in our output.

$ gem list | grep ',' | cut -d ' ' -f 1,3 | sed 's/,//' | sed 's/)//'
childprocess 0.2.2
rack 1.3.5

Looking nice and clean.

To run our gem uninstall command, we know we need to prefix the version with -v

$ gem list | grep ',' | cut -d ' ' -f 1,3 | sed 's/,//' | sed 's/)//' | sed 's/ / -v /'
childprocess -v 0.2.2

Ok, so now at the start of the list we want to put gem uninstall -I . We can use the regular expression ‘^’ to match the beginning of the line. We’ll need sed to evaluate our regular expressions…

$ gem list | grep ',' | cut -d ' ' -f 1,3 | sed 's/,//' | sed 's/)//' | sed 's/ / -v /' | sed -r 's/^/gem uninstall -I/'
sed: illegal option -- r
usage: sed script [-Ealn] [-i extension] [file ...]
       sed [-Ealn] [-i extension] [-e script] ... [-f script_file] ... [file ...]

Ugh. -r is the switch used in the GNU version of sed. I’m on Mac OS X which comes with BSD sed, which uses -E.

$ gem list | grep ',' | cut -d ' ' -f 1,3 | sed 's/,//' | sed 's/)//' | sed 's/ / -v /' | sed -E 's/^/gem uninstall -I /'
gem uninstall -I childprocess -v 0.2.2

Ok. That looks like its the list of commands that we want to run. Since the next step will be the big one, before we actually run all the commands, let’s check that we can do so safely. A nice trick is to echo out the commands.

$ gem list | grep ',' | cut -d ' ' -f 1,3 | sed 's/,//' | sed 's/)//' | sed 's/ / -v /' | sed -E 's/^/echo gem uninstall -I /' | sh
gem uninstall -I childprocess -v 0.2.2

Ok, so evaluating through sh works. Let’s remove the echo:

$ gem list | grep ',' | cut -d ' ' -f 1,3 | sed 's/,//' | sed 's/)//' | sed 's/ / -v /' | sed -E 's/^/gem uninstall -I /' | sh
Successfully uninstalled childprocess-0.2.2
Removing rails
Successfully uninstalled rails-3.1.1

I have no idea why that rails gem gets the extra line of output. But it looks like it all went ok. Let’s remove the ‘sh’ again and check:

$ gem list | grep ',' | cut -d ' ' -f 1,3 | sed 's/,//' | sed 's/)//' | sed 's/ / -v /' | sed -E 's/^/gem uninstall /'
gem uninstall jquery-rails -v 1.0.16

Oh, that’s right, the jquery-rails gem had two versions. Let’s uninstall that, then. We press the up arrow twice to get back the command line that ends with | sh, and run that. Great: we’re all done!

Let’s look at the final command again:

gem list | grep ',' | cut -d ' ' -f 1,3 | sed 's/,//' | sed 's/)//' | sed 's/ / -v /' | sed -E 's/^/gem uninstall -I /' | sh
  1. gem list shows us the locally installed gems
  2. | grep ',' limits that output to lines with a comma in it, that is, gems with multiple versions installed
  3. | cut -d ' ' -f 1,3 splits the remaining lines by spaces, then picks fields one and three
  4. | sed 's/,//' removes all , from the output
  5. | sed 's/)//' removes all ) from the output
  6. | sed 's/ / -v /' replaces the (one remaining) space with -v
  7. | sed -E 's/^/gem uninstall -I /' puts gem uninstall -I at the start of every line
  8. | sh evaluates all the lines in the output as commands

Note that, as a reusable program or installable shell script, this command really isn’t good enough. For example:

  • It does not check the exit codes / statuses of the commands ran, instead it just assumes they run successfully
  • It assumes the output of gem list will always match our expectations (for example, that the output header does not have a comma in it, or that gem names or versions cannot contain space, comma, or -- this may be true but I wouldn't know for sure)
  • It assumes that -I is the only switch needed to prevent gem list from ever asking us questions

The best approach for a reusable command would probably be to write a script in ruby that used the RubyGems API. However, that would be much more work than writing our one-liner, which is “safe enough” for this use-once approach.

(For the record, this didn’t solve my rails + cucumber woes. Instead, it turns out I had run bundle install --without test at some point, and bundler remembers the --without. So I needed rm Gemfile.lock; bundle install --without ''.)

Bye bye, London!

My last project at the BBC is finished, as is the related contract. My worldly possessions are in boxes and being shipped. The lease on my London flat has ended. A goodbye party (how unlike me!) was had. I’m now at my parents, sleeping, reading, playing with the dogs, resting.

I don’t think I’ll miss London very much (too big and busy, too expensive, takes too long to get out of), but I will definitely miss all the people a lot. Those I worked with and those I partied with, but especially those I just plain hung out with. I think my fondest memories will be of having a beer in the park on a sunny afternoon, or having fancy lunch at a notting hill restaurant, or strolling along south bank amidst the tourists.

What’s next? I have my sights set on Amsterdam, to work and maybe to live. I’m looking at buying a car which I’ve actually never done before. I’m also looking for interesting work. Talking to a few interesting folks already, it seems like Amsterdam is filled with startups these days 🙂

Oracle starts at $198/year or $0.16/hour

I learned that spacewalk requires oracle the other day. To an open source weenie that sounds bad, but how much of a problem is it in practice?

Oracle XE is free, allowing up to 4GB of data, which seems like it would last you about 2 years of running spacewalk with a few thousand packages and a few hundred machines.

Oracle One costs $180 per user with a 5 user minimum, installable on one machine with two sockets (so up to 12 core for an AMD opteron). Including one year of support that adds up to about $1000.

But that’s a perpetual license. A 1-year oracle license is $36 per user, so adding in support, for one year or oracle on one machine you pay just shy of $200.

The next upgrade after oracle one is oracle standard, which costs $350 per user perpetual, or $70 per user per year.

So to get started with oracle you can use Oracle XE. When you run out of space you buy a decent database server machine and a $200 oracle license, allowing for 4 middleware nodes and one administrator. You have to pay that $200 once a year. Or you can run Oracle One in the cloud with Amazon RDS from $0.16 per hour.

When you need a resilient set up you will need oracle standard so you can cluster with Oracle RAC. You’ll need a license for each server (which can now have 4 sockets so up to 24 core), so your cost increases to 2 * 5 * 70 = $700/year.

You can also deploy oracle standard on Amazon RDS, where you’ll pay from $0.11 per hour per VM in addition to your license. Though it seems like at the moment, amazon RDS replication doesn’t work for oracle, so there’s probably no point in using that. Instead, you’ll have to switch to creating your own EBS-backed AMIs and installing oracle into them (or, perhaps, use an oracle-provided AMI).

I can find nothing that states there’s a limit to Oracle RAC scale with oracle standard. So you could potentially build big, BIG clusters this way.

$350/server/year is not exactly free of course, but this actually gets you really good clustering. That bit of capex gets you the ease of deployment and use of Oracle RAC. For some use cases, it’s probably easily the cheapest option if you take into account the opex of doing anything else. For example if Oracle Locator, the subset of Oracle Spatial that’s available in the standard edition, is good enough for you, this seems easier than figuring out your own clustered PostGIS setup. Postgres or mysql replication would’ve been just fine for spacewalk though.

Oracle Enterprise has additional features that aren’t really relevant for most scale-out (webby) stuff: using more than 4 sockets, some of the advanced administration/audit/backup tools, advanced security features, oracle spatial, compression, table partitioning, and some other bits. Enterprise still costs about a gazillion dollars.

We have to erase this idea that “oracle is ridiculously expensive” out of our brains. It just isn’t true anymore: there’s some very reasonable oracle database offerings for reasonable prices. Instead, the idea should now be “oracle enterprise is ridiculously expensive”. For things like spacewalk, where you don’t need the advanced stuff, it seems oracle is now almost always a very reasonable backend that probably can have a competitive TCO.

Creating a minimal Ubuntu Server VM

The other day I documented a kickstart process for creating base CentOS VM images. I tried but failed to get ubuntu installed using kickstart. Rather than learn the normal ubuntu process I figured I’d install manually. Beyond that, the approach is basically the same:

  • Download ubuntu server, I chose the 64-bit LTS iso (10.04).
  • create new empty VM called vanilla-ubuntu.
  • Resize the disk image to 10GB.
  • set RAM to 256MB.
  • disable just about all options including audio support, usb support, printer support, file sharing, etc.
  • set to NAT networking.
  • attach ubuntu iso to cdrom drive.
  • Accept all installer defaults, but select OpenSSH server as additional system to install
  • Log in as created user
  • run sudo passwd root to set the root password
  • sudo apt-get update && sudo apt-get -y dist-upgrade
  • to work around this bug, sudo rm /etc/motd.tail
  • sudo shutdown -h now.
  • open the settings pane, open the hard disk panel, and select “clean up disk”.
  • disconnect and remove the floppy drive. Disconnect the DVD drive.
  • (optional), open the directory containing the VM location in the terminal and mv *.vmwarevm/* . && rm -r *.vmwarevm. When you next open the VM (you will need to use File > Open), select “I moved it”.
  • Make a backup copy of the virtual machine files, this is your clean VMWare-independent base VM.
  • Start the virtual machine.
  • in the Fusion menu, select Virtual Machine > Install VMWare tools, and install the tools:
    mkdir /mnt/cdrom
    mount /dev/cdrom /mnt/cdrom
    cd /tmp
    tar zxf /mnt/cdrom/VMwareTools*.tar.gz
    ./ --default
    rm -Rf /tmp/vmware-tools-distrib
  • Shut down the virtual machine
  • Make a backup copy of the virtual machine files, this is your clean VMWare-ready base VM.

One thing that’s easy to notice is how lightning fast ubuntu starts up. The other thing to notice is just how old CentOS / RHEL 5 really is, and IMNSHO how overdue RHEL 6 was (and CentOS 6 is):

Ubuntu 10.04 LTS CentOS 5.6
# uname -r
# python -V
Python 2.6.5
# perl -v
This is perl, v5.10.1 ...
# apt-cache show ruby1.9.1 | grep Version
# uname -r
# python -V
Python 2.4.3
# perl -v
This is perl, v5.8.8 ...
# yum info ruby.x86_64 | grep Version
Version    : 1.8.5

Installing spacewalk

Spacewalk is the software behind Red Hat Network. I’ve been meaning to take a closer look for a while now. With a new release out that’s got improved cobbler integration I decided it was time to take the plunge. Having a shiny web GUI on top of cobbler seems like it could be rather neat.

Getting spacewalk installed is a bit of work since it (a) is huge, (b) needs an oracle database. But the published installation instructions worked fine for me:

Get a new Centos VM: ./ spacewalk. Note that Oracle express is 32 bit-only, so I decided to just do everything as a 32 bit install. Change the VM settings to use 1.5GB ram which seems like it is just about enough for development/demo purposes (specified minimum is 2GB).

Now, follow spacewalk instructions to set up Oracle Express:

$ sudo bash
# cat >>/etc/hosts <>/etc/hosts <   oracle-instantclient11.2-basic- \
>   oracle-instantclient11.2-sqlplus-

# export SWREPO=
# export EPEL=
# rpm -Uvh $SWREPO/RHEL/5/i386/spacewalk-repo-1.4-1.el5.noarch.rpm
# rpm -Uvh $EPEL/5/i386/epel-release-5-4.noarch.rpm
# rpm -Uvh $SWREPO//RHEL/5/i386/spacewalk-client-repo-1.4-1.el5.noarch.rpm

# yum -y install oracle-lib-compat

# /etc/init.d/oracle-xe configure
HTTP port for Oracle Application Express: 9055
Database listener port: 1521
Password for SYS/SYSTEM: changeme12
Start at boot: y
# sqlplus 'sys/@//localhost/XE as sysdba'
SQL> create user spacewalk identified by spacewalk default tablespace users;
SQL> grant dba to spacewalk;
SQL> quit

# sqlplus spacewalk/spacewalk@//localhost/XE
SQL> alter system set processes = 400 scope=spfile; 
SQL> alter system set "_optimizer_filter_pred_pullup"=false scope=spfile; 
SQL> alter system set "_optimizer_cost_based_transformation"=off scope=spfile; 
SQL> quit  
# /sbin/service oracle-xe restart

Well that wasn’t so bad. I seem to recall installing oracle as being much harder and taking at least a day or two if you know what you’re doing. Maybe it’s just that the XE edition is so much easier, or maybe Oracle have just learned a trick or two in the 5? (wow!) years since I last touched it.

Now we’re ready to install spacewalk. The right yum repo configs were already imported, above, so we don’t need to do that again.

# yum install curl
# curl -o /tmp/gpg-key
# mv /tmp/gpg-key /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
# yum install spacewalk-oracle
Transaction Summary
Install     287 Package(s)
Upgrade       0 Package(s)

Total download size: 198 M

One of these packages results in the enabling of SELINUX. Let’s disable it again:

# echo 0 > /selinux/enforce
# vi /etc/selinux/config # change to SELINUX=disabled
# reboot

And now we can configure:

spacewalk-setup --disconnected
Database service name (SID)? XE
Username? spacewalk
Password? spacewalk
Admin Email Address? root@localhost
Should setup configure apache's default ssl server
    for you (saves original ssl.conf) [Y]? y
CA certificate password? spacewalk
Re-enter CA certificate password? spacewalk
Organization? spacewalk
Organization Unit [spacewalk.sandbox]? spacewalk
Email Address [root@localhost]? root@localhost
City? London
State? UK
Country code (Examples: "US", "JP", "IN",
    or type "?" to see a list)? GB

Now open https://spacewalk.sandbox/ in a web browser and create the admin account spacewalk/spacewalk.

Now to figure out how to make use of this beast….here’s a pretty good presentation from DevOps London about it:

Dev Ops meeting at youDevise, 3/3/2011 (Part 1 of 2) from youdevisetech on Vimeo.

Creating a minimal CentOS VM using VMWare Fusion on Mac OS X

I’ve written before about bootstrapping my development virtual infrastructure. When I did that I used an existing bare image VM I had lying around, but with CentOS 6 around the corner I wanted to document a process for creating a bare image from scratch.

First, you need CentOS 5.6 ISOs, so download CentOS if needed.

We need to create a floppy drive with a FAT or EXT2 filesystem, with a ks.cfg file on it. On the mac this seems trickier than it should be. If you have an existing linux VM you can use that. If not, make a new default VM, then

  • attach the CentOS 5.6 part1 DVD iso as the DVD drive.
  • Run installer. Settings do not matter. Disable packages to make install go quickly.
  • shut down VM.
  • echo ' ' > floppy144.flp, add floppy drive with that file.
  • start VM.

Let’s make our floppy and put a kickstart file on it:

fdformat /dev/floppy
mkfs.ext2 /dev/floppy
mount /dev/floppy /mnt/floppy

cat >/mnt/floppy/ks.cfg <<END
authconfig --enableshadow --enablemd5
bootloader --location=mbr --driveorder=sda
firewall --disable
keyboard us-acentos
lang en_US.UTF-8
network --device eth0 --bootproto dhcp
rootpw changeme12
selinux --disabled
timezone --utc Europe/London
clearpart --all --initlabel
part /boot --fstype ext3 --size=100 --ondisk=sda
part pv.2 --size=0 --grow --ondisk=sda
volgroup VolGroup00 --pesize=32768 pv.2
logvol / --fstype ext3 --name=LogVol00 --vgname=VolGroup00 --size=1024 --grow
logvol swap --fstype swap --name=LogVol01 --vgname=VolGroup00 --size=256 --grow --maxsize=512
%packages --excludedocs --nobase

Note among other things the above settings disable the iptables firewall and disable selinux and use a plaintext password changeme12. You should understand what that means and be happy with the security implications before you continue.

Shut down the VM, or at least unmount the floppy. If you made that VM just to create the floppy file, you can now throw that VM away. If you’re writing your own kickstart file you may want to keep it around though in case you made a mistake, so you can go back easily and edit the file on the floppy.

Now we’re ready to start work on our image:

  • create new empty VM called vanilla-sandbox.
  • Resize the disk image to 10GB.
  • set RAM to 256MB.
  • disable just about all options including audio support, usb support, printer support, file sharing, etc.
  • set to NAT networking.
  • attach CentOS part1 DVD to cdrom drive.
  • attach floppy144.flp as a new floppy drive.
  • Start the VM.
  • At the prompt, type linux ks=floppy.
  • Wait until prompted, then press enter to reboot.
  • log in as root.
  • run yum -y update.
  • run find /var/cache -type f | xargs rm -f.
  • shutdown -h now.
  • open the settings pane, open the hard disk panel, and select “clean up disk”.
  • disconnect and remove the floppy drive. Disconnect the DVD drive.
  • (optional), open the directory containing the VM location in the terminal and mv *.vmwarevm/* . && rm -r *.vmwarevm. When you next open the VM (you will need to use File > Open), select “I moved it”.
  • Make a backup copy of the virtual machine files, this is your clean VMWare-independent base VM.
  • Start the virtual machine.
  • in the Fusion menu, select Virtual Machine > Install VMWare tools, and install the tools:
    mkdir /mnt/cdrom
    mount /dev/cdrom /mnt/cdrom
    cd /tmp
    tar zxf /mnt/cdrom/VMwareTools*.tar.gz
    yum install -y perl
    ./ --default
    umount /mnt/cdrom
    rm -Rf /tmp/vmware-tools-distrib
  • In the Fusion menu, select Virtual Machine > CD/DVD > Disconnect CD/DVD
  • Shut down the virtual machine
  • Make a backup copy of the virtual machine files, this is your clean VMWare-ready base VM.

So now you have one squeaky clean VM image and a documented process for recreating it. Of course, most of this process isn’t particularly specific to CentOS, or VMWare, or Mac OS X. If you use different install media, these instructions work without changes for various versions of Fedora and Red Hat Enterprise Linux. According to this official ubuntu help page the same basic approach may work for recent Ubuntu releases (though replacing yum with apt-get).

I seem to remember that on linux there’s various more convenient ways to do this stuff. For example, you can install into a loopback device. But for now, this will do.