Updating my web presence

I made my first website on Geocities. I started my first blog when they were still called journals. I was working on social networking software before it was called that. It meant I was on Facebook, Twitter and other platforms long before most ‘normal’ people.

After being early to join the social web I was also early to leave, for the usual reasons (offline life is more fun than online life!). I hadn’t touched this blog in about 8 years, and have barely posted on Twitter for 10 years. When facebook introduced banner images, I used it to inform my facebook friends I was not actually logging in to facebook. If you wanted to reach me, you could choose only from phone, SMS, E-mail and (reluctantly) WhatsApp.

But I’m no longer so worried about internet addiction or getting misunderstood about my defenses against it. In 2020 it seems it has become normal to use social networking not at all or only in moderation, and many communities seem to (once again) welcome casual and intermittent participation. I might, just might, stop lurking around and try again to contribute something useful in the public sphere.

As a first step, I’ve cleaned up my websites and am making some hobby software projects public over on GitHub.

Image of my dipping my toes into the watter on a beach

Creating a minimal CentOS VM using VMWare Fusion on Mac OS X

I’ve written before about bootstrapping my development virtual infrastructure. When I did that I used an existing bare image VM I had lying around, but with CentOS 6 around the corner I wanted to document a process for creating a bare image from scratch.

First, you need CentOS 5.6 ISOs, so download CentOS if needed.

We need to create a floppy drive with a FAT or EXT2 filesystem, with a ks.cfg file on it. On the mac this seems trickier than it should be. If you have an existing linux VM you can use that. If not, make a new default VM, then

  • attach the CentOS 5.6 part1 DVD iso as the DVD drive.
  • Run installer. Settings do not matter. Disable packages to make install go quickly.
  • shut down VM.
  • echo ' ' > floppy144.flp, add floppy drive with that file.
  • start VM.

Let’s make our floppy and put a kickstart file on it:

fdformat /dev/floppy
mkfs.ext2 /dev/floppy
mount /dev/floppy /mnt/floppy

cat >/mnt/floppy/ks.cfg <<END
authconfig --enableshadow --enablemd5
bootloader --location=mbr --driveorder=sda
firewall --disable
install
cdrom
keyboard us-acentos
lang en_US.UTF-8
network --device eth0 --bootproto dhcp
rootpw changeme12
selinux --disabled
timezone --utc Europe/London
skipx
text
clearpart --all --initlabel
part /boot --fstype ext3 --size=100 --ondisk=sda
part pv.2 --size=0 --grow --ondisk=sda
volgroup VolGroup00 --pesize=32768 pv.2
logvol / --fstype ext3 --name=LogVol00 --vgname=VolGroup00 --size=1024 --grow
logvol swap --fstype swap --name=LogVol01 --vgname=VolGroup00 --size=256 --grow --maxsize=512
%packages --excludedocs --nobase
@Core
END

Note among other things the above settings disable the iptables firewall and disable selinux and use a plaintext password changeme12. You should understand what that means and be happy with the security implications before you continue.

Shut down the VM, or at least unmount the floppy. If you made that VM just to create the floppy file, you can now throw that VM away. If you’re writing your own kickstart file you may want to keep it around though in case you made a mistake, so you can go back easily and edit the file on the floppy.

Now we’re ready to start work on our image:

  • create new empty VM called vanilla-sandbox.
  • Resize the disk image to 10GB.
  • set RAM to 256MB.
  • disable just about all options including audio support, usb support, printer support, file sharing, etc.
  • set to NAT networking.
  • attach CentOS part1 DVD to cdrom drive.
  • attach floppy144.flp as a new floppy drive.
  • Start the VM.
  • At the prompt, type linux ks=floppy.
  • Wait until prompted, then press enter to reboot.
  • log in as root.
  • run yum -y update.
  • run find /var/cache -type f | xargs rm -f.
  • shutdown -h now.
  • open the settings pane, open the hard disk panel, and select “clean up disk”.
  • disconnect and remove the floppy drive. Disconnect the DVD drive.
  • (optional), open the directory containing the VM location in the terminal and mv *.vmwarevm/* . && rm -r *.vmwarevm. When you next open the VM (you will need to use File > Open), select “I moved it”.
  • Make a backup copy of the virtual machine files, this is your clean VMWare-independent base VM.
  • Start the virtual machine.
  • in the Fusion menu, select Virtual Machine > Install VMWare tools, and install the tools:
    mkdir /mnt/cdrom
    mount /dev/cdrom /mnt/cdrom
    cd /tmp
    tar zxf /mnt/cdrom/VMwareTools*.tar.gz
    yum install -y perl
    ./vmware-install.pl --default
    umount /mnt/cdrom
    cd
    rm -Rf /tmp/vmware-tools-distrib
    
  • In the Fusion menu, select Virtual Machine > CD/DVD > Disconnect CD/DVD
  • Shut down the virtual machine
  • Make a backup copy of the virtual machine files, this is your clean VMWare-ready base VM.

So now you have one squeaky clean VM image and a documented process for recreating it. Of course, most of this process isn’t particularly specific to CentOS, or VMWare, or Mac OS X. If you use different install media, these instructions work without changes for various versions of Fedora and Red Hat Enterprise Linux. According to this official ubuntu help page the same basic approach may work for recent Ubuntu releases (though replacing yum with apt-get).

I seem to remember that on linux there’s various more convenient ways to do this stuff. For example, you can install into a loopback device. But for now, this will do.

Installing VMWare Studio on VMware Fusion on Mac OS X

VMWare Studio is a free virtual appliance from VMware that allows you to “author, configure, deploy and customize virtual machines”. There’s no instructions how to get it running on the mac, but it works just fine with VMWare Fusion:

  • Register with VMWare
  • (Buy), download and install VMWare Fusion
  • Download and install VMWare OVF tool
  • Download VMWare Studio VMDK disk, unzip
  • Download VMWare Studio OVF 1.0 spec file
  • Convert VMware Studio OVF to VMX: /opt/vmware/ovftool/ovftool -tt=VMX VMware_Studio-2.5.0.0-387333_OVF10.ovf VMware_Studio-2.5.0.0-387333_VMX.vmx
  • Import VMWare Studio VMX into VMWare Fusion

Start the VMware Studio VM. Read and agree to the license agreement. Set a root password. Open the provided URL in your web browser.

$ ssh 192.168.1.75
root@localhost:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 8.04.4 LTS
Release:        8.04
Codename:       hardy

VMWare Studio looks like it consists of lighttpd serving up a GWT frontend that talks to a sfcb CIM backend, where CIM is a standard that’s part of the bigger Web-based Enterprise Management standard. Presumably they’re using the same CIM setup as, say, vSphere. Pretty cool setup.

I’ve built a few VMs and it all seems to work ok. The one gotcha is setting the build settings when defining VM profiles. Pick VMWare Workstation, and point it at /Library/Application Support/VMware Fusion to use the vmrun in VMWare Fusion.

I’ve also tried importing the OVF into VirtualBox which also works. So if you have a VMware install somewhere else (I think even VMWare Player would work since it provides a vmrun command) you could avoid buying Fusion.

I don’t think I would actually want to use VMWare Studio for building linux virtual machines; the scripted approach of cobbler seems preferable. OTOH, VMWare Studio provides good windows support, and can be used to create windows VMs from scratch. I can imagine:

  • define a windows build in VMWare Studio that outputs OVF
  • write a script that interacts with VMWare Studio through CIM to create new windows virtual machines
  • load those windows virtual machines into your hypervisor of choice if it supports OVF
  • if it does not support OVF but it supports VMDK/VMX, use VWWare OVFtool to convert to VMDK/VMX, then load into your hypervisor of choice (for example use qemu-img convert or import into amazon)

That seems a considerably lighter weight approach than going with Microsoft System Center Configuration Manager 2007 Operating System Deployment (Microsoft’s name for its cobbler). Interesting. Then again, if you’re managing a lot of windows host you may be invested in SCCM already?

Setting up vmware fusion, puppet and gitolite

The goal today is to start a rather ‘complete’ local dev environment on my laptop, from scratch, suitable for playing with continuous integration and deployment tools. My host is a mac. I’ll use VMWare Fusion for virtualization, running CentOS 5.5 guests. I’ll be setting up poor man’s vm cloning, and after that gitolite for version control, puppet for configuration management.

The main goal of all this is actually playing around with configuration management, so I’m not going to bother with backups or redundancy or any high availability config, however I will be deploying some basic java webapps and some basic PHP frontends to exercise the config management put in place.

Unfortunately, bootstrapping such a setup is a lot of work so I’ll write down a detailed installation log. This way, it may be less work next time, as per my own advice on bootstrapping. Maybe it helps you, too 🙂

1. Virtual Machine Setup

All these things will be deployed in virtual machines so as to match the production environment as much as possible. So the first step is to make it easy to create identical virtual machines that can be spun up and down on demand. Probably the best way is to use cobbler, but here’s how I did things:

1.1 Create a base vanilla VM

1.2 Create a setup to be able to clone VMs

As part of an effort to learn a bit about vmware, I wrote a simple script, that goes into ~/Documents/Virtual Machines:

#!/usr/bin/env python
# make-sandbox.py: clones virtual machines

import sys
import re
import os
import shutil

def usage():
    print "./make-sandbox.py [name]"
    print "  name should match ^[a-z][A-Z0-9]{1,15}$"

def fail(msg):
    print msg
    sys.exit(1)

def isVmConfigFile(name):
    return re.match("^.*?vanilla-sandbox\.(vmdk|vmsd|vmx|vmxf)$",
            name)

def writeNewConfig(src, dst, renamer):
    s = open(src, 'r').read()
    s = renamer(s)
    print "writing custom", dst
    f = open(dst, 'w')
    f.write(s)
    f.close()

def sandboxCopyTree(src, dst, renamer):
    names = os.listdir(src)

    print "mkdir", dst
    os.makedirs(dst)
    errors = []
    for name in names:
        srcname = os.path.join(src, name)
        dstname = renamer(os.path.join(dst, name))
        
        try:
            if os.path.islink(srcname):
                linkto = os.readlink(srcname)
                os.symlink(linkto, dstname)
                print "ln -s", srcname, dstname
            elif os.path.isdir(srcname):
                sandboxCopyTree(srcname, dstname, renamer)
            elif srcname.endswith(".log"):
                continue
            elif isVmConfigFile(srcname):
                writeNewConfig(srcname, dstname, renamer)
            else:
                print "cp", srcname, dstname
                shutil.copy2(srcname, dstname)
        except (IOError, os.error), why:
            errors.append((srcname, dstname, str(why)))
        # catch the Error from the recursive copytree so
        # that we can continue with other files
        except shutil.Error, err:
            errors.extend(err.args[0])
    try:
        shutil.copystat(src, dst)
    except OSError, why:
        errors.extend((src, dst, str(why)))
    if errors:
        raise shutil.Error(errors)

def sandboxRenameMaker(name):
    def renamer(dst):
        return dst.replace("vanilla", name)
    return renamer

if __name__ == "__main__":
    if len(sys.argv) != 2:
        usage()
        sys.exit(1)
    
    name = sys.argv[1]
    if not re.match('^[a-z][a-z0-9]{1,15}$', name):
        usage()
        sys.exit(1)

    basename = "%s-sandbox" % (name,)
    if os.path.exists(basename):
        fail("%s already exists?" % (basename,))
    
    sandboxCopyTree("vanilla-sandbox", basename,
            sandboxRenameMaker(name))

The script is pretty basic and hardcodes some stuff it probably shouldn’t, but it works ok for me. You can find various other imperfect scripts that do similar things on the vmware fusion forums. This script expects a directory ./vanilla-sandbox containing a vm named vanilla-sandbox, and the intended name of the new virtual machine as an argument.

1.3 Create some VMs

Invoke the new script like so:

cd ~/Documents/Virtual\ Machines && ./make-sandbox.py puppet

Which results in a virtual machine named puppet-sandbox. The virtual machine is not ready for use yet. Additional steps:

  • In VMWare Fusion, select File > Open..., then open the new VM.
  • Start the VM. VMWare will ask whether you moved or copied the VM. Select “I copied…”
  • log in as root
  • vi /etc/sysconfig/network, change hostname to match vm name (i.e. puppet.sandbox)
  • vi /etc/hosts, change hostname to match vm name
  • hostname [hostname], change hostname for the running system

Yes, this manual edit of the network settings is kind of icky, but I looked at how cobbler integrates vmware and koan and it just seemed a bit too much work for me right now. Perhaps I’ll look at that later.

Specifically for puppet, I want the machine running it to have a static IP, so I can put a static entry into /etc/hosts on the guest OS and have that always work. So, for the puppet machine, network config gets an extra step:

cp /etc/sysconfig/network-scripts/ifcfg-eth0
        /etc/sysconfig/network-scripts/ifcfg-eth0.bak
cat >/etc/sysconfig/network-scripts/ifcfg-eth0 <<END
# Advanced Micro Devices [AMD] 79c970 [PCnet32 LANCE]
DEVICE=eth0
BOOTPROTO=static
IPADDR=172.16.64.3
NETMASK=255.255.255.0
NETWORK=172.16.64.0
BROADCAST=172.16.64.255
ONBOOT=yes
GATEWAY=172.16.64.2
END
vi /etc/resolv.conf (172.16.64.2 is DNS)
service network restart

Then, on the host, echo "172.16.64.3 puppet.sandbox puppet" >> /etc/hosts.

Note 172.16.64.x is the default network used by vmware fusion NAT, vmnet8. You can find these details in /Library/Application Support/VMware Fusion/vmnet8/dhcpd.conf, which I believe really ought to learn about this, too:

host puppet {
    hardware ethernet 00:0C:29:3E:FC:56;
    fixed-address 172.16.64.3;
}

where that ethernet address is generated by vmware fusion, and you can find it with cat ~/Documents/Virtual Machines/puppet-sandbox/puppet-sandbox.vmx | grep generatedAddress. Restart vmware’s networking fanciness with /Library/Application Support/VMware Fusion/boot.sh --restart. Now restart networking on the puppet guest and check it’s working ok:

/etc/init.d/network restart
ping -c 1 www.google.com

2. Bootstrapping services

Once you have yourselves some base VMs one way or another, the next step is to get the combo of puppet and gitolite up properly. Normally these really need dedicated machines but I’m trying to conserve RAM so they’ll have to fit together on one VM for now.

2.1 Basic puppet server install

This bit is very easy:

yum install puppet-server ruby-shadow
mkdir -p /etc/puppet/manifests
puppetmaster --genconfig > /etc/puppet/puppet.conf
cat >/etc/puppet/manifests/site.pp <<END

file { "/etc/sudoers":
  owner => root, group => root, mode => 440
}

END
service puppetmaster start
puppetd --test

2.2 Basic gitolite install

We’ll actually use this puppet to install gitolite for us, then move the puppet config into gitolite once it’s set up.

Note that gitolite currently needs git 1.6.2+, so you need to get that from somewhere, it isn’t currently in CentOS 5 or EPEL. For this reason, I added the webtatic yum repo config earlier. Probably not a good idea for a production environment!

Here’s all the bits and pieces to get gitolite set up:

yum install git

# move /etc/sudoers resource to a module
cd /etc/puppet
mkdir -p /etc/puppet/modules/sudo/manifests
cat > /etc/puppet/modules/sudo/manifests/init.pp <<END
class sudo {
    file { "/etc/sudoers":
      owner => root, group => root, mode => 440
    }
}
END

# create a gitolite module
mkdir -p /etc/puppet/modules/gitolite/{manifests,files}

cat > /etc/puppet/modules/gitolite/files/install-gitolite.sh <<END
#!/usr/bin/env bash
# initial system install of gitotis. Run as root.

set -e

cd /home/git

if [[ ! -d "gitolite-source" ]]; then
    git clone git://github.com/sitaramc/gitolite gitolite-source
fi
cd gitolite-source
git checkout v1.5.8
mkdir -p /usr/local/share/gitolite/conf
        /usr/local/share/gitolite/hooks
src/gl-system-install /usr/local/bin
        /usr/local/share/gitolite/conf
        /usr/local/share/gitolite/hooks
END

cat > /etc/puppet/modules/gitolite/files/setup-gitolite.sh <<END
#!/usr/bin/env bash
# initial for-gitolite-user setup of gitolite. Run as gitolite.

set -e

/usr/local/bin/gl-setup /home/git/lsimons.pub
END

# note: truncated ssh key for blog post
cat > /etc/puppet/modules/gitolite/files/lsimons.pub <<END
ssh-rsa AAAA...== lsimons@...
END

cat > /etc/puppet/modules/gitolite/files/gitolite-rc <<END
\$GL_PACKAGE_CONF = '/usr/local/share/gitolite/conf';
\$GL_PACKAGE_HOOKS = '/usr/local/share/gitolite/hooks';
\$REPO_BASE="repositories";
\$REPO_UMASK = 0077;         # gets you 'rwx------'
\$PROJECTS_LIST = \$ENV{HOME} . "/projects.list";
\$GL_ADMINDIR=\$ENV{HOME} . "/.gitolite";
\$GL_LOGT="\$GL_ADMINDIR/logs/gitolite-%y-%m.log";
\$GL_CONF="\$GL_ADMINDIR/conf/gitolite.conf";
\$GL_KEYDIR="\$GL_ADMINDIR/keydir";
\$GL_CONF_COMPILED="\$GL_ADMINDIR/conf/gitolite.conf-compiled.pm";
\$GIT_PATH="";
\$GL_BIG_CONFIG = 0;
\$GL_NO_DAEMON_NO_GITWEB = 0;
\$GL_NO_CREATE_REPOS = 0;
\$GL_NO_SETUP_AUTHKEYS = 0;
\$GL_GITCONFIG_KEYS = "";
\$HTPASSWD_FILE = "";
\$RSYNC_BASE = "";
\$SVNSERVE = "";
\$GL_WILDREPOS = 0;
\$GL_WILDREPOS_PERM_CATS = "READERS WRITERS";
1;
END

cat > /etc/puppet/modules/gitolite/manifests/init.pp <<END
class gitolite {
    package { git:
        ensure => latest
    }
    
    group { git:
        ensure => present,
        gid => 802
    }
    
    user { git:
        ensure => present,
        gid => 802,
        uid => 802,
        home => "/home/git",
        shell => "/bin/bash",
        require => Group["git"]
    }
    
    file {
        "/home/git":
            ensure => directory,
            mode => 0750,
            owner => git,
            group => git,
            require => [User["git"], Group["git"]];

        "/home/git/install-gitolite.sh":
            ensure => present,
            mode => 0770,
            owner => git,
            group => git,
            require => File["/home/git"],
            source => "puppet:///modules/gitolite/install-gitolite.sh";
            
        "/home/git/setup-gitolite.sh":
            ensure => present,
            mode => 0770,
            owner => git,
            group => git,
            require => File["/home/git"],
            source => "puppet:///modules/gitolite/setup-gitolite.sh";
            
        "/home/git/lsimons.pub":
            ensure => present,
            mode => 0660,
            owner => git,
            group => git,
            require => File["/home/git"],
            source => "puppet:///modules/gitolite/lsimons.pub";
        
        "/home/git/.gitolite.rc":
            ensure => present,
            mode => 0660,
            owner => git,
            group => git,
            require => File["/home/git"],
            source => "puppet:///modules/gitolite/gitolite-rc";
    }
    
    exec {
        "/home/git/install-gitolite.sh":
            cwd => "/home/git",
            user => root,
            require => [
                        File["/home/git/install-gitolite.sh"],
                        Package["git"]
                ];

        "/home/git/setup-gitolite.sh":
            cwd => "/home/git",
            user => git,
            environment => "HOME=/home/git",
            require => [
                    Exec["/home/git/install-gitolite.sh"],
                    File["/home/git/setup-gitolite.sh"],
                    User["git"]
                ]
    }
}
END

# update the site config to use the sudo and gitolite modules
cat > /etc/puppet/manifests/modules.pp <<END
import "sudo"
import "gitolite"
END

cat > /etc/puppet/manifests/nodes.pp <<END
node basenode {
    include sudo
}

node "puppet.sandbox" inherits basenode {
    include gitolite
}
END

cat > /etc/puppet/manifests/site.pp <<END
import "modules"
import "nodes"
END

# invoke puppet once to apply the new config
puppetd -v --test

So now we have gitolite installed on the server. So far so good.

2.3 Creating an scm git repo

I now need a repository in which to put the puppet configs. I was originally planning to have a repo ‘scm’ and a directory ‘puppet’ within it, so that I could have /etc/scm, with /etc/puppet a symlink to /etc/scm/puppet. It turns out puppet doesn’t support symlinks for /etc/puppet, so I ended up fiddling about a bit…

# on client
git clone git@puppet:gitolite-admin
cd gitolite-admin
cat >>conf/gitolite.conf <<END

repo    scm
        RW+     =   lsimons

END
git add conf/gitolite.conf 
git commit -m "Adding 'scm' repo"
git push origin master
cd ..

2.4 Making puppet use the config from the scm repo

First we need to get the existing config into version control:

mkdir scm
cd scm
git init
cat >> .git/config <<END
[remote "origin"]
    url = git@puppet:scm
    fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
    remote = origin
    merge = refs/heads/master
END
scp -r root@puppet:/etc/puppet/* .
git add *
git commit -m "Check in initial puppet config"
git push --all

Next, get the config out of version control and underneath puppet, and automate this process:

# install from-git puppet config on server

cd /etc
mv puppet puppet.bak
mkdir puppet
chown git puppet
cd puppet
sudo -u git git clone file:///home/git/repositories/scm.git puppet

# install post-receive hook to update /etc/puppet after a push

su - git
cat > /home/git/repositories/scm.git/hooks/post-receive <<END
#!/usr/bin/env bash
(cd /etc/puppet; env -i git pull -q origin master)
END
chmod u+x /home/git/repositories/scm.git/hooks/post-receive
exit

That little bit of env -i git inside that hook had me baffled for a bit. It turns out that I needed to empty the environment before invoking git from inside of a hook, because otherwise it’ll pick up the GIT_DIR variable. D’oh!

With this config re-set up, there should be effectively 0 changes when we run puppet. Let’s check:

puppet /etc/puppet/manifests/site.pp

2.5 On (not) putting all the puppet.sandbox config in puppet

Note that installing the post-receive hook from the previous step is not puppeted. The reason for this is one of synchronization. For example, if puppet somehow creates that post-receive file before the scm repository exists, gitolite will complain. It seems easier to have puppet not touch things managed by gitolite and vice versa.

Similarly, the installation of puppet itself is not puppeted, leaving the configuration of puppet.sandbox not something that can be completely automatically rebuilt.

Instead, rebuilding this box should be done by first re-following the instructions above, and then restoring the contents of the git@puppet:gitolite-admin and git@puppet:scm repositories from their current state (or latest backup). For my current purposes, that’s absolutely fine.

3. Setting up puppet dashboard

I also had a look at installing puppet dashboard. Because I know ruby and rails and gems can be a big dependency hell I figured I didn’t even want to try it in a VM, and instead I “just” got it running on my mac.

3.1 MySQL, ruby and mac os x

Puppet dashboard is built using ruby on rails and suggests using mysql for persistence (in retrospect I should not have listened and used sqlite :-)). Ruby on Rails apparently accesses MySQL through the mysql gem. The MySQL gem has to link against both the native mysql library and the native ruby library. Fortunately, I’m aware enough of the potential pain that I tend to carefully install the most compatible version of systems like this:

$ file `which ruby`
/usr/local/bin/ruby: Mach-O executable i386
$ file `which mysql`
/usr/local/mysql/bin/mysql: Mach-O executable i386
$ ruby --version
ruby 1.8.7 (2009-06-12 patchlevel 174) [i686-darwin9.7.0]
$ mysql --version
mysql  Ver 14.14 Distrib 5.1.31,
        for apple-darwin9.5.0 (i386) using readline 5.1

You’d hope that the mysql gem build picks up on all this ok, but that’s not quite the case. Instead, you really have to be quite explicit:

vi /usr/local/lib/ruby/1.8/i686-darwin9.7.0/rbconfig.rb
# change to CONFIG["ARCH_FLAG"] = "-arch i386"

sudo gem uninstall mysql
sudo env ARCHFLAGS="-arch i386" gem install --verbose \
    --no-rdoc --no-ri mysql \
    -- --with-mysql-dir=/usr/local \
    --with-mysql-config=/usr/local/mysql/bin/mysql_config
cd /usr/local/mysql/lib/
sudo ln -s mysql .
cd .

3.2 Look, ma, it’s a rails app

After this, fortunately it’s easy again.

tar zxf ...puppet-dashboard...
mv ... ~/puppet-dashboard
cd ~/puppet-dashboard
rake RAILS_ENV=production db:create
rake RAILS_ENV=production db:migrate
./script/server -e production
open http://localhost:3000/

Works like a charm. To get some data to see requires tweaking the puppet VM:

cd /Users/lsimons/puppet-dashboard/ext/puppet
vi puppet_dashboard.rb
# change HOST to 172.16.64.1
ssh root@puppet mkdir -p /var/lib/puppet/lib/puppet/reports
scp puppet_dashboard.rb root@puppet:/var/lib/puppet/lib/puppet/reports/
exit

cd ~/dev/scm/
vi puppet/puppet.conf
# report = true for [puppetd]
# reports = puppet_dashboard for [puppetmasterd]
git add puppet/puppet.conf
git commit -m "Enable reporting"
git push

4. Recap

So now we have:

  • A working, documented, repeatable process for creating new VMs
  • A working, documented, repeatable process for bootstrapping puppet
  • A neat version-controlled way of changing the puppet config
  • An installation of a puppet master that serves up the latest config
  • A puppeted installation of gitosis
  • A not-so-great but working installation of puppet dashboard
  • A few more VMs to configure

Administrative stuff to take care of when moving to the UK

As I’ve found out recently, the UK is not exactly the easiest country in the world for foreigners to move to, and it also isn’t exactly the most digitized. Navigating the maze of different rules and institutions depends on doing things in the right order. Here is that order:

  1. Get a pre-paid mobile phone – just walk into a shop and buy one. Lots of institutions want a UK phone number; you can’t get a mobile subscription until you have proof of residency. Get a pre-paid USB 3G dongle. You should get your dongle from Vodafone because with them your pre-paid bandwidth does not expire after a month.
  2. Get a residential address where you can receive mail. Lots of institutions require one. It cannot be a business address and it cannot be a post box. So ask a friend or collegue! Alternatively if you are going to go for a flatshare arrangement you might be able to get that by paying cash up front and that becomes your address.
  3. Get a bank account with Barclays. They seem to be the only bank that can easily deal with foreigners moving to the country. You cannot apply online. Find the local branch using the barclays website, and ring them or walk in for an appointment. A lot of the people at the branch will think they cannot help you, but you should usually be able to find at least one account manager that knows that (s)he can help you. For your appointment you will need your UK phone number, your UK address and two forms of ID (for example passport and drivers’ license). If you have an ID that has an address on it (does not have to be your UK address) that helps. Make it clear that you are already employed in the UK, that you get paid well, that your salary will go into this new account, and you are also happy to put some significant amount of cash in your new account immediately. Be prepared for the appointment to take at least two hours. Make sure to get online banking. You get your bank account number and sort code number immediately and your bank card and online banking details in 5-7 working days.
  4. Register for online banking on the barclays website.
  5. Give your bank details (bank, account number, sort code, local branch) to your employer.
  6. Get a flat or flatshare. findaproperty.com is the best site to use for finding a flat, moveflat.co.uk is the best site for finding a flatshare. Offering to pay rent in advance tends to take care of most housing agent concerns (like you not having a referral from another UK housing agent). You also have to be able to set up a standing order, for which you need bank account number, sort code number, and address details of your local bank branch.
  7. If you got your own flat, get a BT landline. You cannot do this online, instead call 0800 800 150. I called Friday at 7:30 pm and was helped quickly. You can order broadband at the same time if you want. Don’t bother trying to get a phone line anywhere else – its significantly more painful than going with BT.
  8. Walk into your local Barclays branch to change your address details. Bring your bank card, account details and identification.
  9. Give your new address details to your employer.
  10. Register for council tax with your local council. How to do so depends on the council; check their website. I e-mailed my council who then sent me forms to fill out and send back.
  11. Register for utilities (electricity, water, gas). Your landlord or agent should be able to help figure this out, though utilities are handled through a commercial setup so you have to pick a provider. Again you should set up standing orders/direct debit for paying your bill.
  12. If you own a TV, register for a TV License. You can do this online. Set up direct debit.
  13. Once your BT phoneline is set up, you can opt to get Sky satellite TV. If you live in a house or small block of flats you can do this online, otherwise you have to check with your landlord about whether there is a minidish for your block, and if there is, call sky at 08442 410 137.
  14. Once you get a utility bill, you can finally get a mobile phone subscription plan. When you walk into the shop, bring a utility bill, proof of ID (i.e. passport), bank and address details, and your landline phone number. (faking a utility bill is obviously not very hard, and I’ve heard reports from people that have done this to get a subscription.)
  15. Give your new phone details to your employer. If you care about your bank being able to call you, pay yet another visit to the local branch.
  16. Once you’ve received salary in your new bank account at least twice, you can apply for a credit card. With Barclays you can do this online as long as you’re already registered for online banking.
  17. Get a National insurance number. I have yet to try this out. According to this website I have to call 0845 600 0643 (8.00 am to 6.00 pm Monday to Friday) and then go through an interview.
  18. Give your employer your national insurance number.

I was happy to have several collegues around that had been through the same process so I got many useful tips from them, but even then I found that all in all it takes quite a bit of calendar time for all these things to get sorted out. Because of that, I recommend to start with the first steps as soon as you know you’re moving to the UK.

Have you recently moved to the UK? Did you do things the same way or did perhaps find a more efficient route? Any hints or tips to share?

A 99.9% gmail SLA costs $50/year

There’s been some reports around the web about people having their gmail accounts disabled, getting bad customer support from google, etc etc.

So, after reading all that, you’ve decided you really want a service-level agreement for your gmail, and priority phone support? Pay for it.

Google apps premier edition costs 50 dollars (40 euros) a year. Besides some additional features, it comes with priority email support, a dedicated priority support phone number, and a 99.9% uptime guarantee.

While in this day and age 99.9% is not exactly impressive, at just over $4/month it is pretty cheap.

my day job now involves world of warcraft

Joost (where I work) is working with SK Gaming on doing live coverage of world of warcraft raids (announcement).

I haven’t quite convinced anyone yet that this means that actually playing world of warcraft should be part of my job description, but I do get to watch SK in-game footage as part of testing in-development versions of our software!

Working a bit on this stuff really makes me want to get back into raiding. If only I could find the time! In the meantime this is probably the closest I’ll get…

So, in the meantime I’m hoping we can get actual live coverage of some powerleveling (which I once tried to get good at). Apparently some people can do 1-70 in two days now! S(l)ick.