Skip to content

Installing VMWare Studio on VMware Fusion on Mac OS X

May 30, 2011

VMWare Studio is a free virtual appliance from VMware that allows you to “author, configure, deploy and customize virtual machines”. There’s no instructions how to get it running on the mac, but it works just fine with VMWare Fusion:

  • Register with VMWare
  • (Buy), download and install VMWare Fusion
  • Download and install VMWare OVF tool
  • Download VMWare Studio VMDK disk, unzip
  • Download VMWare Studio OVF 1.0 spec file
  • Convert VMware Studio OVF to VMX: /opt/vmware/ovftool/ovftool -tt=VMX VMware_Studio-2.5.0.0-387333_OVF10.ovf VMware_Studio-2.5.0.0-387333_VMX.vmx
  • Import VMWare Studio VMX into VMWare Fusion

Start the VMware Studio VM. Read and agree to the license agreement. Set a root password. Open the provided URL in your web browser.

$ ssh 192.168.1.75
root@localhost:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 8.04.4 LTS
Release:        8.04
Codename:       hardy

VMWare Studio looks like it consists of lighttpd serving up a GWT frontend that talks to a sfcb CIM backend, where CIM is a standard that’s part of the bigger Web-based Enterprise Management standard. Presumably they’re using the same CIM setup as, say, vSphere. Pretty cool setup.

I’ve built a few VMs and it all seems to work ok. The one gotcha is setting the build settings when defining VM profiles. Pick VMWare Workstation, and point it at /Library/Application Support/VMware Fusion to use the vmrun in VMWare Fusion.

I’ve also tried importing the OVF into VirtualBox which also works. So if you have a VMware install somewhere else (I think even VMWare Player would work since it provides a vmrun command) you could avoid buying Fusion.

I don’t think I would actually want to use VMWare Studio for building linux virtual machines; the scripted approach of cobbler seems preferable. OTOH, VMWare Studio provides good windows support, and can be used to create windows VMs from scratch. I can imagine:

  • define a windows build in VMWare Studio that outputs OVF
  • write a script that interacts with VMWare Studio through CIM to create new windows virtual machines
  • load those windows virtual machines into your hypervisor of choice if it supports OVF
  • if it does not support OVF but it supports VMDK/VMX, use VWWare OVFtool to convert to VMDK/VMX, then load into your hypervisor of choice (for example use qemu-img convert or import into amazon)

That seems a considerably lighter weight approach than going with Microsoft System Center Configuration Manager 2007 Operating System Deployment (Microsoft’s name for its cobbler). Interesting. Then again, if you’re managing a lot of windows host you may be invested in SCCM already?

Getting started with vagrant on Mac OS X

May 26, 2011

Vagrant wants an up-to-date RubyGems. From experience changing the ruby install bundled with Mac OS X can be a bad idea. I considered using rvm but I don’t think I need all that power here. So I’ll be my usual old-fashioned self, and install my own ruby in /usr/local.

  • Download latest ruby source code (1.9.2-p180 at time of writing), ./configure && make && make install (this results in a 64-bit only build)
  • create a new terminal, check gem environment gives the new ruby, run gem update --system && gem update
  • if you now get a lot of deprecation warnings, follow this advice to get rid of them: gem pristine --all --no-extensions (there are no native extension installed yet so no need to worry about those)
  • if you get any instructions about running rdoc-data, ignore the instructions, according to this bug report it’s not needed for ruby 1.9.2.

So now we have up-to-date ruby and up-to-date RubyGems. Good. Let’s get virtualbox and vagrant:

Get started:

vagrant box add lucid32 http://files.vagrantup.com/lucid32.box
mkdir test1
cd test1
vagrant init
vi Vagrantfile
# change config.vm.box to "lucid32"
# set config.ssh.max_tries to something high, say 50000
# set config.ssh.timeout to something high, say 3000000
vagrant up
vagrant ssh
echo test
exit

Works for me.

My preferred hypervisor on the mac is VMWare Fusion though. I already have a bunch of custom scripts set up to create and manage VMs and puppet them up. I could migrate those over to VirtualBox I guess and then change all my scripts to use vagrant, but I don’t really see that it offers me any benefit right now. Definitely a useful tool though and it’ll be interesting to follow its development.

stripping illegal characters out of xml in python

March 17, 2011

XML 1.0 does not allow all characters in unicode:

Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]

It often trips up developers (like, today, me) that end up having, say, valid unicode, with valid characters like VT (\x1B), or ESC (\x1B), and suddenly they are producing invalid XML. A decent way to deal with this is to strip out the invalid characters. For example this stack overflow post shows how to do this with perl:

$str =~ s/[^\x09\x0A\x0D\x20-\x{D7FF}\x{E000}-\x{FFFD}\x{10000}-\x{10FFFF}]//go;

Unfortunately the equivalent does not quite work with Python, since \x{10000}-\x{10FFFF} needs to be expressed as \U00010000-\U0010FFFF which not all versions of python seem to accept as part of a regular expression character class.

So people end up doing messy-looking things in python. But I figured out that if I invert the character class, the biggest character I need to write is \uFFFF, which the python regex engine does acccept. Yay:

import re

# xml 1.0 valid characters:
#    Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]
# so to invert that, not in Char ::
#       x0 - x8 | xB | xC | xE - x1F 
#       (most control characters, though TAB, CR, LF allowed)
#       | #xD800 - #xDFFF
#       (unicode surrogate characters)
#       | #xFFFE | #xFFFF |
#       (unicode end-of-plane non-characters)
#       >= 110000
#       that would be beyond unicode!!!
_illegal_xml_chars_RE = re.compile(u'[\x00-\x08\x0b\x0c\x0e-\x1F\uD800-\uDFFF\uFFFE\uFFFF]')

def escape_xml_illegal_chars(val, replacement='?'):
    """Filter out characters that are illegal in XML.
    
    Looks for any character in val that is not allowed in XML
    and replaces it with replacement ('?' by default).
    
    >>> escape_illegal_chars("foo \x0c bar")
    'foo ? bar'
    >>> escape_illegal_chars("foo \x0c\x0c bar")
    'foo ?? bar'
    >>> escape_illegal_chars("foo \x1b bar")
    'foo ? bar'
    >>> escape_illegal_chars(u"foo \uFFFF bar")
    u'foo ? bar'
    >>> escape_illegal_chars(u"foo \uFFFE bar")
    u'foo ? bar'
    >>> escape_illegal_chars(u"foo bar")
    u'foo bar'
    >>> escape_illegal_chars(u"foo bar", "")
    u'foo bar'
    >>> escape_illegal_chars(u"foo \uFFFE bar", "BLAH")
    u'foo BLAH bar'
    >>> escape_illegal_chars(u"foo \uFFFE bar", " ")
    u'foo   bar'
    >>> escape_illegal_chars(u"foo \uFFFE bar", "\x0c")
    u'foo \x0c bar'
    >>> escape_illegal_chars(u"foo \uFFFE bar", replacement=" ")
    u'foo   bar'
    """
    return _illegal_xml_chars_RE.sub(replacement, val)

Setting up a new mac

March 8, 2011

I got a new macbook pro. Rather than migrate my settings automatically I decided I’d go for a little spring cleaning by buillding it up from scratch.

Initial setup stuff

  • start software update
  • go through system preferences, add appropriate paranoia
  • open keychain utility, add appropriate paranoia
  • run software update
  • insert Mac OS X CD, install XCode
  • remove all default bookmarks from safari
  • open iTunes, sign in, turn off annoying things like ping
  • download, extract, install, run once, enter serial (where needed)
    • Chrome
    • Firefox
      • set master password
    • Thunderbird
      • set master password
    • Skype
    • OmniGraffle
    • OmniPlan
    • TextMate
    • VMware Fusion
    • Colloquy
    • IntelliJ IDEA; disable most enterpriseish plugins; then open plugin manager and install
      • python plugin
      • ruby plugin
      • I have years of crap in my intellij profile which I’m not adding to this machine
    • VLC
    • Things
  • download stunnel, extract, sudo mkdir /usr/local && sudo chown $USER /usr/local, ./configure --disable-libwrap, make, sudo make install
  • launch terminal, customize terminal settings (font size, window size, window title, backgroundc color) and save as default

Transfer from old mac

  • copy over secure.dmg, mount, set to automount on startup
  • import certs into keychain, firefox, thunderbird
  • set up thunderbird with e-mail accounts
  • copy over ~/.ssh and ~/.subversion
  • set up stunnel for colloquy, run colloquy, add localhost as server, set autojoin channels, change to tabbed interface
  • copy over keychains
  • copy over Office, run office setup assistant
  • copy over documents, data
  • copy over virtual machines
  • open each virtual machine, selecting “I moved it”
  • copy over itunes library
  • plug in, pair and sync ipad and iphone

Backup

  • set up time machine

Development packages

I like to know what is in my /usr/local, so I don’t use MacPorts or fink or anything like it.

  • download and extract pip, python setup.py install
  • pip install virtualenv Django django-piston south django-audit-log django-haystack httplib2 lxml
    • Ah mac 10.6 has a decent libxml2 and libxslt so lxml just installs. What a breeze.
  • download and install 64-bit mysql .dmg from mysql.com, also install preference pane
  • Edit ~/.profile, set CLICOLOR=yes, set PATH, add /usr/local/bin/mysql to path, source ~/.profile
    • again, I have years of accumulated stuf in my bash profile that I’m dumping. It’s amazing how fast bash starts when it doesn’t have to execute a gazillion lines of shell script…
  • code>pip install MySQL-python, install_name_tool -change libmysqlclient.16.dylib /usr/local/mysql/lib/libmysqlclient.16.dylib /Library/Python/2.6/site-packages/_mysql.so
  • take care of the many dependencies for yum, mostly standard ./configure && make && make install of
    • pkg-config
    • gettext
    • libiconv
    • gettext again, --with-libiconv-prefix=/usr/local
    • glib, --with-libiconv=gnu
    • popt
    • db-5.1.19.tar.gz, cd build_unix && ../dist/configure --enable-sql && make && make install, cp sql/sqlite3.pc /usr/local/lib/pkgconfig/, cd /usr/local/BerkeleyDB.5.1/include && mkdir db51 && cd db51 && ln -s ../*.h .
    • neon, ./configure --without-gssapi --with-ssl
    • rpm (5.3), export CPATH=/usr/local/BerkeleyDB.5.1/include, sudo mkdir /var/local && sudo chown $USER /var/local, ./configure --with-db=/usr/local/BerkeleyDB.5.1 --with-python --disable-nls --disable-openmp --with-neon=/usr/local && make && make install
    • pip install pysqlite pycurl
    • urlgrabber, sudo mkdir /System/Library/Frameworks/Python.framework/Versions/2.6/share && chown $USER sudo mkdir /System/Library/Frameworks/Python.framework/Versions/2.6/share && python setup.py install
    • intltool
    • yum-metadata-parser, export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/lib/pkgconfig; python setup.py install
  • install yum. Yum comes without decent install scripts (the idea is you install it from rpm). I did some hacks to get somewhere reasonable:
    cat Makefile | sed -E -e 's/.PHONY(.*)/.PHONY\1 install/' > Makefile.new
    mv Makefile.new Makefile
    cat etc/Makefile | sed -E -e 's/install -D/install/' > etc/Makefile.new
    mv etc/Makefile.new etc/Makefile
    make DESTDIR=/usr/local PYLIBDIR=/Library/Python/2.6 install
    mv /usr/local/Library/Python/2.6/site-packages/* /Library/Python/2.6/site-packages/
    mv /usr/local/usr/bin/* /usr/local/bin/
    mkdir /usr/local/sbin
    mv /usr/local/usr/sbin/* /usr/local/sbin/
    rsync -av /usr/local/usr/share/ /usr/local/share/
    rm -Rf /usr/local/usr/
    cat /usr/local/bin/yum | sed -E -e 's|/usr/share/yum-cli|/usr/local/share/yum-cli|' > /tmp/yum.new
    mv /tmp/yum.new /usr/local/bin
    chmod +x /usr/local/bin/yum
    

That’s how far I got last night, resulting in 86GB of disk use (most of which is VMs and iTunes library), just enough to be productive on my current $work project. I’m sure there’s weeks of tuning in my future.

Engineering IT Supply Manifesto

January 15, 2011

I came across an Engineering IT Supply Manifesto. It’s a long rant from a former engineering manager at facebook who makes the case that taking longer than a business day to supply IT equipment (laptops, monitors, keyboards, …) is stupid.

Amen to that.

When I worked at Joost the IT folks kept a stack of spare preconfigured macbooks and thinkpads in a cupboard (famously the macbook stack was higher than the thinkpad stack, because the macs broke more). The experience was pretty great for the developers (going from the downer of “crap my laptop broke I hope I committed recently” to the upper of “ooh shiny new laptop” so quickly), and also for the sysadmins (actually getting thanked for their hard work and careful planning), and I’m pretty sure it meant a net save of money.

As a contractor I’m used to keeping a spare laptop around. I don’t have one right now, but OTOH there’s an apple store 20 mins from work and 30 mins from home, so I know I can get away without that spare. I also tend to buy faster/better gear pretty frequently. At 2 years my current laptop really deserves replacing, just haven’t gotten around to it yet because I can’t decide what size to switch to.

Hmm. I suspect EU public sector organizations of a certain size have to do “big chunk” EU procurement for their IT…anyone know of public sector organizations that have drafted decent framework agreements compatible with this manifesto? 🙂

Setting up vmware fusion, puppet and gitolite

January 4, 2011

The goal today is to start a rather ‘complete’ local dev environment on my laptop, from scratch, suitable for playing with continuous integration and deployment tools. My host is a mac. I’ll use VMWare Fusion for virtualization, running CentOS 5.5 guests. I’ll be setting up poor man’s vm cloning, and after that gitolite for version control, puppet for configuration management.

The main goal of all this is actually playing around with configuration management, so I’m not going to bother with backups or redundancy or any high availability config, however I will be deploying some basic java webapps and some basic PHP frontends to exercise the config management put in place.

Unfortunately, bootstrapping such a setup is a lot of work so I’ll write down a detailed installation log. This way, it may be less work next time, as per my own advice on bootstrapping. Maybe it helps you, too 🙂

1. Virtual Machine Setup

All these things will be deployed in virtual machines so as to match the production environment as much as possible. So the first step is to make it easy to create identical virtual machines that can be spun up and down on demand. Probably the best way is to use cobbler, but here’s how I did things:

1.1 Create a base vanilla VM

1.2 Create a setup to be able to clone VMs

As part of an effort to learn a bit about vmware, I wrote a simple script, that goes into ~/Documents/Virtual Machines:

#!/usr/bin/env python
# make-sandbox.py: clones virtual machines

import sys
import re
import os
import shutil

def usage():
    print "./make-sandbox.py [name]"
    print "  name should match ^[a-z][A-Z0-9]{1,15}$"

def fail(msg):
    print msg
    sys.exit(1)

def isVmConfigFile(name):
    return re.match("^.*?vanilla-sandbox\.(vmdk|vmsd|vmx|vmxf)$",
            name)

def writeNewConfig(src, dst, renamer):
    s = open(src, 'r').read()
    s = renamer(s)
    print "writing custom", dst
    f = open(dst, 'w')
    f.write(s)
    f.close()

def sandboxCopyTree(src, dst, renamer):
    names = os.listdir(src)

    print "mkdir", dst
    os.makedirs(dst)
    errors = []
    for name in names:
        srcname = os.path.join(src, name)
        dstname = renamer(os.path.join(dst, name))
        
        try:
            if os.path.islink(srcname):
                linkto = os.readlink(srcname)
                os.symlink(linkto, dstname)
                print "ln -s", srcname, dstname
            elif os.path.isdir(srcname):
                sandboxCopyTree(srcname, dstname, renamer)
            elif srcname.endswith(".log"):
                continue
            elif isVmConfigFile(srcname):
                writeNewConfig(srcname, dstname, renamer)
            else:
                print "cp", srcname, dstname
                shutil.copy2(srcname, dstname)
        except (IOError, os.error), why:
            errors.append((srcname, dstname, str(why)))
        # catch the Error from the recursive copytree so
        # that we can continue with other files
        except shutil.Error, err:
            errors.extend(err.args[0])
    try:
        shutil.copystat(src, dst)
    except OSError, why:
        errors.extend((src, dst, str(why)))
    if errors:
        raise shutil.Error(errors)

def sandboxRenameMaker(name):
    def renamer(dst):
        return dst.replace("vanilla", name)
    return renamer

if __name__ == "__main__":
    if len(sys.argv) != 2:
        usage()
        sys.exit(1)
    
    name = sys.argv[1]
    if not re.match('^[a-z][a-z0-9]{1,15}$', name):
        usage()
        sys.exit(1)

    basename = "%s-sandbox" % (name,)
    if os.path.exists(basename):
        fail("%s already exists?" % (basename,))
    
    sandboxCopyTree("vanilla-sandbox", basename,
            sandboxRenameMaker(name))

The script is pretty basic and hardcodes some stuff it probably shouldn’t, but it works ok for me. You can find various other imperfect scripts that do similar things on the vmware fusion forums. This script expects a directory ./vanilla-sandbox containing a vm named vanilla-sandbox, and the intended name of the new virtual machine as an argument.

1.3 Create some VMs

Invoke the new script like so:

cd ~/Documents/Virtual\ Machines && ./make-sandbox.py puppet

Which results in a virtual machine named puppet-sandbox. The virtual machine is not ready for use yet. Additional steps:

  • In VMWare Fusion, select File > Open..., then open the new VM.
  • Start the VM. VMWare will ask whether you moved or copied the VM. Select “I copied…”
  • log in as root
  • vi /etc/sysconfig/network, change hostname to match vm name (i.e. puppet.sandbox)
  • vi /etc/hosts, change hostname to match vm name
  • hostname [hostname], change hostname for the running system

Yes, this manual edit of the network settings is kind of icky, but I looked at how cobbler integrates vmware and koan and it just seemed a bit too much work for me right now. Perhaps I’ll look at that later.

Specifically for puppet, I want the machine running it to have a static IP, so I can put a static entry into /etc/hosts on the guest OS and have that always work. So, for the puppet machine, network config gets an extra step:

cp /etc/sysconfig/network-scripts/ifcfg-eth0
        /etc/sysconfig/network-scripts/ifcfg-eth0.bak
cat >/etc/sysconfig/network-scripts/ifcfg-eth0 <<END
# Advanced Micro Devices [AMD] 79c970 [PCnet32 LANCE]
DEVICE=eth0
BOOTPROTO=static
IPADDR=172.16.64.3
NETMASK=255.255.255.0
NETWORK=172.16.64.0
BROADCAST=172.16.64.255
ONBOOT=yes
GATEWAY=172.16.64.2
END
vi /etc/resolv.conf (172.16.64.2 is DNS)
service network restart

Then, on the host, echo "172.16.64.3 puppet.sandbox puppet" >> /etc/hosts.

Note 172.16.64.x is the default network used by vmware fusion NAT, vmnet8. You can find these details in /Library/Application Support/VMware Fusion/vmnet8/dhcpd.conf, which I believe really ought to learn about this, too:

host puppet {
    hardware ethernet 00:0C:29:3E:FC:56;
    fixed-address 172.16.64.3;
}

where that ethernet address is generated by vmware fusion, and you can find it with cat ~/Documents/Virtual Machines/puppet-sandbox/puppet-sandbox.vmx | grep generatedAddress. Restart vmware’s networking fanciness with /Library/Application Support/VMware Fusion/boot.sh --restart. Now restart networking on the puppet guest and check it’s working ok:

/etc/init.d/network restart
ping -c 1 www.google.com

2. Bootstrapping services

Once you have yourselves some base VMs one way or another, the next step is to get the combo of puppet and gitolite up properly. Normally these really need dedicated machines but I’m trying to conserve RAM so they’ll have to fit together on one VM for now.

2.1 Basic puppet server install

This bit is very easy:

yum install puppet-server ruby-shadow
mkdir -p /etc/puppet/manifests
puppetmaster --genconfig > /etc/puppet/puppet.conf
cat >/etc/puppet/manifests/site.pp <<END

file { "/etc/sudoers":
  owner => root, group => root, mode => 440
}

END
service puppetmaster start
puppetd --test

2.2 Basic gitolite install

We’ll actually use this puppet to install gitolite for us, then move the puppet config into gitolite once it’s set up.

Note that gitolite currently needs git 1.6.2+, so you need to get that from somewhere, it isn’t currently in CentOS 5 or EPEL. For this reason, I added the webtatic yum repo config earlier. Probably not a good idea for a production environment!

Here’s all the bits and pieces to get gitolite set up:

yum install git

# move /etc/sudoers resource to a module
cd /etc/puppet
mkdir -p /etc/puppet/modules/sudo/manifests
cat > /etc/puppet/modules/sudo/manifests/init.pp <<END
class sudo {
    file { "/etc/sudoers":
      owner => root, group => root, mode => 440
    }
}
END

# create a gitolite module
mkdir -p /etc/puppet/modules/gitolite/{manifests,files}

cat > /etc/puppet/modules/gitolite/files/install-gitolite.sh <<END
#!/usr/bin/env bash
# initial system install of gitotis. Run as root.

set -e

cd /home/git

if [[ ! -d "gitolite-source" ]]; then
    git clone git://github.com/sitaramc/gitolite gitolite-source
fi
cd gitolite-source
git checkout v1.5.8
mkdir -p /usr/local/share/gitolite/conf
        /usr/local/share/gitolite/hooks
src/gl-system-install /usr/local/bin
        /usr/local/share/gitolite/conf
        /usr/local/share/gitolite/hooks
END

cat > /etc/puppet/modules/gitolite/files/setup-gitolite.sh <<END
#!/usr/bin/env bash
# initial for-gitolite-user setup of gitolite. Run as gitolite.

set -e

/usr/local/bin/gl-setup /home/git/lsimons.pub
END

# note: truncated ssh key for blog post
cat > /etc/puppet/modules/gitolite/files/lsimons.pub <<END
ssh-rsa AAAA...== lsimons@...
END

cat > /etc/puppet/modules/gitolite/files/gitolite-rc <<END
\$GL_PACKAGE_CONF = '/usr/local/share/gitolite/conf';
\$GL_PACKAGE_HOOKS = '/usr/local/share/gitolite/hooks';
\$REPO_BASE="repositories";
\$REPO_UMASK = 0077;         # gets you 'rwx------'
\$PROJECTS_LIST = \$ENV{HOME} . "/projects.list";
\$GL_ADMINDIR=\$ENV{HOME} . "/.gitolite";
\$GL_LOGT="\$GL_ADMINDIR/logs/gitolite-%y-%m.log";
\$GL_CONF="\$GL_ADMINDIR/conf/gitolite.conf";
\$GL_KEYDIR="\$GL_ADMINDIR/keydir";
\$GL_CONF_COMPILED="\$GL_ADMINDIR/conf/gitolite.conf-compiled.pm";
\$GIT_PATH="";
\$GL_BIG_CONFIG = 0;
\$GL_NO_DAEMON_NO_GITWEB = 0;
\$GL_NO_CREATE_REPOS = 0;
\$GL_NO_SETUP_AUTHKEYS = 0;
\$GL_GITCONFIG_KEYS = "";
\$HTPASSWD_FILE = "";
\$RSYNC_BASE = "";
\$SVNSERVE = "";
\$GL_WILDREPOS = 0;
\$GL_WILDREPOS_PERM_CATS = "READERS WRITERS";
1;
END

cat > /etc/puppet/modules/gitolite/manifests/init.pp <<END
class gitolite {
    package { git:
        ensure => latest
    }
    
    group { git:
        ensure => present,
        gid => 802
    }
    
    user { git:
        ensure => present,
        gid => 802,
        uid => 802,
        home => "/home/git",
        shell => "/bin/bash",
        require => Group["git"]
    }
    
    file {
        "/home/git":
            ensure => directory,
            mode => 0750,
            owner => git,
            group => git,
            require => [User["git"], Group["git"]];

        "/home/git/install-gitolite.sh":
            ensure => present,
            mode => 0770,
            owner => git,
            group => git,
            require => File["/home/git"],
            source => "puppet:///modules/gitolite/install-gitolite.sh";
            
        "/home/git/setup-gitolite.sh":
            ensure => present,
            mode => 0770,
            owner => git,
            group => git,
            require => File["/home/git"],
            source => "puppet:///modules/gitolite/setup-gitolite.sh";
            
        "/home/git/lsimons.pub":
            ensure => present,
            mode => 0660,
            owner => git,
            group => git,
            require => File["/home/git"],
            source => "puppet:///modules/gitolite/lsimons.pub";
        
        "/home/git/.gitolite.rc":
            ensure => present,
            mode => 0660,
            owner => git,
            group => git,
            require => File["/home/git"],
            source => "puppet:///modules/gitolite/gitolite-rc";
    }
    
    exec {
        "/home/git/install-gitolite.sh":
            cwd => "/home/git",
            user => root,
            require => [
                        File["/home/git/install-gitolite.sh"],
                        Package["git"]
                ];

        "/home/git/setup-gitolite.sh":
            cwd => "/home/git",
            user => git,
            environment => "HOME=/home/git",
            require => [
                    Exec["/home/git/install-gitolite.sh"],
                    File["/home/git/setup-gitolite.sh"],
                    User["git"]
                ]
    }
}
END

# update the site config to use the sudo and gitolite modules
cat > /etc/puppet/manifests/modules.pp <<END
import "sudo"
import "gitolite"
END

cat > /etc/puppet/manifests/nodes.pp <<END
node basenode {
    include sudo
}

node "puppet.sandbox" inherits basenode {
    include gitolite
}
END

cat > /etc/puppet/manifests/site.pp <<END
import "modules"
import "nodes"
END

# invoke puppet once to apply the new config
puppetd -v --test

So now we have gitolite installed on the server. So far so good.

2.3 Creating an scm git repo

I now need a repository in which to put the puppet configs. I was originally planning to have a repo ‘scm’ and a directory ‘puppet’ within it, so that I could have /etc/scm, with /etc/puppet a symlink to /etc/scm/puppet. It turns out puppet doesn’t support symlinks for /etc/puppet, so I ended up fiddling about a bit…

# on client
git clone git@puppet:gitolite-admin
cd gitolite-admin
cat >>conf/gitolite.conf <<END

repo    scm
        RW+     =   lsimons

END
git add conf/gitolite.conf 
git commit -m "Adding 'scm' repo"
git push origin master
cd ..

2.4 Making puppet use the config from the scm repo

First we need to get the existing config into version control:

mkdir scm
cd scm
git init
cat >> .git/config <<END
[remote "origin"]
    url = git@puppet:scm
    fetch = +refs/heads/*:refs/remotes/origin/*
http://master
    remote = origin
    merge = refs/heads/master
END
scp -r root@puppet:/etc/puppet/* .
git add *
git commit -m "Check in initial puppet config"
git push --all

Next, get the config out of version control and underneath puppet, and automate this process:

# install from-git puppet config on server

cd /etc
mv puppet puppet.bak
mkdir puppet
chown git puppet
cd puppet
sudo -u git git clone file:///home/git/repositories/scm.git puppet

# install post-receive hook to update /etc/puppet after a push

su - git
cat > /home/git/repositories/scm.git/hooks/post-receive <<END
#!/usr/bin/env bash
(cd /etc/puppet; env -i git pull -q origin master)
END
chmod u+x /home/git/repositories/scm.git/hooks/post-receive
exit

That little bit of env -i git inside that hook had me baffled for a bit. It turns out that I needed to empty the environment before invoking git from inside of a hook, because otherwise it’ll pick up the GIT_DIR variable. D’oh!

With this config re-set up, there should be effectively 0 changes when we run puppet. Let’s check:

puppet /etc/puppet/manifests/site.pp

2.5 On (not) putting all the puppet.sandbox config in puppet

Note that installing the post-receive hook from the previous step is not puppeted. The reason for this is one of synchronization. For example, if puppet somehow creates that post-receive file before the scm repository exists, gitolite will complain. It seems easier to have puppet not touch things managed by gitolite and vice versa.

Similarly, the installation of puppet itself is not puppeted, leaving the configuration of puppet.sandbox not something that can be completely automatically rebuilt.

Instead, rebuilding this box should be done by first re-following the instructions above, and then restoring the contents of the git@puppet:gitolite-admin and git@puppet:scm repositories from their current state (or latest backup). For my current purposes, that’s absolutely fine.

3. Setting up puppet dashboard

I also had a look at installing puppet dashboard. Because I know ruby and rails and gems can be a big dependency hell I figured I didn’t even want to try it in a VM, and instead I “just” got it running on my mac.

3.1 MySQL, ruby and mac os x

Puppet dashboard is built using ruby on rails and suggests using mysql for persistence (in retrospect I should not have listened and used sqlite :-)). Ruby on Rails apparently accesses MySQL through the mysql gem. The MySQL gem has to link against both the native mysql library and the native ruby library. Fortunately, I’m aware enough of the potential pain that I tend to carefully install the most compatible version of systems like this:

$ file `which ruby`
/usr/local/bin/ruby: Mach-O executable i386
$ file `which mysql`
/usr/local/mysql/bin/mysql: Mach-O executable i386
$ ruby --version
ruby 1.8.7 (2009-06-12 patchlevel 174) [i686-darwin9.7.0]
$ mysql --version
mysql  Ver 14.14 Distrib 5.1.31,
        for apple-darwin9.5.0 (i386) using readline 5.1

You’d hope that the mysql gem build picks up on all this ok, but that’s not quite the case. Instead, you really have to be quite explicit:

vi /usr/local/lib/ruby/1.8/i686-darwin9.7.0/rbconfig.rb
# change to CONFIG["ARCH_FLAG"] = "-arch i386"

sudo gem uninstall mysql
sudo env ARCHFLAGS="-arch i386" gem install --verbose \
    --no-rdoc --no-ri mysql \
    -- --with-mysql-dir=/usr/local \
    --with-mysql-config=/usr/local/mysql/bin/mysql_config
cd /usr/local/mysql/lib/
sudo ln -s mysql .
cd .

3.2 Look, ma, it’s a rails app

After this, fortunately it’s easy again.

tar zxf ...puppet-dashboard...
mv ... ~/puppet-dashboard
cd ~/puppet-dashboard
rake RAILS_ENV=production db:create
rake RAILS_ENV=production db:migrate
./script/server -e production
open http://localhost:3000/

Works like a charm. To get some data to see requires tweaking the puppet VM:

cd /Users/lsimons/puppet-dashboard/ext/puppet
vi puppet_dashboard.rb
# change HOST to 172.16.64.1
ssh root@puppet mkdir -p /var/lib/puppet/lib/puppet/reports
scp puppet_dashboard.rb root@puppet:/var/lib/puppet/lib/puppet/reports/
exit

cd ~/dev/scm/
vi puppet/puppet.conf
# report = true for [puppetd]
# reports = puppet_dashboard for [puppetmasterd]
git add puppet/puppet.conf
git commit -m "Enable reporting"
git push

4. Recap

So now we have:

  • A working, documented, repeatable process for creating new VMs
  • A working, documented, repeatable process for bootstrapping puppet
  • A neat version-controlled way of changing the puppet config
  • An installation of a puppet master that serves up the latest config
  • A puppeted installation of gitosis
  • A not-so-great but working installation of puppet dashboard
  • A few more VMs to configure

PowerDNS 29.2.22 on Mac OS X 10.5.8

March 15, 2010

Based on this hint and the official docs I got PowerDNS running on my mac.

Prerequisites

  • mac os x developer tools
  • mysql 5.0 or later (I’m using 5.1.31-osx10.5-x86)

Installation steps

  • Download boost library (I’m using 1_42_0) and extract
  • Download PowerDNS source distribution (I’m using 29.2.22) and extract
  • Compile and install:
$ CXXFLAGS="-I/Users/lsimons/Downloads/boost_1_42_0 -DDARWIN" ./configure \
    --with-mysql=/usr/local/mysql-5.1.31-osx10.5-x86 \
    --with-mysql-includes=/usr/local/mysql-5.1.31-osx10.5-x86/include \
    --without-pgsql \
    --without-sqlite \
    --without-sqlite3 \
    --prefix=/usr/local/pdns-2.9.22
$ make
$ sudo make install
$ cd /usr/local/pdns-2.9.22/etc
$ sudo cp pdns.conf-dist pdns.conf
$ vi pdns.conf
# look for the line #launch, just below add into pdns.conf:
#   launch=gmysql
#   gmysql-host=127.0.0.1
#   gmysql-user=root
#   gmysql-dbname=pdnstest
$ cd ../bin
$ sudo cp /Users/lsimons/Downloads/pdns-2.9.22/pdns/pdns .
$ sudo cp /Users/lsimons/Downloads/pdns-2.9.22/pdns/precursor .

Set up mysql database

Create pdns.sql:

CREATE TABLE domains (
    id              INT UNSIGNED NOT NULL PRIMARY KEY auto_increment,
    name            VARCHAR(255) NOT NULL,
    master          VARCHAR(128) DEFAULT NULL,
    last_check      INT DEFAULT NULL,
    type            VARCHAR(6) NOT NULL,
    notified_serial INT DEFAULT NULL, 
    account         VARCHAR(40) DEFAULT NULL,

    UNIQUE INDEX name_index (name)
) ENGINE=InnoDB;

CREATE TABLE records (
    id              INT UNSIGNED NOT NULL PRIMARY KEY auto_increment,
    domain_id       INT DEFAULT NULL,
    name            VARCHAR(255) DEFAULT NULL,
    type            VARCHAR(6) DEFAULT NULL,
    content         VARCHAR(255) DEFAULT NULL,
    ttl             INT DEFAULT NULL,
    prio            INT DEFAULT NULL,
    change_date     INT DEFAULT NULL,

    INDEX rec_name_index (name),
    INDEX nametype_index (name, type),
    INDEX domain_id (domain_id)
) ENGINE=InnoDB;

create table supermasters (
    ip              VARCHAR(25) NOT NULL, 
    nameserver      VARCHAR(255) NOT NULL, 
    account         VARCHAR(40) DEFAULT NULL
) ENGINE=InnoDB;

GRANT SELECT ON supermasters TO pdns;
GRANT ALL ON domains TO pdns;
GRANT ALL ON domains TO pdns@localhost;
GRANT ALL ON records TO pdns;
GRANT ALL ON records TO pdns@localhost;

Create pdns_sample_data.sql:

INSERT INTO domains (name, type) values ('test.com', 'NATIVE');
INSERT INTO records (domain_id, name, content, type,ttl,prio) 
    VALUES (1,'test.com','localhost ahu@ds9a.nl 1','SOA',86400,NULL);
INSERT INTO records (domain_id, name, content, type,ttl,prio)
    VALUES (1,'test.com','dns-us1.powerdns.net','NS',86400,NULL);
INSERT INTO records (domain_id, name, content, type,ttl,prio)
    VALUES (1,'test.com','dns-eu1.powerdns.net','NS',86400,NULL);
INSERT INTO records (domain_id, name, content, type,ttl,prio)
    VALUES (1,'www.test.com','199.198.197.196','A',120,NULL);
INSERT INTO records (domain_id, name, content, type,ttl,prio)
    VALUES (1,'mail.test.com','195.194.193.192','A',120,NULL);
INSERT INTO records (domain_id, name, content, type,ttl,prio)
    VALUES (1,'localhost.test.com','127.0.0.1','A',120,NULL);
INSERT INTO records (domain_id, name, content, type,ttl,prio)
    VALUES (1,'test.com','mail.test.com','MX',120,25);

Populate mysql database:

$ echo "CREATE DATABASE pdnstest" | mysql -uroot -e
$ mysql -uroot < pdns.sql
$ mysql -uroot < pdns_sample_data.sql

Run pdns

$ cd /usr/local/pdns-2.9.22
$ sudo bin/pdns start

Test

$ dig www.test.com @127.0.0.1
...
www.test.com.		120	IN	A	199.198.197.196
...