So what? Give it 2 me! It’s pronounced walruses.

Some of my favorites on the new joost:

• Pink - So What
• Green Day - Boulevard of Broken Dreams
• Madonna - Give It 2 Me
• Beyonce & Shakira - Beautiful Liar
• Tooth & Claw Episode 1: WALRUSES

subscribe to my joost feed

Let us know what you think of the new joost!

how to install python MySQLdb on mac os x 10.5 (leopard)

I was happy to find that someone’s already posted instructions on how to install mysql-python on mac os x 10.5.

how to install lxml python module on mac os 10.5 (leopard)

lxml is an xml library for python that doesn’t suck. It needs a recent libxml2 and libxslt. Mac OS X does not come with recent versions, and 10.5 breaks completely if you force it to try and use a recent version.

I used to use MacPorts for everything (which comes with py25-lxml) but ran into some issues with 10.5. So back to manual installs, it is.

• assuming your python is the mac os x default…
• make sure no traces of other pythons in your $PATH
• download and install libxml2:

     ./configure --prefix=/usr/local/libxml2-2.7.0
     make
     sudo make install
     cd /Library/Python/2.5/site-packages
     sudo ln -s /usr/local/libxml2-2.7.0/lib/python2.5/site-packages/* .
  

• download and install libxslt:

     ./configure --prefix=/usr/local/libxslt-1.1.24 --with-libxml-prefix=/usr/local/libxml2-2.7.0
     make
     sudo make install
     cd /Library/Python/2.5/site-packages
     sudo ln -s /usr/local/libxslt-1.1.24/lib/python2.5/site-packages/* .
  

• download and install lxml:

     sudo python setup.py install \
       --with-xml2-config=/usr/local/libxml2-2.7.0/bin/xml2-config \
       --with-xslt-config=/usr/local/libxslt-1.1.24/bin/xslt-config
  

Open source product-centric business model

1. Build a community around a specific free thing.
2. With that community’s help, design some products that people want, and return the favor by making the products free in raw form (source code).
3. Let those with more money than time/skill/risk-tolerance buy the more polished version of those products. (That may turn out to be almost everyone)
4. Do it again and again, building a 40% margin into the products to pay the bills.

(Adapted slightly from The time/money formula of free).

That pretty much summarizes the business model for sleepycat, MySQL, etc. You could also call it the “provide free lunch for volume, with a fractional but large up-sell” model.

Simulating erratic failures in python

Imagine you’re using some library for interfacing with MySQL from python. And MySQL is dying from under your application. And you’re not dealing with it gracefully. So you get greeted with:

OperationalError: (2006, 'MySQL server has gone away')

If you re-start your app, the problem will go away for a bit. So how can you test (unit test, even) that you’re degrading gracefully?

Here’s one example. Rather more clean and generic and simple than what you typically do in something statically typed like java.

import random
from MySQLdb import OperationalError

class Store:
    def get_val(self, a):
        return a.lower()
    def execute(self, *args, **kwargs):
        return "OK"

class ErroringStore:
    """Wrapper around something that has an execute(…) to make it error."""
    def __init__(self, delegate):
        self.delegate = delegate
    def __getattr__(self, name):
        if name == "execute":
            if int(random.random()*2) == 1:
                def execute_error(*args, **kwargs):
                    raise OperationalError(2006, ‘MySQL server has gone away’)
                return execute_error
            else:
                return getattr(self.delegate, name)
        elif hasattr(self.delegate, name):
            return getattr(self.delegate, name)
        raise AttributeError, name
    def __setattr__(self, name, value):
        if name == "delegate":
            self.__dict__[name] = value
        else:
            setattr(self.delegate, name, value)

store = Store()
erroring = ErroringStore(store)

# these are ok
erroring.get_val("BLAAT")

# this doesn’t exist
try:
    erroring.nomethod()
    raise "should result in AttributeError"
except AttributeError:
    pass

# this sometimes throws an exception
errors = 0
for i in range(1, 100):
    try:
        erroring.execute("SELECT TRUE;")
    except:
        errors += 1
print "Failed %d times out of 100" % errors

my day job now involves world of warcraft

Joost (where I work) is working with SK Gaming on doing live coverage of world of warcraft raids (announcement).

I haven’t quite convinced anyone yet that this means that actually playing world of warcraft should be part of my job description, but I do get to watch SK in-game footage as part of testing in-development versions of our software!

Working a bit on this stuff really makes me want to get back into raiding. If only I could find the time! In the meantime this is probably the closest I’ll get…

So, in the meantime I’m hoping we can get actual live coverage of some powerleveling (which I once tried to get good at). Apparently some people can do 1-70 in two days now! S(l)ick.

Can I connect my console to my apple cinema display?

No.

XBOX 360 and PS3 both supply a HDMI signal.

Apple’s cinema displays only accept a digital DVI signal.

There is no way to convert the HDMI signal from xbox into a DVI signal.

One company (forget which one) sold a special HDMI-to-DVI converter box for a bit, but they’re no longer selling it and aren’t intending to start selling something like it again.

If you’re buying a LCD computer display, buy one that has a HDMI input, or at least composite or VGA.

(figured I’d put this out here since it’s the 2nd time someone asked me, and I remember being really frustrated when I couldn’t get it to work)

Serious PHP, part 1

I used PHP 3 a lot back in the day, and I built some seriously interesting stuff with PHP 4 before moving on to what was then consider more “serious” technology. Recently I’ve been looking at PHP again, just to figure out how the famous LAMP stack works these days. Care to follow along?

In the below I assume you’re on Mac OS X and you sort-of know your way around unix.

Installing a bunch of packages under Mac OS X

We’re going to use a whole bunch of stuff…let’s get the package in one go (hint: you can install multiple ports at a time, see man port)

• Install MacPorts
• Install apache2:
  sudo port install apache2
• Install MySQL 5:
  sudo port install mysql5
  (You will need to follow some on-screen instructions to finish configuration)
• Install SQLite:
  sudo port install sqlite3
• Install Perl 5.8:
  sudo port install perl5.8
• Install Apache-Test:
  sudo port install p5-apache-test
• Install PHP 5:
  sudo port install php5 +apache2+mysql5+pear+sqlite
(you may want to add some more options, see what’s available using port variants php5; You may also need to follow some on-screen instructions to enable the extension within apache)
sudo cp /opt/local/etc/php.ini-recommended /opt/local/etc/php.ini
• Configure pear:
  sudo pear config-set php_ini /opt/local/etc/php.ini
sudo pear upgrade-all
sudo pear channel-update pear.php.net
sudo pear channel-update pecl.php.net
• Install APC:
  sudo pear install APC
• Install Memcached:
  sudo port install memcached
• Install memcache for python:
  sudo port install php5-memcache
  (You will need to follow some on-screen instructions to enable the extension)

Setting up MySQL replication

MySQL replication is a technique that helps with scaling up your application. Sooner or later your database tends to become a bottleneck, and in that case what you do is make sure your SELECT statements go to database slaves. It is easier to get this right from the start, rather than re-write your application later. So, set up some replication locally…Buy and read High Performance MySQL to get a good grasp for the details; it’s the best book on the subject.

After you’ve RTFMed a bunch, assuming you have a basic mysql install set up…here’s the magic…

# set the root password
mysqladmin -u root password 'blaat'

# shortcut
my="mysql -uroot -pblaat"

# db we'll be using
echo "DROP DATABASE IF EXISTS test;" | $my
echo "CREATE DATABASE test;" | $my
echo "GRANT ALL
  ON test.* TO 'test'@'%'
  IDENTIFIED BY 'test'" | $my

# prep for replication
echo "GRANT
    REPLICATION SLAVE,
    REPLICATION CLIENT
  ON *.* TO 'repl'@'%'
  IDENTIFIED BY 'blaat'" | $my

# I always manage to forget this one...
echo "FLUSH PRIVILEGES;" | $my

# enable binary logging and set server-id on to-be-master
sudo cat > /opt/local/etc/mysql5/my.cnf <<END
[mysqld]
log-bin
server-id = 1
END

# create config for first slave to run on port 3307
sudo cat > /opt/local/etc/mysql5/my_3307.cnf <<END
[mysqld]
server-id = 2
master-host = 127.0.0.1
master-user = repl
master-password = blaat
master-port = 3306
port = 3307
socket = /opt/local/var/run/mysql5/mysqld_3307.sock
pid-file = /opt/local/var/db/mysql5_3307/my.pid
datadir = /opt/local/var/db/mysql5_3307
log-error = /opt/local/var/db/mysql5_3307/my.error.log
END

# create config for second slave to run on port 3308
sudo cat > /opt/local/etc/mysql5/my_3307.cnf <<END
[mysqld]
server-id = 2
master-host = 127.0.0.1
master-user = repl
master-password = blaat
master-port = 3306
port = 3308
socket = /opt/local/var/run/mysql5/mysqld_3308.sock
pid-file = /opt/local/var/db/mysql5_3308/my.pid
datadir = /opt/local/var/db/mysql5_3308
log-error = /opt/local/var/db/mysql5_3308/my.error.log
END

# stop the current mysql database
sudo killall mysqld_safe

# remove any previous/old binary log files
rm /opt/local/var/db/mysql5/*-bin*

# initialize data directories for the slaves
mkdir /opt/local/var/db/mysql5_3307
chown mysql:mysql /opt/local/var/db/mysql5_3307
rsync -av /opt/local/var/db/mysql5/ /opt/local/var/db/mysql5_3307/
mkdir /opt/local/var/db/mysql5_3308
chown mysql:mysql /opt/local/var/db/mysql5_3308
rsync -av /opt/local/var/db/mysql5/ /opt/local/var/db/mysql5_3308/

Pfoeey. Let’s see if that worked (save the below to a script file):

#!/usr/bin/env bash
# starts 3 mysql servers
sudo mysqld_safe >/opt/local/var/db/mysql5/my.out 2>&1 &
sudo mysqld_safe --defaults-file=/opt/local/etc/mysql5/my_3307.cnf \
  >/opt/local/var/db/mysql5_3307/my.out 2>&1 &
sudo mysqld_safe --defaults-file=/opt/local/etc/mysql5/my_3308.cnf \
  >/opt/local/var/db/mysql5_3308/my.out 2>&1 &

Let’s see if we got replication going:

echo "CREATE TABLE repl_test ( id int(11) PRIMARY KEY );" \
    | $my test
tail /opt/local/var/db/mysql5_3307/my.error.log
tail /opt/local/var/db/mysql5_3308/my.error.log

Setting up Memcached

Memcached needs essentially no setup. Try this (save the below to a script file):

#!/usr/bin/env bash
# starts 2 memcached servers
hostname=127.0.0.1
memperdaemon=128
daemonport1=11211
daemonport2=11212

memcached -d -l $hostname -m $memperdaemon -p $daemonport1
memcached -d -l $hostname -m $memperdaemon -p $daemonport2

Setting up Apache to use virtual hosting

We’ll have apache listen on a separate port where our app can be all by itself without getting disturbed by your other dev projects that invade its URL space.

• mkdir ~/doodle
• mkdir ~/doodle/htdocs
• Edit
/opt/local/apache2/conf/httpd.conf
look for the line
  #Include conf/extra/httpd-vhosts.conf
  and un-comment it.
• Then change that file:

sudo cat >/opt/local/apache2/conf/extra/httpd-vhosts.conf <<END
NameVirtualHost *:80
<VirtualHost *:80>
DocumentRoot "/opt/local/apache2/htdocs"
</VirtualHost>

Listen 8810
NameVirtualHost *:8810
<VirtualHost *:8810>
    ServerName localhost:8810
    DocumentRoot $HOME/doodle/htdocs
    <Directory "$HOME/doodle/htdocs">
      Options Indexes FollowSymLinks
      AllowOverride None
      Order allow,deny
      Allow from all
    </Directory>
</VirtualHost>

• Test the config:
sudo /opt/local/apache2/bin/apachectl configtest
  then start (restart if started)
sudo /opt/local/apache2/bin/apachectl stop
sudo /opt/local/apache2/bin/apachectl start

Make pecl/filter more pendantic

This is a pretty awkward PHP extension that implements user-submitted variable sanitization rather efficiently. Take cross-site scripting seriously, and protect against it!

• Edit
  /opt/local/etc/php.ini
  look for the line
[filter]
  and add
filter.default = “special_chars”
filter.flags = FILTER_FLAG_ENCODE_HIGH
• Restart apache.

When are we going to do any PHP?

Let’s see if part 2 has more actual PHP in it. The above list of dependencies should give you some idea of where I’m going :-). For now try this:

cat > ~/doodle/htdocs/phpinfo.php <<END
<?php
    phpinfo();
?>
END

open http://localhost:8810/phpinfo.php

how to serve decent drinks

Look what I found on joost (disclaimer: I work there) hidden away in the cooking category:

• how to store wine
• how to serve wine
• making margaritas
  
• making cosmopolitans
• making mojitos
• making bloody maries

oh, and once you’re past that first hangover…

• lots of margarita recipes

feedburner messing up my feed??

I’ve tried to disable flickr consumption for now. How weird. Thanks to ColmMac for reporting