A couple of weeks ago we brewed our first ever batch of beer. Rather than start with something simple like an extract brew, I decided to create my own all-grain American IPA recipe. The beer is ready to be bottled now, then it will need another 2 weeks in the bottles for conditioning. By mid-January we will finally be able to taste it, hopefully it will at least resemble drinkable beer.

Static http file server in Go

Go is a great language for building network based applications. It comes with some excellent tools for creating web-apps out of the box.

I often want to create a “simple http server” to serve up the current directory, usually I reach for python -m SimpleHTTPServer, but in the spirit of re-inventing the wheel I decided to see how Go could handle this task.

It turned out to be remarkably simple. Go comes with a static file server as part of the net/http package, in this example I’ve added a couple of flags that allow specifying the port and the root filesystem path for the process.

// httpserver.go
package main

import (

var port = flag.String("port", "8080", "Define what TCP port to bind to")
var root = flag.String("root", ".", "Define the root filesystem path")

func main() {
	panic(http.ListenAndServe(":"+*port, http.FileServer(http.Dir(*root))))

The actual meat of the program is the second line inside the main function. http.ListenAndServe accepts an address to listen on as the first argument, and an object which implements the http.Handler interface as the second, in this case http.FileServer. If ListenAndServe returns an error (most likely because another process is using the desired port) then the process will panic and exit.

If you’ve got Go installed then this can be run directly.

$ go run httpserver.go

Or you can compile it to a standalone binary.

$ go build httpserver.go
$ ./httpserver

The file server implementation that Go provides even handles serving index.html from a directory if no file is specified, and provides a directory listing if there is no index.html present.

For more details check out Go’s implementation of http.FileServer.

The code shown in this article is available on GitHub.

Raspberry Pi GPIO Hacking

I’ve been playing with my Raspberry Pi starter kit today. It comes with a clear plastic case for mounting the Pi onto, but as I’ve already got a case I’m just using the breadboard and the components that were supplied with the kit. As well as the breadboard the kit includes the following components:

  • 12 × LEDs (4 each of Red, Yellow and Green)
  • 12 × 330Ω resistors (for the LEDs)
  • 2 × 10kΩ resistors
  • 2 × mini push buttons
  • 10 × male to female jumper wires

The example I was following is for a simple traffic lights system. The hardware components are wired into the Raspberry Pi board using its GPIO pins. These General Purpose Input Output pins can be controlled using software so they provide a simple way to connect to external hardware to the Pi.

In addition to the familiar USB, Ethernet and HDMI ports, the R-Pi offers lower-level interfaces intended to connect more directly with chips and subsystem modules. RPi Low-level peripherals introduction

Once I got the traffic lights wired up and working I started hacking on the software side of things. The example python code didn’t cleanup the GPIO when I pressed ctrl-c, so the lights remained in the position they were in when I interrupted the process. To fix this I installed a signal handler to run the GPIO cleanup code and exit cleanly.

def cleanup_gpio(signal, frame):

# Install signal handler to cleanup GPIO when user sends SIGINT
signal.signal(signal.SIGINT, cleanup_gpio)

Raspberry Pi plugged into breadboard

The code I’ve been tinkering with is on GitHub. This includes the example C code, which is pretty much untouched other than adding a limit to the number of loops so that the GPIO cleanup code gets run. There’s also the example python code which is what I’ve mainly been playing with.

There is a file in the repository that flashes the lights in sequence which is essentially a modified version of the traffic lights script.

For my first attempt at Raspberry Pi hacking this was very successful, it’s a real buzz to see hardware and software coming together in something of your own making.

The next stage of this project will involve linking this traffic light system into a web service to create a status indicator.

Slow dotfiles

For a while now my shell has been taking a very long time to start up. It wasn’t so noticeable on more powerful machines, but on my late 2010 MacBook Air with “only” 2GB of RAM, it was very noticeable.

Last weekend I decided to finally sit down and figure out what was causing the slow startup times. I stepped through the startup process in my dotfiles, commenting out lines until it was fast again, it didn’t take long to find the offending line in my zshrc.

for module ($DOTFILES/**/*.zsh) source "$module"

This line uses shell globbing to load up all the files that end in .zsh in the dotfiles repo, its pretty central to the way the dotfiles work, without it no custom zsh scripts are loaded. So how could this line be causing such a slow startup?

The way globbing works is it expands the pattern into a list of files. In this case it checks all the files in the dotfiles directory to see if they end with .zsh.

This shouldn’t be a problem, but I had configured my vim plugins to be installed into a directory within the dotfiles repo, which had its contents gitignored, you can see the commit on GitHub. This meant I could install vim plugins without having to track them in the repo. So the shell globbing was checking all of the files in the vim plugins as well as the dotfiles.

To solve this problem I spent an hour removing my vim configuration from my dotfiles and putting it into its own repository. I’d wanted to do this for a while anyway because I found myself changing my vim configuration a lot more than anything else in the dotfiles.

I made this change on my faster work laptop, which doesn’t suffer as much with the slow startup. When I got round to pulling the changes down to my Air the difference was very noticeable, new shells pop into existence in under a second, like they should.

It’s been 6 months since I made the commit which began the slowdown. I’ve been putting up with slow shell startups for all that time, when actually it only took a couple of hours in total to identify and fix the problem.

So don’t be like me and ignore performance problems when you know they’re there, take the time to fix them

Removing code

If we wish to count lines of code, we should not regard them as “lines produced” but as “lines spent”: the current conventional wisdom is so foolish as to book that count on the wrong side of the ledger. Edsger W. Dijkstra

I take great pleasure in reducing the size of a codebase.

When you first start a project there is a lot of code churn. As your understanding of a problem changes the code changes and grows with it.

As time goes on you need to remove code that is no longer used. One strategy that I often see is commenting the offending code out. This adds a lot of noise to the code when you’re reading it, a much better strategy is to just remove the unused code, if you need it back you can ask your VCS.

When you don’t take the time to remove old code and features, it’s often more confusing when you come to add new code and features.

Resque Rails Auth

Recently I found myself wanting to access the resque-web ui on a live application. I had considered just running resque-web as a separate process, but after reading this article I realised that I could mount resque directly in the router, awesome!

However, the application doesn’t use devise for authentication, so I wanted an easy way to restrict resque-web to admins.

Using rails 3 router’s advanced constraints you can pass a :constraints options with an object that responds to matches? and receives the current request as an argument.

Since the current user’s id is stored in the session, we can simply retrieve the user and check if they’re an admin.

class AdminRestriction
  def self.matches?(request)
    user_id = request.env['rack.session'][:user_id]
    user = User.find_by_id(user_id)
    return user && user.admin?

MyApplication::Application.routes.draw do
  mount Resque::Server => '/resque', :constraints => AdminRestriction
  # Other application routes.

The AdminRestriction class performs the actual checks, in the router it is simply passed as a constraint.

First we pull the user_id out of the session, then we attempt to get the user from the database. Finally we check that we’ve found a user and that they are an admin.

If the user tries to access /resque and they are not an admin, they simply get a 404 error.

This technique can be used with any rack application, or indeed with any regular route, just pass a :constraints option (see the match method docs), and the constraints that you apply can use any part of the request to decide if it matches. You can restrict access by ip address, or do routing based on the request’s geolocation.

The possibilities are endless.

Test Driven Development Lifecycle

Start at the top level with a user requirement. This will ensure that you are trying to solve the right problem in the first place.

Cucumber / Steak

Write a simple high level requirement in cucumber or steak. For example if you were writing an api and you wanted to allow people to post activity.

First you would write out the requirement in plain english.

Feature: Creating activity
  As an activity producer
  I want to post activity to the api
  In order to share it with the world

  Scenario: POST to /activity
    Given I am an authorized activity producer
    When I post some activity
    Then I should receive a 200 success status

This defines the high level picture of what the application aims to achieve. From this point the next step is to write some steps to properly test that the features are working.

Obviously the features won’t be working yet, because that’s not how test/behaviour/real-world driven development works. If you start writing code before you really know what you are building, then it is likely that you will at some point, perhaps without realising, implement the wrong feature. More likely though, being that smart martin that you are, you will implement the correct feature, but unwittingly leave a bug in it. No harm, you say, bugs happen in software and it’s probably only one line of code that needs changing. But if the bug doesn’t manifest itself for a few days, weeks, months, then it will take an ever increasing amount of time for you to track it down. Then once you have isolated it, you have to ensure it won’t happen again.

This all comes down to testing. If you can easily run a bank of tests that confirm that your application is behaving at night you will, infact, sleep better at night. For some this is reason enough to write tests, but wait, there’s more. If you have a comprehensive collection of tests you can use to verify that your application is working correctly, then you are in a very good position to do some refactoring. No more worrying if you’ve broken some seemingly unrelated piece of functionality when you make a change to the application.

Controller tests

The next layer down the stack is the controller, this makes sense, since this is the layer that manages incoming requests. So after writing the initial acceptance test in a high level language, then implementing the steps necessary to test this logic in a slightly lower level language, you now drop down to a relativly granular level. This is where you start to stub out the main functionality of the application. So in the case of our activity example above, you would at this stage be stubbing out the functionality of the models, and just dealong with the way that controllers handle requests.

This layer may be preceeded by another, slightly higher level one in which the routing for the application is set up, however this is quite a web application specific area.

Model tests

The model tests are the core of the testing universe, they are very small, low level details about the various methods that a model provides.

The model tests are the most important tests, as they contain the business logic for your application. But they are often also the easiest tests to write. This is because a lot of the time you will be dealing with primitive data types. No external services to worry about. This is a good reason to push as much logic into the model as possible, it is far easier to test this that it is to put it in your controller then try to stub it out.

Other tests

All of this testing malarkey is great and everything, but sometimes you have got to just use the product to uncover random bugs, but don’t worry, that’s not to say we can’t still use testing to help us. Instead of jumping right in and fixing that bug like a good boy scout hacker, write a failing test that reproduces the bug. This test may exists as (m)any layer(s) of the test stack. Often you will need to write a test in the acceptance layer to perform the same interaction that caused the bug, then once that has caught the bug, go down the layers and write tests for the code paths that the bug touches, these failing tests will then (all going to plan) be green once you have fixed the bug, and as a billy bonus you’ve got yourself another test or two that will ensure many good nights sleep in the future.

Final thoughts

The testing pattern of development is still not used as a default development technique. Many (most) tutorials you find out there will focus on the functionality that they are trying to tutor you on, but a vital part of implementing any functionality is to ensure a long lifespan by testing it thoroughly.

Testing will seem like a chore at first, but once you get the hang of it, the whole process becomes a pattern, that if followed with a small bit of dicipline, will lead you to enlightenment, it will allow you to model your ideas with more structure with the tools available.

In many ways it’s like learning a new language, with more languages, you see more ways of doing things. With testing, you’ll see new ways to express your applications logic using high level language.

subscribe via RSS