Continuous Integration with CIJoe

CIJoe lacks a great many things that I usually expect in a CI server.  But it is now unclear if they are really all that needed, or just added by CI marketing.  Here is now I usually get CIJoe working with the least Effort.

The abridged version:

# rvm
$ bash < <( curl http://rvm.beginrescueend.com/releases/rvm-install-head )
$ rvm install 1.8.7-p330
$ rvm use 1.8.7-p330@foo --create
$ gem install bundler
$ gem install cijoe
$ git clone git://github.com/XXX/foo.git
$ nohup cijoe -p 4000 foo &
$ git config --add cijoe.runner "bundle install 2>&1 && rake ci"

  1. Install RVM
Docs on installing RVM are here.  I use this to isolate my version of Ruby and Gems from other CIJoe servers that I will setup on the same machine.  This reduces the headaches later

1.1. Install Ruby
With RVM installing new versions of ruby is easy with RVM.  For example I use ruby 1.8.7-p330 on one of my projects so I run rvm install 1.8.7-p330.

1.2. Create a Gemset
For every CI Server I run I use a different gem set.  Yes it uses more disk space but it saves so many headaches if two repos need different versions of different gems it is worth it.  You can run it with the "rvm use" command.  Lets say my project is called "FOO" I would run rvm use 1.8.7-p330@foo --create to switch to ruby 1.8.7-p330 using gemset foo and create the gemset if it doesn't already exist.

1.3. Install Bundler
Bundler is a manager for RubyGems that aids in getting the correct version of the correct gems installed.  I install it so that before every make I run "bundle install" to make sure that the CI server has all the gems it needs for the test to pass.  And if it fails because of a gem problem then it means the correct gems were not noted in the GemFile and therefore it will break in production.

2. Install CIJoe
CIJoe is a gem and it should be installed in the same gemset that the repo will be run out of.  So if the code to test is Foo then run rvm use 1.8.7-p330@foo --create; gem install cijoe.

2.1. Clone your repo
git clone git://github.com/XXX/foo.git or something like that

2.2. Run CIJoe
I run it with nohup and & so it is in the background and always running.  I also give it a known port so that I blow a hole in the firewall for it (see below for why).

nohup cijoe -p 4000 foo &

3. Configure your repo
This is both a strength and weakness of CIJoe: all the configuration for what to run and how is stored in the Git repo.  The strength is that it is stored with the repo, the weakness is that .git/config and .git/hooks are not pushed to remote repos so if you need to start over you will need to recreate that config from scratch.

3.1. Add the runner command
CIJoe only allows running of a single command but it executes it in "sh" so you can use "&&" to chain them.  However, to make things easier on myself I always create a rake task called "ci" and execute that. Also, to make sure that all the gems are correctly installed I run bundler first.

Note: When chaining commands via && make sure you redirect stderr to stdout since CIJoe only reads stdout, otherwise you will get nothing in the output file.  The last command will already have stderr redirected so do not do it twice.

git config --add cijoe.runner "bundle install 2>&1 && rake ci"

3.2. Adding Notification
CIJoe just executes .git/hook/build-failed on a failure and .git/hook/build-worked on a success.  The examples show how to setup these up.

4. Auto build
Any HTTP PUSH to CIJoe will cause a build to be kicked off.  This is useful for auto-kicking a build when a new commit is pushed.

4.1. Blow a hole in the Firewall
This is so that github can get into your build server, since I assume you have your build server behind a firewall.  Your config will depend on Firewall/Router.

4.2. Configure Github
Just add a Post-Receive hook via the github admin screen.

BDD and TDD

Apparently I have been doing TDD wrong for years (who knew?).  There are plenty of resources out there to help, but here are the ones I found useful in the order of most to least useful.

1. BDD - TDD done well?  Basically explained away my misconceptions about BDD.  I thought it a replacement to TDD (unit testing) and it is not.
2. Roman Numeral Kata in Ruby.  I had thought the point of testing was to test working code and never really bridged to understand how good code comes from bad code that is tested.  The video shows by adding tests that define the "correct" code 1 at a time and coding only to the current tests that elegant solutions are likely.
3. TDD Problem.  It is nice to read articles and watch movies, but until you have actually attempted to solve a real life TDD problem things are not going to make sense.  This site is full of problems.
4. Gherkin.  This is the language used by Cucumber to define BDD.
5. How I Learned To Love Testing.  Demo of by testing up a test env things are easier.
6. How to adopt TDD and ensure adherence?  The question posed is strikingly similar to oppositions that I have heard in the past; mostly stemming from misconceptions about testing.  The selected answer is well formed, and is the main reason to read the question.
7. Ruby Koans.  Helps people learn ruby by presenting tests that all fail and requiring the user to code until they pass.  The tests are very basic, but it is a good foray into TDD
8. Cucumber Tutorials and Related Blog Posts.  Just a list of other resources.

Sinatra Proxy

Anyone that has developed for the web has come accross the Same Origin Policy problem.  Basically it says that for an AJAX call to be valid it must come from the same host as the page, using the same port, subdomain, and protocol.  I find there are many cases where I need to gather information from remote hosts, which leaves me two choices: proxy the request through my server, or use an iframe.

In general since it is JSON/XML data I am interested in then an iframe will not work so I have to create a proxy.  Below is what I do using Sinatra.

Required Gems

1. Sinatra
2. Net::HTTP
3. CGI
4. URI

Directory Structure

This is just the default that I use for all sinatra apps because it makes things easier.  Sinatra will automatically serve files in the public directory if a file by that name exists

• index.html - static file that includes all other files
• public - place where other static files will go
• javascript
• stylesheets
• images
• server.rb - the sinatra app

Server.rb

/remote/*
The * tells sinatra to blindly take everything up to the "?" and put it in params[:splat].  I then split out the first item for the server name and port.  You could also split out username/password if you want, but I rarely use HTTP Basic Auth so I never do.

I delete the key 'splat' and anything I do not want to forward so that they are not URL encoded to the remote.

urlencode
A simple function that re-encodes params that were placed in the params hash.  Sinatra does a good job of taking params that look like color[]=red&color[]=blue and turning them into {"color" => ["red","blue"]}.  Therefore I check the value's class for Array and if so I add the "[]" to the param name.

Proxy
The proxy code is a call to Net::HTTP returning the response body.  I could also have returned the headers, but in this case I do not care.



In Action

/remote/medic/this	->	/medic:80/this
/remote/medic:4567/this	->	/medic:4567/this
/remote/medic:4567/this?_dc=1	->	/medic:4567/this
/remote/medic/this?foo=bar	->	/medic:80/this?foo=bar
/remote/medic/this?foo[]=bar&foo[]=baz	->	/medic:80/this?foo[]=bar&foo[]=baz