Jon's Blog

Multiple Knife Environments

2010-07-12T00:00:00Z

At work we have two Chef environments, staging and production, so that we don’t break our entire infrastructure because of a typo. Changes go to staging, and once we know they work we push to production.

To do that, I use a Knife config file that looks something like this:

CHEF_BASE = "/path/to/svn_root"
CHEF_ENV = ENV['CHEF_ENV'] || "staging"
case CHEF_ENV
  when 'staging'
    CHEF_SERVER = "https://chef.staging.example.org:443"
    CHEF_REPO = "#{CHEF_BASE}/trunk"
  when 'production'
    CHEF_SERVER = "https://chef.prod.example.org:443"
    CHEF_REPO = "#{CHEF_BASE}/branches/production"
end

client_key               "#{ENV['HOME']}/.chef/#{CHEF_ENV}.pem"
validation_key           "#{CHEF_REPO}/config/keys/#{CHEF_ENV}.pem"
chef_server_url          CHEF_SERVER
log_level                :info
log_location             STDOUT
node_name                `whoami`.chomp
validation_client_name   "chef-validator"
cache_type               'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path            ["#{CHEF_REPO}/cookbooks", "#{CHEF_REPO}/site_cookbooks"]

knife({
  :rackspace_api_username => "username",
  :rackspace_api_key  => "api_key",
  :environment => CHEF_ENV
})

Using that, by default any knife command I run will go to the staging server, but if do this: CHEF_ENV=production knife cookbook upload -a then it will go to the production server instead.

This does depend on some conventions we have in place in our Chef repository. As you can see, we keep validation keys in SVN, and each user’s client is named after their user login.

Footnote

How do we know they work? Nagios, which under the staging environment is effectively used as a test suite – if everything stays green, then it’s considered to be working.

Starting a Testable Sinatra Application

2010-07-04T00:00:00Z

I’m getting into using Sinatra for smaller web applications, instead of the behemoth that is Rails.

For many purposes, Rails is still perfect, but I often find that it can be overkill if all you need is to display a couple of pages, and possibly provide an API. Here’s a quick guide on creating a Sinatra application that can be tested using RSpec and Cucumber.

Install Bundler, and Create a Gemfile

Bundler lets us manage the list of gems that our application depends on. I’m using it on all new projects now, so that I don’t have to spend half an hour remembering what needs to be installed before starting.

gem install bundler
cd /path/to/example_app
bundle init

That will create Gemfile in the example application’s root directory, which we can use to specify the gems our application needs. Put the following in it for now:

source :gemcutter

gem "sinatra"
gem "unicorn"
gem "haml"

group :test do
  gem "cucumber-sinatra"
  gem "cucumber"
  gem "capybara"
  gem "rspec"
end

Now when you run bundle install it’ll install any dependencies that are required.

Create a Stub Application

This is going to be a really simple stub application for now, since this article isn’t really trying to teach you how to use Sinatra.

Put the following in lib/application.rb.

require 'sinatra/base'
require 'haml'

class Application < Sinatra::Base
  set :app_file, __FILE__
  set :inline_templates, true

  get '/' do
    haml :index
  end
end

__END__
@@ index
!!!
%html
  %head
    %title A test application
  %body
    %h1 Hello, world!

Just to check that it’s all working, dump this in config.ru, which will load everything using Bundler.

$: << "lib"

require 'application'
run Application

And run it with bundle exec unicorn – loading bundle exec ensures all the gems specified have been loaded from the appropriate location before running the application itself.

Add a Slice of Cucumber

Now we’re going to make this testable with Cucumber:

bundle exec cucumber-sinatra init Application lib/application.rb

That will tell you about the files it’s generating. Lets have a look at features/support/env.rb which is where Cucumber gets configured. You’ll see that it simply loads application.rb and then tells Capybara (the component that actually runs web apps) that the application under test is Application.

Lets write a feature to test that our simple application is in fact working. Put this in features/hello.feature.

Feature: Welcoming new developers
As a software developer
I want the world to be welcomed
So I get a fuzzy feeling of success

Scenario: Loading the welcome page
  When I go to the home page
  Then I should see "Hello, world!"

When you run it with bundle exec features/hello.feature it should pass.

Cucumber Profiles

By using profiles we can make use of the WIP (Work in Progress) tag to reduce the time it takes to run our features when we’re only interested in one or two features.

Start by creating cucumber.yml and put the following in it:

<% common = "--strict features" %>
default: --format progress <%= common %>
wip: --format pretty --tags @wip <%= common %>
ok: --format pretty --tags ~@wip <%= common %>

This will provide you with three profiles, default, wip, and ok for different tasks. default will run all features, in progress mode – this is probably the one you want to use for continuous integration.

wip and ok are opposites, with wip running anything with the @wip tag applied, and ok doing the opposite.

Now lets create some rake tasks to run them – put this in Rakefile:

require 'cucumber'
require 'cucumber/rake/task'

namespace :features do
  Cucumber::Rake::Task.new(:all) do |t|
    t.profile = "default"
  end

  Cucumber::Rake::Task.new(:ok) do |t|
    t.profile = "ok"
  end

  Cucumber::Rake::Task.new(:all) do |t|
    t.profile = "wip"
  end
end

Now when you run the rake tasks you’ll get the appropriate features run.

And now for some RSpec

Finally, we’re going to set up RSpec. This probably won’t be at all unusual to you if you’ve done it before, but I’m going to demonstrate it as well.

In your Rakefile:

require 'spec/rake/spectask'

namespace :spec do
  desc "Run all examples"
  Spec::Rake::SpecTask.new('spec') do |t|
    t.spec_files = FileList['spec/**/*_spec.rb']
  end
end

Then put this in spec/spec_helper.rb to make sure anything in lib/ can be loaded correctly:

$: << File.expand_path(File.dirname(__FILE__) + '/../lib')

And that’s about it. Create a spec directory, and start filling it up with specs ending in _spec.rb, like this one (in spec/application_spec.rb):

require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
require 'application'

describe Application do
  it { should_not be_nil }
end

Now when you run rake spec:all if all went well you’ll see a passing spec.

On With the Code

That’s as far as I’m going to go with this for now. Go write some code.

You can find the end result of this in the sinatra-example project under my GitHub account.

Installing an Ubuntu 10.04 Server Under VirtualBox

2010-06-22T00:00:00Z

When installing Ubuntu 10.04 Server as a VirtualBox VM, don’t use the “Install Minimal Virtual Machine” option in the installer options.

Strangely, that causes installing it as a VM to fail spectacularly, with the boot loader failing to find any hard disks. A standard install seems to work fine.

Creating a new Rails 3 Application with Mongoid, RSpec, Cucumber and Shoulda

2010-06-16T00:00:00Z

This article will work through creating a new Rails 3 application ready to start building a site using Mongoid, and testing it with Cucumber and RSpec.

I’m going to assume you’re familiar with the basics, and you’ve got a copy of Ruby 1.8.7 and MongoDB installed and running. If that’s not the case, go and get it setup using RVM or similar.

As much as I’d love to be using Ruby 1.9.2, ruby-debug doesn’t work with it yet, and I’m not about to give up my precious debugger.

Creating the Application

You’ll need to install the pre-release version (currently 3.0.0 beta 4) before we start:

# gem install rails --version "3.0.0.beta4"

Once that’s done, you can create a new application. We’re going to skip some of the default components, because we’ll be using alternatives.

# rails new example_app --skip-activerecord \
                        --skip-prototype \
                        --skip-testunit
create  
create  README
create  Rakefile
create  config.ru
... lots more create statements ...
create  tmp/pids
create  vendor/plugins
create  vendor/plugins/.gitkeep

Install the Gems

Once our application has been generated we need to specify the gems we’ll be using in the Gemfile, which is used by Bundler to manage dependencies.

In your application root, open up Gemfile and replace it with the following, which should be fairly self explanatory.

source "http://rubygems.org"

gem 'rails', '3.0.0.beta4'
gem 'mongoid', :git => 'git://github.com/durran/mongoid.git'
# C extension to improve MongoDB performance
gem 'bson_ext'

group :development do
  gem 'unicorn'
  gem 'ruby-debug'
end

group :test do
  gem 'capybara'
  gem 'cucumber-rails'
  gem 'rspec-rails', '>= 2.0.0.beta.12'
  gem 'factory_girl'
  gem 'mongoid-rspec', :require => false
end

And then run bundle install to install all the dependencies.

Install Mongoid

The next few steps are mostly just running generators, thanks to the fancy new integration available with Rails 3.

# rails g mongoid:config

Then open up config/mongoid.yml and make any changes neccesary for your machine. The defaults work for me when using a copy of Mongo installed using Homebrew.

At this point, when you generate a new model, you’ll get a Mongoid document rather then an ActiveRecord model.

Install RSpec

Installing RSpec is very similar:

# rails g rspec:install

Then insert the following after require ‘rspec/rails’ in spec/spec_helper.rb

require 'mongoid-rspec'

And this after the Rspec.configure line:

include Mongoid::Matchers

That’s it for vanilla RSpec. Nothing more to be done. If you run rake now you should see.

(in /Users/jon/Dropbox/example_app)
No examples matching ./spec/**/*_spec.rb could be found

Install Cucumber

We’re going to install Cucumber telling it to use Capybara so that we can test Javascript as well, and not to modify database.yml

rails g cucumber:install --capybara --skip-database

We’ll also need to provide it with a placeholder task for db:test:prepare, because it expects that to be present as well. If it’s not their, you’ll get the error Don’t know how to build task ‘db:test:prepare’. Put the following in lib/tasks/db.rake

namespace :db do
  namespace :test do
    task :prepare do
      # Placeholder for Cucumber
    end
  end
end

And the Cucumber environment needs changing to deal with ActiveRecord not being present as well. Edit features/support/env.rb and comment out or remove the line below.

Cucumber::Rails::World.use_transactional_fixtures = true

While you’re in there, remove the if statement around Database Cleaner’s setup so that the database is also emptied out for MongoDB.

# if defined?(ActiveRecord::Base)
  begin
    require 'database_cleaner'
    DatabaseCleaner.strategy = :truncation
  rescue LoadError => ignore_if_database_cleaner_not_present
  end
# end

Now that all that has been done, you should be able to run rake cucumber:ok and get this output:

Using the default profile...


0 scenarios
0 steps
0m0.000s

Testing Sugar

Now we’re going to add in Factory Girl to the test suite to make things that little bit more conveniant for us. It even comes with Cucumber steps to build models from factories.

Create a directory for factories. We’ll eventually fill this with a file for each model type, in the format model_name_factories.rb.

# mkdir spec/factories

And then require factory_girl and the factories for both RSpec and Cucumber.

In spec/spec_helper.rb, after you require mongoid-rspec

require 'factory_girl'
Dir.glob(Rails.root.join("spec", "factories", "*.rb").to_s).each do |f|
  require f
end

And in features/support/env.rb, after Capybara configuration the same, followed by:

require 'factory_girl/step_definitions'

Check out Thoughtbot’s article on the bundled Cucumber steps.

Bonus! jQuery

You may have noticed my call to —skip-prototype. That’s because we’re going to use jQuery instead.

curl http://github.com/rails/jquery-ujs/raw/master/src/rails.js > public/javascripts/rails.js

And then when you load jQuery, load rails.js as well, and you can use the standard Rails helpers for Javascript and it’ll all just work.

And Your Done

Thats it. You’re ready to rock and roll now.

A Blog, You Say?

2010-06-15T00:00:00Z

Yes, a blog! I’m once again going to try writing regularly. Whether or not this descends into the usual deriliction of previous attempts we’ll have to see, but I’m hopeful.

I’m using a nice simple tool, in the shape of Toto which is a very minimalist blogging engine, in the hope that using nothing but plain text will help me to focus more on writing, and less on endlessly tweaking things.

Topics to be covered include:

Software development, particularly using Ruby
Systems administration, focusing on automated infrastructures
Games
Cooking
And a bit of everything else for good measure