Joseph Wilk

Testing your tests is kind of crazy. However when writing a library of Cucumber step definitions which will be used in many projects it started to make sense to test my tests.

• The step definitions are the code.
• It helps reduce fear of breaking lots of projects which use the steps.
• The tests/specs show examples of how to use the step definitions.

It is important to note that I’m not imply TDD/BDDing these step definitions. My use-case is adding tests afterwards when it comes time to extract them to a library.

How to test step definitions

Exercise the full step (with Rspec)

The common way of testing complex steps is to extract all the ruby from the step definitions and then just test that. But this way of testing does not exercise the step definitions from the outside, getting as close as possible to how they will be used. It also does not provide examples of how to use the step definitions.

So I sat down with Matt Wynne, who started this discussion and we thrashed out some Rspec macros for testing whole step definitions.

If we were testing this step definition (icalendar_steps.rb):

require 'icalendar'

module Cucumber
  module Stepdefs
    module Icalendar
      def response_calendars
        ::Icalendar.parse(response.body)
      end

      def response_events
        response_calendars.length.should == 1
        response_calendars.first.events
      end
    end
  end
end

Before('@ical') do
  extend Cucumber::Stepdefs::Icalendar
end

Then /^the iCalendar should have exactly (\d+) events?$/ do |number_of_events|
  response_events.length.should == number_of_events.to_i
end

Our spec would look like this (Note: we test the Before hook as well as the step definition):

describe 'icalendar_steps' do
  step_file File.dirname(__FILE__) + '/../../../lib/cucumber/stepdefs/icalendar_steps'

  # Test that the Before hook is not called when there is no tag
  without_tags do
    it "should not mix in any calendar related methods" do
      world_methods.should_not include('response_calendars')
      world_methods.should_not include('response_events')
    end
  end

  # Test the Before hook mixes in the right methods when tagged with @ical
  with_tag '@ical' do
    ['response_calendars', 'response_events'].each do |method|
      it "should add the #{method} to world" do
        world_methods.should include(method)
      end
    end

    the_step "the iCalendar should have exactly 1 event" do
      describe "when 1 calendar with 0 events is in the response body" do
        before(:each) do
          world.stub!(:response).and_return(mock_response_with_0_events)
        end

        it_should_fail_with(Spec::Expectations::ExpectationNotMetError)
      end

      describe "when 1 calendar with 1 event is in the response body" do
        before(:each) do
          world.stub!(:response).and_return(mock_response_with_1_event)
        end

        it_should_pass
      end
    end
  end
end

Experiment’s Source code

You can see the source on Github:

git clone git://github.com/mattwynne/cucumber-step_definitions.git

If this experiment proves successful these macros will make their way to a nice gem.

I was speaking in Edinburgh at Scotland on Rails 2009 about Cucumber and Rspec.

You can watch the recorded full talk.

I’ve also posted the slides from the presentation and uploaded the screencasts used in the presentation in both high and low resolutions. They are accessible from links within the presentation.

Here are some of the useful links from the presentation:

• Cuke’s official website – http://cukes.info
• Cucumer’s github wiki – http://wiki.github.com/aslakhellesoy/cucumber
• Rspec’s website – http://rspec.info
• The Rspec book – Pragmatic book store – http://www.pragprog.com/titles/achbd/the-rspec-book
• In-memory browser testing – Webrat – http://wiki.github.com/brynary/webrat
• Browser based testing – Watir – http://wtr.rubyforge.org/
• Browser based testing – Selenium – http://seleniumhq.org/
• Headless Browser testing – Celerity – http://celerity.rubyforge.org/
• Using Celerity from within Ruby – Culerity – http://upstream-berlin.com/2009/01/28/culerity-full-stack-rails-testing-with-cucumber-and-celerity/
• Website for checking regular expressions – Rubular – http://www.rubular.com/

I would like to thank the organisers, everyone who came to listen and speak at the conference. It was a pleasure to be a part of such an enthusiastic group of people in such a beautiful venue.

I’m really excited to be giving a talk at this years Scotland on Rails conference in Edinburgh.

I’ll be talking about working outside-in with Cucumber and RSpec. Having used Cucumber and as a member of the Cucumber core developer team I hope to share lots of experiences and lessons about getting the most out of the tool.

It’s looking like a great line up with with some really interesting presentations across a broad number of topics. The keynotes speakers are Michael Feathers and Marcel Molina, Jnr.

If you’re going to be in Edinburgh for the conference or have any burning questions about Cucumber, let me know.

I’ve been developing multiple systems using Rspec stories for a little while now. There are a lot of great resources to get you started with a taste of what you can do with stories. Some of the resources I found useful where:

• http://peepcode.com/products/rspec-user-stories
• http://blog.davidchelimsky.net/2008/6/16/slides-from-railsconf
• http://www.benmabey.com/2008/02/04/rspec-plain-text-stories-webrat-chunky-bacon/
• http://dannorth.net/whats-in-a-story
• http://evang.eli.st/blog/2007/10/8/story-runner-top-to-bottom-screencast

However once I had understood the basic idea I struggled to find practical examples and general guidance on writing real stories. So I’ve collected some of the lessons I’ve learnt along the way with story examples taken from real systems and how I’ve improved them as I learnt. Most examples are from web based applications.

Read the rest of this entry »

Rspec-rails is a rails plugin which brings the Rspec Ruby Behaviour Driven Development framework to rails along with some rails specific helpers. One of these hugely useful helper functions is:

mock_model(model_class, options_and_stubs = {})

This creates a mock object with the common methods stubbed out. It also allows you to specify other methods you want to stub.
Read the rest of this entry »

When using Rspec stories you have plain text stories which we call the ’story’ file and the ’story steps’ file that maps the plain text story to programmatic code. Generally you end up with your story files not being DRY. This is not a worry, your stories are the domain specific languages detailing your acceptance/integration tests. Its like saying that your Rails Models are not DRY because they repeat lots of 'has_one'!
Read the rest of this entry »