Testing your tests is kind of crazy. However when writing a library of Cucumber step definitions which will be used in many projects it started to make sense to test my tests.

• The step definitions are the code.
• It helps reduce fear of breaking lots of projects which use the steps.
• The tests/specs show examples of how to use the step definitions.

It is important to note that I’m not imply TDD/BDDing these step definitions. My use-case is adding tests afterwards when it comes time to extract them to a library.

How to test step definitions

Exercise the full step (with Rspec)

The common way of testing complex steps is to extract all the ruby from the step definitions and then just test that. But this way of testing does not exercise the step definitions from the outside, getting as close as possible to how they will be used. It also does not provide examples of how to use the step definitions.

So I sat down with Matt Wynne, who started this discussion and we thrashed out some Rspec macros for testing whole step definitions.

If we were testing this step definition (icalendar_steps.rb):

require 'icalendar'

module Cucumber
  module Stepdefs
    module Icalendar
      def response_calendars
        ::Icalendar.parse(response.body)
      end

      def response_events
        response_calendars.length.should == 1
        response_calendars.first.events
      end
    end
  end
end

Before('@ical') do
  extend Cucumber::Stepdefs::Icalendar
end

Then /^the iCalendar should have exactly (\d+) events?$/ do |number_of_events|
  response_events.length.should == number_of_events.to_i
end

Our spec would look like this (Note: we test the Before hook as well as the step definition):

describe 'icalendar_steps' do
  step_file File.dirname(__FILE__) + '/../../../lib/cucumber/stepdefs/icalendar_steps'

  # Test that the Before hook is not called when there is no tag
  without_tags do
    it "should not mix in any calendar related methods" do
      world_methods.should_not include('response_calendars')
      world_methods.should_not include('response_events')
    end
  end

  # Test the Before hook mixes in the right methods when tagged with @ical
  with_tag '@ical' do
    ['response_calendars', 'response_events'].each do |method|
      it "should add the #{method} to world" do
        world_methods.should include(method)
      end
    end

    the_step "the iCalendar should have exactly 1 event" do
      describe "when 1 calendar with 0 events is in the response body" do
        before(:each) do
          world.stub!(:response).and_return(mock_response_with_0_events)
        end

        it_should_fail_with(Spec::Expectations::ExpectationNotMetError)
      end

      describe "when 1 calendar with 1 event is in the response body" do
        before(:each) do
          world.stub!(:response).and_return(mock_response_with_1_event)
        end

        it_should_pass
      end
    end
  end
end

Experiment’s Source code

You can see the source on Github:

git clone git://github.com/mattwynne/cucumber-step_definitions.git

If this experiment proves successful these macros will make their way to a nice gem.