Examples alone are not a Specification
24 Feb
The Gherkin syntax used by Cucumber enforces that Feature files contain scenarios which are examples of the behaviour of a feature. However Gherkin has no constraints on if there is a specification present. Examples are great at helping us understand specifications but they are not specifications themselves.
What do we mean when we say specification?
definition: A detailed, exact statement of particulars
In a Gherkin feature the specification lives here:
Lets look at a real example:
A Feature with just Examples
A Cucumber example based on a feature (which I have modified) from the test library Rspec rspec-expectations:
Feature: be_within matcher
Scenario: basic usage
Given a file named "be_within_matcher_spec.rb" with:
"""
describe 27.5 do
it { should be_within(0.5).of(27.9) }
it { should be_within(0.5).of(27.1) }
it { should_not be_within(0.5).of(28) }
it { should_not be_within(0.5).of(27) }
# deliberate failures
it { should_not be_within(0.5).of(27.9) }
it { should_not be_within(0.5).of(27.1) }
it { should be_within(0.5).of(28) }
it { should be_within(0.5).of(27) }
end
"""
When I run `rspec be_within_matcher_spec.rb`
Then the output should contain all of these:
| 8 examples, 4 failures |
| expected 27.5 not to be within 0.5 of 27.9 |
| expected 27.5 not to be within 0.5 of 27.1 |
| expected 27.5 to be within 0.5 of 28 |
| expected 27.5 to be within 0.5 of 27 |
So where is the explanation of what be_within does? If I want to know how be_within works I want a single concise explanation not 5/6 different examples. Examples add value later to validate that specification.
A Feature with both Specification and Examples
Lets add back in the specification part of the Feature. *drum roll*
Feature: be_within matcher
Normal equality expectations do not work well for floating point values.
Consider this irb session:
> radius = 3
=> 3
> area_of_circle = radius * radius * Math::PI
=> 28.2743338823081
> area_of_circle == 28.2743338823081
=> false
Instead, you should use the be_within matcher to check that the value
is within a delta of your expected value:
area_of_circle.should be_within(0.1).of(28.3)
Note that the difference between the actual and expected values must be
smaller than your delta; if it is equal, the matcher will fail.
Scenario: basic usage
Given a file named "be_within_matcher_spec.rb" with:
"""
describe 27.5 do
it { should be_within(0.5).of(27.9) }
it { should be_within(0.5).of(27.1) }
it { should_not be_within(0.5).of(28) }
it { should_not be_within(0.5).of(27) }
# deliberate failures
it { should_not be_within(0.5).of(27.9) }
it { should_not be_within(0.5).of(27.1) }
it { should be_within(0.5).of(28) }
it { should be_within(0.5).of(27) }
end
"""
When I run `rspec be_within_matcher_spec.rb`
Then the output should contain all of these:
| 8 examples, 4 failures |
| expected 27.5 not to be within 0.5 of 27.9 |
| expected 27.5 not to be within 0.5 of 27.1 |
| expected 27.5 to be within 0.5 of 28 |
| expected 27.5 to be within 0.5 of 27 |
Thats better, we can get an explanation of why this method exists and how to use it.
Imagine RSpec without the specification
I think of a Cucumber feature without a specification much like an Rspec example without any English sentence/description.
context "" do
it "" do
user = Factory(:user)
user.generate_password
user.activate
get "/session/new", :user_id => user.id
last_response.should == "Welcome #{user.name}"
end
end
Feels a little odd doesn’t it.
Cucumber Features as Documentation (for real)
Rspec is an example of a project that has taken its Cucumber features and published them as its documentation. Just browse through those features and it quickly highlights how important it is to have a specification as well as examples. Imagine an API with nothing but examples, leaving you the detective work of trying to work out what the thing actually does.
Documentation needs to explain/specify what something does as well provide examples. If you really want anyone to read your feature provide both examples and a specification.
I’m very excited to be talking at this years 
