Through various meetings at Agile2010 and London agile testing meetings Gojko Adzic and others have been converging on a common language to talk about Acceptance testing and the different phases. In doing so we have fleshed out a lifecycle for Acceptance testing.
Seeing a wider picture of the lifecycle helped me gain a better scope of how Cucumber Features are born. I thought I would share a real example demonstrating how the acceptance testing lifecycle (can) flow at Songkick.com.
Increase the ratio of people signing up
This gives us something to measure and learn from to assess the success of the features built.
User stories – MMFs
Generating ideas through discussion and brainstorming that achieve this business goal. These are captured as Minimal Marketable Features and have our business goal as their value:
In order to increase signups I want visitors to signup through their facebook accounts
We break the minimal marketable features into user stories. We try and break the stories into as small as possible units.
Successful signup through Facebook In order to sign up with as little effort as possible As a non-member I want to signup through my facebook account Failed signup up through Facebook In order to sign with as little effort as possible As a non-member I want to know what I can do to correct errors preventing signup
We have a card with the story narrative representing a token for conversation. We use this card to start discussions with QAs, developers and UX exploring the requirements through concrete examples. Playing the what-if game.
Through this we generate a list of scenarios:
Scenario: Successful signup through facebook Scenario: Failing due to already having a facebook account connected Scenario: Facebook is unavailable
Note that while we express these as scenarios its not implied that all are going to be tested at the acceptance test level. The developer for scaling/performance issues may push a scenario down to lower level tests.
Specification with examples
Pairing with a QA we take the cards and our exploration notes and codify them into Cucumber features.
Feature: Visitor signups up through Facebook In order to reduce friction in signing up As a non-member I want to signup through my facebook account Scenario: Successful signup through facebook Given ... When ... Then ... Scenario:Failing due to already having a facebook account connected Given ... When ... Then ... Scenario: Failed due to Facebook being unavailable Given ... When ... Then ...
Note that there is not a 1-1 mapping between a Feature and a User Story.
Literal automation: Mapping Feature down to step definitions
The developer turns the feature into an executable test at the same time as writing the feature. When using Cucumber this consists of writing the step definitions.
The feature is continuously run upon every commit (Using a Continuous integration server)
Exposing features for all to read. Using a web based system such as Relish the features are easily browserable by everyone yet they live in the codebase.