Through various meetings at Agile2010 and London agile testing meetings Gojko Adzic and others have been converging on a common language to talk about Acceptance testing and the different phases. In doing so we have fleshed out a lifecycle for Acceptance testing.

Seeing a wider picture of the lifecycle helped me gain a better scope of how Cucumber Features are born. I thought I would share a real example demonstrating how the acceptance testing lifecycle (can) flow at Songkick.com.

Example

Business goal

Increase the ratio of people signing up

This gives us something to measure and learn from to assess the success of the features built.

User stories – MMFs

Generating ideas through discussion and brainstorming that achieve this business goal. These are captured as Minimal Marketable Features and have our business goal as their value:

In order to increase signups
I want visitors to signup through their facebook accounts

We break the minimal marketable features into user stories. We try and break the stories into as small as possible units.

Successful signup through Facebook
In order to sign up with as little effort as possible
As a non-member
I want to signup through my facebook account




Failed signup up through Facebook
In order to sign with as little effort as possible
As a non-member
I want to know what I can do to correct errors preventing signup

Key examples

We have a card with the story narrative representing a token for conversation. We use this card to start discussions with QAs, developers and UX exploring the requirements through concrete examples. Playing the what-if game.

Through this we generate a list of scenarios:

Scenario: Successful signup through facebook

Scenario: Failing due to already having a facebook account connected

Scenario: Facebook is unavailable

Note that while we express these as scenarios its not implied that all are going to be tested at the acceptance test level. The developer for scaling/performance issues may push a scenario down to lower level tests.

Specification with examples

Pairing with a QA we take the cards and our exploration notes and codify them into Cucumber features.

Feature: Visitor signups up through Facebook 
  In order to reduce friction in signing up
  As a non-member
  I want to signup through my facebook account 

Scenario: Successful signup through facebook
    Given ...
    When ...
    Then ...

Scenario:Failing due to already having a facebook account connected
    Given ...
    When ...
    Then ...

Scenario: Failed due to Facebook being unavailable
    Given ...
    When ...
    Then ...

Note that there is not a 1-1 mapping between a Feature and a User Story.

Literal automation: Mapping Feature down to step definitions

The developer turns the feature into an executable test at the same time as writing the feature. When using Cucumber this consists of writing the step definitions.

Continuous validation

The feature is continuously run upon every commit (Using a Continuous integration server)

Living documentation

Exposing features for all to read. Using a web based system such as Relish the features are easily browserable by everyone yet they live in the codebase.