Comments on: Metrics for Plain Text Acceptance Tests

By: Joseph Wilk

Joseph Wilk — Mon, 30 Nov 2009 23:29:39 +0000

Thanks for all the feedback. I’m focusing on my coding experiments on this in my rotten project: http://github.com/josephwilk/rotten

By: Aslak Hellesøy

Aslak Hellesøy — Tue, 24 Nov 2009 18:44:51 +0000

Perryn Fowler has a vaguely related post: http://www.jroller.com/perryn/entry/given_when_then_and_how

Might be hard to detect automatically…

By: Ben Mabey

Ben Mabey — Mon, 23 Nov 2009 18:31:03 +0000

One more thought relating to the scenario length smell is the step to scenario ratio. (I think Matt has brought this up on the ML before.) We could find out how much reuse of steps is occurring by looking at the length of scenarios versus the number of actual steps used. The number that this tool produced would be very subjective as some projects may choose to take a very declarative approach and sacrifice step reuse.

re: ignoring warnings, the other code smell detectors use a YAML file to do this so I think that would be the best option.

By: Greg Hnatiuk

Greg Hnatiuk — Mon, 23 Nov 2009 02:59:17 +0000

Speaking of putting comments in the feature files, I think comments themselves in features files are smell.

Narratives for Features, Scenarios, Outlines, Backgrounds, and Examples are too permissive to have to need to explain things further with comments.

By: Aslak Hellesøy

Aslak Hellesøy — Mon, 23 Nov 2009 01:38:14 +0000

I think this is a very interesting idea. Gherkin (the library) is pretty mature now, so implementing this can be done with a simple gherkin listener.

Another thing that drives me crazy is formatting – when people left-align steps with the word following the keyword, and not the keyword itself. That could be detectable.

Another one could be detection of duplication – and suggest refactoring to Scenario Outline.

I’m building a CLI for gherkin itself, and I’d be happy to include a smell detector in the gherkin gem!

I’m not sure how you would ignore warnings – putting comments in the feature files to achieve that is ugly noise. Maybe a YAML file could do it – referring to files and names. In order to void orphans in the YAML file we could also check for orphans and error if they are found.

By: Ben Mabey

Ben Mabey — Mon, 23 Nov 2009 00:46:39 +0000

Hi Joe,
I like the idea. I think it would be best to phrase these as “feature smells”. As you point out, these measures of feature quality can be very subjective. Just like code smells these would be indicators that something might be wrong but doesn’t necessarily dictate that it is. With that in mind it would be nice if such a Gherkin-smell detector allowed you to ignore warnings and set thresholds much like the code smell detectors.

One thing that really bugs me is when I see variable names in the features. I.E. “Given 10 admin_users exist”. I see this a lot in table headers too. So maybe have a Gherkin-smell for code-isms creeping into the features. (An obvious exception to this would be if you are creating a tool for developers.) I’m sure I could think of some more but I think you’ve outlined the big ones.