Background and general feature style

[jbpros]

I'm currently adding background support to -js (yey!).

As usual, I start by adding content to the related feature to cucumber-features. There were several discussions about the style of the features and I don't think we came up with a clear decision just yet, except for core.feature.

The style in core.feature is quite declarative. There are feature source fragments (i.e. scenarios and whole features) included though.

Today I'm wondering if we should stick to this "not-totally-declarative" style for the the other features or not.

Feature: Background

  Background allows you to add some context to the scenarios in a
  single feature. A Background is much like a scenario containing a
  number of steps. The difference is when it is run. The background is
  run before each of your scenarios but after any of your Before
  Hooks.

  Scenario: One scenario and a background
    Given the following feature:
      """
      Feature: testing scenarios
        Background: 
          Given a step passes

        Scenario:
          When a step passes
      """
    And the step "a step passes" has a passing mapping
    When Cucumber runs the feature
    Then the feature passes

  Scenario: One scenario and a background
    Given a background with:
      """
      Given a step passes
      """
    And a scenario "passing scenario" with:
      """
      When a a step passes
      """
    And the step "a step passes" has a passing mapping
    When Cucumber executes the scenario "passing scenario"
    Then the background passes
    And the scenario passes

  Scenario: One scenario and a background
    Given a passing background
    When Cucumber executes a scenario
    Then the background passes
    And the scenario passes

Those three scenarios describe the same behaviour, only their style is different. Which of them do you think is the best?

• The first one is something we tried to avoid as much as possible, it seemed too verbose (but it's used once in core.feature and once in comments.feature).
• The second one is better at documenting. But based on the discussions we had, documenting the Cucumber functionalities is not the main objective of cucumber-features. The goal of cucumber-features is to provide a strong test suite for implementors (correct me if I'm wrong here).
• The third scenario is more focused on the expected behaviour.

Environment-hooks was written in a totally declarative style and it just feels right to me.

As far as I went with background.feature, it looks like its scenarios are really similar to the ones in core. It's all about checking that steps/scenarios pass, fail and are skipped. This makes it a good candidate for reusing the steps we used in core.

Here is a list of pros/cons for both approaches:

The not-totally-declarative style (second example scenario)

Pros

• step reuse
• better at documenting

Cons

• not totally declarative :)

The declarative style

Pros

• focuses on the expected behaviour

Cons

• leads to more step definitions (may be harder to maintain)

Feel free to improve this list.

[mattwynne]

On 12 Sep 2011, at 10:43, Julien Biezemans wrote:

Scenario: One scenario and a background
Given a passing background
When Cucumber executes a scenario
Then the background passes
And the scenario passes

This one is by far the best documentation, IMO.

You've made a great summary of the pros / cons of declarative / imperative styles. However the fear about maintenance cost of a declarative style isn't warranted. In my experience, if you push enough logic down to support helper methods, you can have as many one-line step definitions / mappings as you like.

I'm keen to help pair up and help this style spread through the team.

[mattwynne]

Having said that, it may be that having some Gherkin examples in the features is also useful as documentation. The key for me is that we drive this from what makes good documentation for the long term, rather than what makes an easy-to-implement test right now.

[jbpros]

Your help is welcome, obviously :)

I'm also more interested in the quality of the suite rather than the ease to write it down in the short term. Let's not be afraid of the time and effort needed to make it great, I say!

But what makes it of "good quality" is still to be determined. The first step might be to agree on a vision for this feature suite.

The current README file reads: "This is the collection of features that are expected to run successfully against any Cucumber implementation.".

This is not complete, IMHO. As stated before, I see two possible targets for cucumber-features:

• Provide a strict and complete testing suite for implementors to build implementations that provide the same set of functionalities as the original Cucumber tool.
• Provide complete and runnable documentation about the functionalities of any Cucumber implementation to users.

Other ideas?

[mattwynne]

Good point, it's important to think about who your customer is!

There is a tension between writing cukes that accurately and concisely document the system's behaviour, and cukes that explain to a user how to use the feature. @dchelimsky has bravely attempted to get both with RSpec 2.0's features, and I think he's stretching Gherkin to it's limits.

Realistically, I think the people who are going to be poring over these features in detail are implementors, or technical users who think they've found a bug in an existing Cucumber and want to see if the edge case is already covered in the specifications. They're the 'systems analysts' of the Cucumber community. I think we should be writing the features for them, but that's just my opinion, let's hear some others! :)

[corroded]

IMHO, the third one is great since it describes the feature better. If you compare the last three lines of the first one to the last one:

And the step "a step passes" has a passing mapping
When Cucumber runs the feature
Then the feature passes

vs

When Cucumber executes a scenario
Then the background passes
And the scenario passes

the latter feels more "understandable" to many. I mean reading the first one for the first time, I was confused as to which "a step passes" has the passing mapping. And what's a "passing mapping"? If I was a newcomer to Cucumber and was reading through the features(just in case there are some people who actually do that instead of read tutorials/blog posts first), I'd be confused with those terms. The declarative style is much clearer since I know exactly what I'm supposed to expect: the background passes(since it is called before the scenario), and the scenario passes.

Just my two cents :)

[msassak]

Assuming that these features are primarily for implementors, bug-hunters and contributors, I've been using the concept of execution life-cycle to great effect in some stuff I'm working on now:

Feature: Backgrounds

  Gherkin features can contain a single background section
  defining steps to be run before each scenario in that feature.
  A background is a bit like a macro that prepends the
  background's steps to each scenario or scenario outline.

  Scenario: Scenario with a background
    Given a passing background with:
      """
      Given I log in as "Bob"
      """
    And a passing scenario "Arithmetic" with:
      """
      When I add 4 and 5
      Then the result is 9
      """
    When Cucumber executes the scenario "Arithmetic"
    Then the life cycle history is:
      | event | name              | status |
      | apply | I log in as "Bob" | passed |
      | apply | I add 4 and 5     | passed |
      | apply | the result is 9   | passed |

No doubt that could be improved, but the big upside of this to me is that it simultaneously shows that background steps are run before scenario steps and that the two are treated identically internally. There really is no difference (nor should there be), from the point of view of execution, between scenarios sharing a background and having the steps in that background prepended to each scenario.

[jbpros]

@msassak I like this approach. It clearly expresses the expected behaviour. And you're right about background steps; they are basically injected into the scenarios at runtime.

This raises one issue though. It was decided that the only gherkin elements needed to run all the features from cucumber-features would be in core.feature. I.e. if core.feature passes against an implementation, any feature from the suite could be run (not passing!) with that implementation.

Maybe this is an unnecessary constraint. But from my recent experience, this rule made it possible to run the suite with -js against itself, which was pretty cool :)

Moreover, even if -rb with Aruba is the official way of testing an implementation, I think that allowing new non-complete implementations to test themselves is quite useful. For example, if you install cucumber.js through NPM today, it will run the features against itself.

But hey, this constraint should not prevent us from writing better features, of course.

[msassak]

I think restricting the Gherkin allowed in cucumber-features limits their usefulness as documentation, and misses an opportunity to set a good example for other Cucumber users, i.e. it would be great if the community could look at these as excellent examples of Gherkin style. That being said, the bootstrap problem is still there, which is why I proposed a bootstrap feature like this a while back. We could tuck it away someplace out of the way, and whomever wanted to develop their own implementation could execute it with Cucumber 1.x and Aruba until the new implementation could stand on its own and start executing the scenarios from cucumber-features as they developed.

[mattwynne]

On 16 Sep 2011, at 03:46, Mike Sassak wrote:

We could tuck it away someplace out of the way, and whomever wanted to develop their own implementation could execute it with Cucumber 1.x and Aruba until the new implementation could stand on its own and start executing the scenarios from cucumber-features as they developed.

Could bootstrap just go in it's own repo?

[msassak]

Sir, you have cut the gordian knot! I'll do that soon if there are no objections.

[jbpros]

The bootstrap feature is great. You've got so many good ideas Mike, it's amazing.

No objections to isolate bootstrap. The only downside I can think of right now is the loss of step definitions available on cucumber-features (we can still submodule it in cucumber-bootstrap later on if needed).

I'll take care of moving DocStrings from core to their own feature.

[msassak]

Added the start of a bootstrap project here: https://github.com/cucumber/cucumber-bootstrap.

[jbpros]

Awesome! Thank you Mike.