Skip to content

Commit

Permalink
Merge pull request #544 from cucumber/cleanup-speedup
Browse files Browse the repository at this point in the history 
Clean up and speed up cucumber test suite
  • Loading branch information
@mvz
mvz authored Mar 7, 2018
2 parents 0f31e48 + d8aafed commit e68d401
Show file tree
Hide file tree
Showing 39 changed files with 558 additions and 652 deletions.
169 changes: 127 additions & 42 deletions CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,20 +18,33 @@ We would love to get help from you as "user" and "contributor".
* Add missing documentation
* Improve our test coverage

The rest of this document is a guide for those maintaining Aruba, and others who would like to submit patches.
The rest of this document is a guide for those maintaining Aruba, and others
who would like to submit patches.

## Contributing to the "aruba" project

It would be great if all people who want to contribute to the "aruba" project – contributors and maintiners – follow the guidelines in this section. There are also "Getting started"-sections both for [contributors](#getting-started-as-a-contributor) and [maintainers]((#getting-started-as-a-maintainer).
It would be great if all people who want to contribute to the "aruba" project
— contributors and maintainers — follow the guidelines in this
section. There are also "Getting started"-sections both for
[contributors](#getting-started-as-a-contributor) and
[maintainers](#getting-started-as-a-maintainer).

### Issues

We appreciate that. But before you do, please learn our basic rules:
About to create a new Github Issue? We appreciate that. But before you do,
please learn our basic rules:

* This is not a support forum. If you have a question, please go to [The Cukes Google Group](http://groups.google.com/group/cukes).
* Do you have an idea for a new feature? Then don't expect it to be implemented unless you or someone else sends a [pull request](https://help.github.com/articles/using-pull-requests). You might be better to start a discussion on [the google group](http://groups.google.com/group/cukes).
* This is not a support forum. If you have a question, please go to
[The Cukes Google Group](http://groups.google.com/group/cukes).
* Do you have an idea for a new feature? Then don't expect it to be implemented
unless you or someone else sends a
[pull request](https://help.github.com/articles/using-pull-requests). It
might be better to start a discussion on
[the google group](http://groups.google.com/group/cukes).
* Reporting a bug? Just follow our comments in the issue template
* We love [pull requests](https://help.github.com/articles/using-pull-requests). The same here: Please consider our comments within the template we provide for your pull request(s).
* We love [pull requests](https://help.github.com/articles/using-pull-requests).
The same here: Please consider our comments within the template we provide
for your pull request(s).


### Pull Requests
Expand All @@ -41,41 +54,54 @@ We appreciate that. But before you do, please learn our basic rules:
Please...

* Fork the project. Make a branch for your change.
* Make your feature addition or bug fix – if you're unsure if your addition will be accepted, open an issue for discussion first
* Make sure your patch is well covered by tests. We don't accept changes that aren't tested.
* Make your feature addition or bug fix — if you're unsure if your
addition will be accepted, open an issue for discussion first
* Make sure your patch is well covered by tests. We don't accept changes that
aren't tested.
* Please do not change the Rakefile, version, or history.
(if you want to have your own version, that is fine but
bump version in a commit by itself so we can ignore when we merge your change)
* Make sure your pull request complies to our development style
* Rebase your branch if needed to reduce clutter in our git history
* Make sure you don't break other people's code – On major changes: First deprecated, than bump major version, than make breaking changes
* Split up your changes into reviewable "git"-commits which combine all lines/files relevant for a single change
(If you want to have your own version, that is fine but
bump version in a commit by itself so we can ignore it when we merge your change.)
* Make sure your pull request complies with our development style.
* Rebase your branch if needed to reduce clutter in our git history.
* Make sure you don't break other people's code — On major changes: First
deprecate, then bump major version, then make breaking changes,
* Split up your changes into reviewable "git"-commits which combine all
lines/files relevant for a single change.
* Send us a pull request.

**Maintainers**

* Use pull requests for larger or controversial changes made by yourself or changes you might expected to break the build
* Commit smaller changes directly to master, e.g. fixing typos, adding tests or adding documentation
* Update [History.md](History.md) when a pull request is merged
* Make sure all tests are green before merging a pull request
* Use pull requests for larger or controversial changes made by yourself or
changes you might expected to break the build.
* Commit smaller changes directly to master, e.g. fixing typos, adding tests or
adding documentation.
* Update [CHANGELOG.md](CHANGELOG.md) when a pull request is merged.
* Make sure all tests are green before merging a pull request.

### Development style

* We try to follow the recommendations in the [Ruby Community Style Guide](https://github.com/bbatsov/ruby-style-guide) and use [`rubocop`](https://github.com/bbatsov/rubocop) to "enforce" it. Please see [.rubocop.yml](.rubocop.yml) for exceptions.
* There should be `action`-methods and `getter`-methods in `aruba`. Only the latter should return values. Please expect the first ones to return `nil`.
* Add documentation (aka acceptance tests) for new features using `aruba`'s steps and place them some where suitable in [here](features/).
* Add unit tests where needed to cover edge cases which are not (directly) relevant for users
* Add developer documentation in [`yardoc`](http://yardoc.org/) to all relevant methods added
* Format your commits messages following those seven rules -- see [this blog post](http://chris.beams.io/posts/git-commit/) for a well written explanation about the why.
* We try to follow the recommendations in the
[Ruby Community Style Guide](https://github.com/bbatsov/ruby-style-guide) and use
[`rubocop`](https://github.com/bbatsov/rubocop) to "enforce" it. Please see
[.rubocop.yml](.rubocop.yml) for exceptions.
* There should be `action`-methods and `getter`-methods in `aruba`. Only the
latter should return values. Please expect the first ones to return `nil`.
* Add documentation (aka acceptance tests) for new features using `aruba`'s
steps and place them some where suitable in [here](features/).
* Add unit tests where needed to cover edge cases which are not (directly)
relevant for users.
* Add developer documentation in [`yardoc`](http://yardoc.org/) to all relevant
methods added.
* Format your commits messages following those seven rules — see
[this blog post](http://chris.beams.io/posts/git-commit/) for a well written
explanation about the why.
1. Separate subject from body with a blank line
2. Limit the subject line to 50 characters
3. Capitalize the subject line
4. Do not end the subject line with a period
5. Use the imperative mood in the subject line
6. Wrap the body at 72 characters
7. Use the body to explain what and why vs. how (optional if subject is self-explanatory)
8. Use Markdown Markup to style your message (only if required)


## Getting started as a "Contributor"

Expand All @@ -94,27 +120,44 @@ bin/bootstrap
Make sure you bootstrap the environment first. Then run the following command
to run the test suite.

~~~bash
```bash
# Run the test suite
bin/test
~~~

If you have problems because our assumptions about your local setup are wrong.
Try this instead. This requires the `docker`-command/project to be installed on
# Or use these rake tasks:

# Run the whole test suite
rake test
# Run RSpec tests
rake test:rspec
# Run Cucumber features
rake test:cucumber
# Run Cucumber features which are "WORK IN PROGRESS" and are allowed to fail
rake test:cucumber_wip
```

If you have problems because our assumptions about your local setup are wrong,
try the follwing instead. This requires the `docker`-command/project to be installed on
your local system.

~~~bash
```bash
# Build the docker container
bundle exec rake docker:build

# Build with disabled cache
bundle exec rake 'docker:build[false]'

# Build image with version tag
bundle exec rake 'docker:build[false, 0.1.0]'

# Run the whole test suite in "docker"-container
RUN_IN_DOCKER=1 bin/test

# Run only selected scenario
RUN_IN_DOCKER=1 bin/test cucumber features/steps/command/shell.feature:14
~~~
```

## Installing your own gems used for development
### Installing your own gems used for development

A `Gemfile.local`-file can be used to have your own gems installed to support
your normal development workflow.
Expand All @@ -127,14 +170,51 @@ gem 'pry-byebug'
gem 'byebug'
~~~

### Running a developer console

If you want to try out the Aruba api, you canuse the interactive aruba console:

```bash
bin/console
```

This will start an IRB console with Aruba's API loaded.

### Linting

Aruba's Rakefile provides the following linting tasks

```bash
bundle exec rake lint # Run all linters
bundle exec rake lint:coding_guidelines # Lint our code with "rubocop"
bundle exec rake lint:licenses # Check for relevant licenses in project
bundle exec rake lint:travis # Lint our .travis.yml
```

### Building and installing your local Aruba version

You can use the following rake tasks to build and install your work-in-progress locally:

```bash
# Build your copy
bundle exec rake rubygem:build
# Build and install your copy
bundle exec rake rubygem:install
# Build and install your copy without network access
bundle exec rake rubygem:install:local
```

## Getting started as a "Maintainer"

### Release Process

* Bump the version number in `lib/aruba/version.rb`
* Make sure `History.md` is updated with the upcoming version number, and has entries for all fixes.
* No need to add a `History.md` header at this point - this should be done when a new change is made, later.
* If a major version is released, update the `still`-branch which points to "old" major version
* Make sure `CHANGELOG.md` is updated with the upcoming version number, and has
entries for all fixes.
* No need to add a `CHANGELOG.md` header at this point - this should be done
when a new change is made, later.
* If a major version is released, update the `still`-branch which points to
"old" major version

Now release it

Expand All @@ -156,17 +236,22 @@ git merge master
git push
~~~

Now send a PR to https://github.com/cucumber/website adding an article about the with details of the new release and merge it - an aruba maintainer should normally allowed to merge PRs on `cucumber/website`. A copy of an old announcement can be used as basis for the new article. After this send an email with the link to the article to cukes@googlegroups.com.
Now send a PR to https://github.com/cucumber/website adding an article
with details of the new release and merge it - an aruba maintainer should
normally allowed to merge PRs on `cucumber/website`. A copy of an old
announcement can be used as basis for the new article. After this send an email
with the link to the article to cukes@googlegroups.com.

### Gaining Release Karma

To become a release manager, create a pull request adding your name to the list below, and include your Rubygems email address in the ticket. One of the existing Release managers will then add you.
To become a release manager, create a pull request adding your name to the list
below, and include your Rubygems email address in the ticket. One of the
existing Release managers will then add you.

Current release managers:
* Aslak Hellesøy ([@aslakhellesoy](http://github.com/aslakhellesoy))
* Dennis Günnewig ([@maxmeyer](http://github.com/maxmeyer), [@dg-rationdata](http://github.com/dg-rationdata))
* Jarl Friis ([@jarl-dk](https://github.com/jarl-dk))
* Matt Wynne ([@mattwynne](http://github.com/mattwynne))
* Aslak Hellesøy ([@aslakhellesoy](https://github.com/aslakhellesoy))
* Matt Wynne ([@mattwynne](https://github.com/mattwynne))
* Matijs van Zuijlen ([@mvz](https://github.com/mvz))

To grant release karma, issue the following command:

Expand Down
52 changes: 41 additions & 11 deletions features/01_getting_started_with_aruba/run_commands.feature
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,15 @@ Feature: Run commands with Aruba

Background:
Given I use a fixture named "getting-started-app"

@requires-bash
Scenario: Bash Program
Given an executable named "bin/aruba-test-cli" with:
"""bash
#!/usr/bin/env bash
echo "Hello, Aruba!"
"""
And a file named "features/hello_aruba.feature" with:
"""
Feature: Getting Started With Aruba
Expand All @@ -22,15 +31,6 @@ Feature: Run commands with Aruba
Hello, Aruba!
\"\"\"
"""

@requires-bash
Scenario: Bash Program
Given an executable named "bin/aruba-test-cli" with:
"""bash
#!/usr/bin/env bash
echo "Hello, Aruba!"
"""
When I successfully run `cucumber`
Then the features should all pass

Expand Down Expand Up @@ -61,6 +61,16 @@ Feature: Run commands with Aruba
puts "Hello, Aruba!"
"""
And a file named "features/hello_aruba.feature" with:
"""
Feature: Getting Started With Aruba
Scenario: First Run of Command
Given I successfully run `aruba-test-cli`
Then the output should contain:
\"\"\"
Hello, Aruba!
\"\"\"
"""
When I successfully run `cucumber`
Then the features should all pass

Expand Down Expand Up @@ -91,6 +101,16 @@ Feature: Run commands with Aruba
print("Hello, Aruba!")
"""
And a file named "features/hello_aruba.feature" with:
"""
Feature: Getting Started With Aruba
Scenario: First Run of Command
Given I successfully run `aruba-test-cli`
Then the output should contain:
\"\"\"
Hello, Aruba!
\"\"\"
"""
When I successfully run `cucumber`
Then the features should all pass

Expand Down Expand Up @@ -121,6 +141,16 @@ Feature: Run commands with Aruba
print "Hello, Aruba!\n";
"""
And a file named "features/hello_aruba.feature" with:
"""
Feature: Getting Started With Aruba
Scenario: First Run of Command
Given I successfully run `aruba-test-cli`
Then the output should contain:
\"\"\"
Hello, Aruba!
\"\"\"
"""
When I successfully run `cucumber`
Then the features should all pass

Expand Down Expand Up @@ -160,15 +190,15 @@ Feature: Run commands with Aruba
}
}
\"\"\"
And I successfully run `javac tmp/HelloArubaApp.java`
And I successfully run `javac tmp/HelloArubaApp.java` for up to 20 seconds
And I cd to "tmp/"
And I successfully run `java HelloArubaApp`
Then the output should contain:
\"\"\"
Hello, Aruba!
\"\"\"
"""
When I successfully run `cucumber`
When I successfully run `cucumber` for up to 21 seconds
Then the features should all pass

@requires-posix-standard-tools
Expand Down
Loading

0 comments on commit e68d401

Please sign in to comment.