feat: testing guidelines #377
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,68 @@ | ||||||
| # Testing | ||||||
|
|
||||||
| ### Why? | ||||||
| Testing is awesome! | ||||||
|
|
||||||
| Testing is useful for developing new features, refactoring with confidence, and making sure new things don't break old things. When other people rely on your code for their own applications, testing helps you make sure things don't break for more than just yourself! | ||||||
|
|
||||||
| Testing is even more important when new maintainers take over a package. Sometimes when we work on a codebase for a long time we forget to articulate all the cognitive load to which we have become accustomed. Good test coverage can alleviate this burden. | ||||||
|
|
||||||
| ### Updates | ||||||
| Node.js has a [strict release pipeline](https://nodejs.org/en/about/releases/) focused on continuous improvement and update. In order to guarantee to your users that the module you made works correctly with the newer version of Node.js, it is a good practice to run your tests across multiple Node.js versions. | ||||||
|
|
||||||
| This can be done developing a CI pipeline. Check out our [CI guidelines](https://github.com/nodejs/package-maintenance/blob/master/README.md). | ||||||
|
|
||||||
| The minimal versions you should focus are: | ||||||
ghinks marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| * LTS (long time support) | ||||||
| * Current | ||||||
|
There was a problem hiding this comment. How does someone find out what these are? We should provide guidance here. There was a problem hiding this comment. agreed, there should be a link here. |
||||||
|
|
||||||
| Supporting older versions of node is encouraged where practical. | ||||||
ghinks marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
|
||||||
| For browser-based modules and modules that are designed to work equally both client and server side it is essential to test on various | ||||||
| versions of your target browser and to publish which environments are supported. | ||||||
|
Comment on lines
+21
to
+22
|
||||||
|
|
||||||
| ### How? | ||||||
| It is a good idea to have unit tests, coverage that matches most use cases for the module, and make sure things work in all supported environments. | ||||||
|
|
||||||
| * **unit test**: test your code | ||||||
| * **integration test**: test your code with other applications dependencies | ||||||
| * **acceptance test**: test your application sticks in performance, heavy load, etc.. | ||||||
|
There was a problem hiding this comment. Capitalize bullets and don't use There was a problem hiding this comment. Also given that you have headings under this further describing what's outlined here, you should probably make the bold parts also be relative links. |
||||||
|
|
||||||
| #### Unit Testing | ||||||
| The initial and fundamental start of any testing strategy should start with a unit testing strategy. For the JavaScript language | ||||||
| the unit test may be the first place to catch code that will not parse. It is also the way to define the expectation for a particular | ||||||
| function or code block. The unit test should describe what to expect from that code when given a defined set of inputs. | ||||||
|
|
||||||
| For a test to be considered a unit test it should consume no remote services. | ||||||
|
|
||||||
| It is generally considered good practice to run units as part of any commit or build strategy. | ||||||
|
|
||||||
| #### Integration Testing | ||||||
| Integration testing in general requires that the code under test be in a package. | ||||||
|
|
||||||
| For many packages integration testing requires that the built package be run via a testing tool upon several environments. The / | ||||||
|
There was a problem hiding this comment.
Suggested change
There was a problem hiding this comment. not sure if this / is intentional or not? There was a problem hiding this comment. typo, not sure how the slash got there. Will remove. There was a problem hiding this comment. @bnb I think that we have satisfied your questions, could we have another review. |
||||||
| field of continuous integration and deployment (CI/CD) is complex. The package being built shall be run through | ||||||
| a CI/CD pipeline. Several integrations are available for GitHub based repositories. (There are also alternatives to GitHub.) | ||||||
|
There was a problem hiding this comment.
we should try to substantiate this beyond just saying it exists. There was a problem hiding this comment. Agreed. Do we want more alternatives for Code like GitLabs or both code and CI/CD ? |
||||||
|
|
||||||
| For many open source projects on Github free integration environments are available. | ||||||
ghinks marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
|
||||||
| #### Acceptance Testing | ||||||
| Every package will have different criteria for acceptance. The maintenance committee has particular interest in making sure | ||||||
| that dependents of packages are not impacted by the release of a new version. We encourage liaison between both the | ||||||
| publisher and the dependents where possible. | ||||||
|
|
||||||
| ### Links to some useful tools | ||||||
|
|
||||||
| * [c8](https://www.npmjs.com/package/c8) | ||||||
| * [citgm](https://www.npmjs.com/package/citgm) | ||||||
| * [Codecov](https://www.npmjs.com/package/codecov) | ||||||
| * [Coveralls](https://www.npmjs.com/package/coveralls) | ||||||
| * [Jest](https://www.npmjs.com/package/jest) | ||||||
| * [Mocha](https://www.npmjs.com/package/mocha) | ||||||
| * [Nock](https://www.npmjs.com/package/nock) | ||||||
| * [nyc](https://www.npmjs.com/package/nyc) | ||||||
| * [tape](https://www.npmjs.com/package/tape) | ||||||
| * [Github CICD](https://docs.github.com/en/actions/building-and-testing-code-with-continuous-integration/about-continuous-integration) | ||||||
| * [Gitlab](https://about.gitlab.com/) | ||||||
| * [Circle CI](https://circleci.com/product/) | ||||||
| * [Travis CI](https://travis-ci.com/) | ||||||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this seems like it's slightly incorrect English.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how this is presented, "working correctly with the newer version of Node.js" is a value that's assumed but I don't think it's one that can in fact reasonably be assumed. From the project perspective yes we want things to work but we've not substantiated here why a maintainer would care.
Also, the phrasing is slightly confusing again.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm confused by this. The previous statement was future looking but how this is approached it's historical or at the very least in the present. Maybe this is just a prose issue, but the sentiment doesn't seem to match the previous section.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After re-reading this after a break of a few days I agree with you @bnb I think some changes are required here. The first sentence about node's strict release schedule should actually be removed and some re-phrasing after wards will be required.