-
Notifications
You must be signed in to change notification settings - Fork 313
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Acceptance Test POC and Setup #41
Conversation
Thanks for the demo! What I'm interested in investigating is:
This would serve as the foundation for our We'll further expand this to include Vue, Angular, etc. by convention with the URL. (For example, |
And some additional questions:
|
@slaymance Great question:
Initially I considered https://github.com/cucumber/common/blob/main/gherkin/MARKDOWN_WITH_GHERKIN.md, but have since changed my mind. It's a 2-day door, but keeping the widely-supported Gherkin syntax in
I did some quick Googling and found https://github.com/mathanpec/react-native-detox-cucumber, https://github.com/Ahmed-Ali/Cucumberish, and others.
Today, we store We'd have to work with security on this in a public repo, but I'm interested in re-using examples across different environments. The My current idea is to leverage Another alternative is to serialize them and load There's also https://docs.cypress.io/api/cypress-api/env. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Solid first PR to get this all set up! Thanks for this. Left a few comments that aren't blocking really but I think would be worth considering.
Issue #, if available: Issue #33
Acceptance Tests
This PR is meant as a proof of concept and initial setup for our Acceptance Testing strategy. Our Acceptance Tests are end-to-end tests which cover supported customer use cases of our Amplify UI components and give us confidence that component rewrites, refactors, and migrations continue to support said use cases.
Tooling
Setup
Cypress requires a running development instance of a sample application on a specified URL that it may interact with in order to test assertions. For the purposes of this POC, I've created a separate Next.js page in our react docs which uses the new
Authenticator
component. I've also configured Amplify with my own credentials for demonstration purposes.Now we can run
yarn dev
from the repository root directory and use the above page for our Cypress tests.Testing Work Flow
Writing Cucumber test definitions
The first step to creating an acceptance test is to use Cucumber's Gherkin language to declare the test steps in a Given, When, Then syntax. Within the
acceptance-tests
workspace (acceptance-tests
->cypress
->tests
->acceptance
), Cucumber tests are defined in files with the.feature
file extension (e.g. Authentication.feature). A cucumber test definition can look like this:The keywords
Given
,When
,Then
,And
, andBut
indicate step definitions for a given scenario. While this Cucumber definition isn't a functioning test yet, it provides a syntax which can be used across frameworks and across test runner implementations.For example, when aiming for feature parity with a native implementation of Amplify UI which won't use Cypress as a test runner, this Cucumber test can be used alongside whichever test runner is used for the native implementation to allow for Behavior Driven Development when writing code.
This syntax is also intuitive regardless of coding language proficiency and can be used to communicate test coverage of use cases for external stakeholders.
Using Cypress for application actions and assertions
Now that our Cypress test scenario is defined, we need to write the actual logic which executes each of the steps defined in our
Authentication.feature
scenario. Next to our feature file, we create a directory of the same name with a typescript file of the same name:This file,
Authentication.ts
will use Cypress and cypress-cucumber-preprocessor to define the logic for each step through interacting with our sample application and making assertions about outcomes of interactions:Note: the methods
.findByRole
and.findByLabelText
are part of Testing Library and not native to the Cypress API. These will be discussed below.Each step for every scenario is clearly defined and its logic laid out with the
Then
step serving as our assertion.Using Testing Library for user-centric and accessible actions
As mentioned above, we defined the logic of our test steps using methods from Testing Library. These allow us to interact with our sample application using methods that more closely align with how a user would interact with an application. Instead of using data attributes for elements and querying by those selectors, we instead can query our UI for elements based on their roles or labels. This forces us to test our application with accessibility in mind first since being unable to query an element on a page by role or label indicates it may lack accessibility.
Running Tests
From the
acceptance-tests
workspace, Cypress tests can be run with the following scripts:yarn cypress:open
: opens a browser which will be used for the test runneryarn cypress:run
: test runner executes in a headless Electron browserNext Steps
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.