Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

E2e test ci #986

Merged
merged 26 commits into from
Dec 15, 2020
Merged

E2e test ci #986

merged 26 commits into from
Dec 15, 2020

Conversation

Oxiang
Copy link
Contributor

@Oxiang Oxiang commented Nov 30, 2020

Problem

Lack of end-to-end front-end testing in the CI pipeline.

Solution

Add end-to-end testing into GitHub action, triggered by push/pull request.
This will spin up a development environment, with 4 minutes timeout.
Subsequently, it will run testcafe test on headless chrome browser.

Others changes

The Otp rate limit is set higher in during testing so that all the tests are able to run without encountering rate limit error.

Add end-to-end folder under jest ignore config to stop jest from running the testcafe tests.

Test Coverage

The test cases originated from the wiki checklist: https://github.com/opengovsg/GoGovSG/wiki/Release-Checklist

Test coverage includes:

Login Page

  • It should respond with invalid email when email does not end with .gov.sg
  • It should not allow submission when email is invalid
  • Invalid OTP should not log the user in
  • Valid OTP should log the user in
  • After trying to enter wrong OTP 3 times, it should respond with OTP not found/expired (a new OTP must be requested)
  • Visiting /user should redirect to login page when not logged in

Sessions

  • Shows the homepage if user does not have an existing session
  • Redirects to /user if user has an existing session (ie logged in previously on the same browser)

URL creation

  • The create url modal opens when the "Create link" button is clicked.
  • It should populate the short url input box on the create url modal with a random string when the refresh icon on the short url input box is pressed
  • It should show the new short url on the users’ links table when a new link is created
  • The new short url should be highlighted on the users' links table when a new link is created
  • It should prevent creation of short urls pointing to long urls hosted on blacklisted domains
  • It should show an success snackbar when a new url has been added
  • It should show an error below the file input when a file larger than 10MB is chosen
  • It should disable the submit button when a file larger than 10MB is chosen
  • It should show the short url on users' link table when a new file link is created
  • The new short url should be highlighted on the users' links table when a new file link is created
  • It should show an success snackbar when a new file link has been added

URL Searching and Download

  • Searching on the user page search bar shows links that are relevant to the search term

Transition Page

  • Accessing a short link with a trailing slash should not result in a broken transition page.
  • Accessing a short link for the first time shows the transition page.
  • After 6 seconds, user is redirected from the transition page to the correct destination long url.
  • A skip button is displayed immediately.
  • Visiting the same short link again does not show the transition page.

Drawer, Editing and Toaster logic

  • Drawer should open with the correct long url and state when a short url row is clicked
  • Drawer can be closed on clickaway or when the close button is clicked.
  • It should set short url active or inactive immediately when the toggle is switched (any caching for that short url is cleared)
  • It should show a success snackbar when long url is changed
  • It should show a success snackbar when link is transferred to another user.
  • Url is updated/saved when user enters a new url, then clicks "save"
  • Url is reverts to original when user enters a new url, then re-opens the drawer without clicking "save"
  • Error validation (red underline + helperText) appears when value in edit long url textfield is invalid
  • "Save" button is disabled (grey and unclickable) when value in edit long url textfield is invalid
  • Unsuccessful link transfers do not close the drawer.
  • Successful link transfers closes the drawer.
  • Toasters to disappear when user clicks on the X only
  • Toasters to disappear after 5sec
  • Toasters to not disappear on clickaway (i.e. to prevent premature closure when user clickaway to save url)
  • It should show an error and not amend the file when a file of size >10mb is selected on replace file

User page - Sort & Filter

  • Clicking on the button at the end of the search input should open the sort and filter panel
  • Links should be sorted by their created time in descending order when enabling sort by Date of creation and clicking apply
  • Inactive links should be filtered out by checking only Active and clicking apply
  • Active links should be filtered out by checking only Inactive and clicking apply
  • File links should be filtered out by checking only Link and clicking apply
  • Non-file links should be filtered out by checking only File and clicking apply
  • Panel should be closed when clicking outside of it
  • Panel should be closed and links sorted by created time with no filtering after clicking on reset. All links and files should be visible.
  • Panel should be closed when apply is clicked

New dev dependencies:

  • testcafe

@Oxiang Oxiang force-pushed the e2e-test-ci branch 3 times, most recently from 963d72e to 65eacb2 Compare December 2, 2020 05:11
@liangyuanruo liangyuanruo requested review from liangyuanruo and removed request for liangyuanruo December 8, 2020 06:20
Copy link
Member

@yong-jie yong-jie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some feedback and discussion points. if it's more efficient to address them in a call i'm up for it

.github/workflows/ci.yml Outdated Show resolved Hide resolved
test/end-to-end/Url-Creation.test.ts Outdated Show resolved Hide resolved
test/end-to-end/User-Page.test.ts Outdated Show resolved Hide resolved
test/end-to-end/Drawer-Login.test.ts Outdated Show resolved Hide resolved
test/end-to-end/Drawer-Login.test.ts Outdated Show resolved Hide resolved
test/end-to-end/Drawer-Login.test.ts Outdated Show resolved Hide resolved
test/end-to-end/Drawer-Login.test.ts Outdated Show resolved Hide resolved
test/end-to-end/Transition-Page.test.ts Outdated Show resolved Hide resolved
test/end-to-end/Transition-Page.test.ts Outdated Show resolved Hide resolved
test/end-to-end/util/dummyFile.txt Outdated Show resolved Hide resolved
@Oxiang Oxiang force-pushed the e2e-test-ci branch 2 times, most recently from c0d9c47 to 1a4dd5b Compare December 9, 2020 09:46
Copy link
Member

@yong-jie yong-jie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick for function documentation + eslint

also, could we rename the files to remove dashes like Drawer-Login.test.ts to DrawerLogin.test.ts etc? This is for consistency with the rest of the codebase 🙏

test/end-to-end/util/First-Link-Handle.ts Outdated Show resolved Hide resolved
test/end-to-end/util/helpers.ts Outdated Show resolved Hide resolved
Copy link
Member

@yong-jie yong-jie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work! feel free to merge once package-lock conflicts are resolved

- delete download folder library
- shift testcafe yml to ci yml
- add new otp rate limit config for end to end test
- change otp rate limit to 10 for development mode
- stick otp rate limit to 5 for production and test
- rename testcafe test in ci
- refactor the tests
- add login procedure
- remove directory test
- reduce setup waiting time to three minutes
- refactor config file
- remove bottom comments
- remove ip tracking test
- remove reset otp test
@Oxiang Oxiang merged commit b52af2d into develop Dec 15, 2020
@Oxiang Oxiang deleted the e2e-test-ci branch December 15, 2020 08:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants