Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Determine base branch dynamically #1791

Merged
merged 59 commits into from
Jun 1, 2023

Conversation

kathy-t
Copy link
Contributor

@kathy-t kathy-t commented May 29, 2023

Description
This PR removes base_branch from the package.json config and adds a script that can determine the base branch dynamically for PRs. The CircleCI jobs that require the base branch, audit and accessibility_test_base call this script to get the value.

Unfortunately there's no built-in way to get the base branch of a PR on CircleCI so the script uses CircleCI's CIRCLE_PULL_REQUEST built-in environment variable that contains the PR's URL to extract the PR number then it uses the GitHub API to get the base branch of the PR. Since the script depends on the existence of a PR, the jobs that need a base branch have a step that check if a PR exists before proceeding. If no PR exists, it exits gracefully.

While I was in there, I removed duplicated code for the accessibility test job and I created reusable commands for the repeated commands that I saw.

I also created reusable jobs for the integration tests because the only difference between each definition was the integration test group name.

EDIT:
This PR moves the npm audit and accessibility tests from CircleCI to GitHub Actions. GitHub Actions support the pr_request event trigger and it provides a built-in environment variable for the base branch of a PR. The workflows will only run when a pull_request event's activity type is opened, synchronize, or reopened. Thus this will not run when the PR is merged to the target branch, which I think is okay because there's not really a "current"/"base" branch when running on the target branch.

Noting that in moving the audit test to GitHub Actions, the CircleCI upload_to_s3 job will happen even if the audit fails. It previously required that the unit tests and audit pass.

The CircleCI audit job previously was not a required check in GitHub. Should the GitHub Action audit job be a required job? Tagging @denis-yuen because I think it has to be configured via the repo settings

Here are the audit and accessibility GitHub Action runs for this PR:

Review Instructions
Create a PR (or find an existing one) and verify that the audit and accessibility jobs are running via GitHub Actions. Can view the jobs accessibility runs here and the audit runs here.

All CircleCI jobs should also pass.

Issue
https://ucsc-cgl.atlassian.net/browse/SEAB-5531

Security
If there are any concerns that require extra attention from the security team, highlight them here.

Please make sure that you've checked the following before submitting your pull request. Thanks!

  • Check that your code compiles by running npm run build
  • Ensure that the PR targets the correct branch. Check the milestone or fix version of the ticket.
  • If this is the first time you're submitting a PR or even if you just need a refresher, consider reviewing our style guide
  • Do not bypass Angular sanitization (bypassSecurityTrustHtml, etc.), or justify why you need to do so
  • If displaying markdown, use the markdown-wrapper component, which does extra sanitization
  • Do not use cookies, although this may change in the future
  • Run npm audit and ensure you are not introducing new vulnerabilities
  • Do due diligence on new 3rd party libraries, checking for CVEs
  • Don't allow user-uploaded images to be served from the Dockstore domain
  • If this PR is for a user-facing feature, create and link a documentation ticket for this feature (usually in the same milestone as the linked issue). Style points if you create a documentation PR directly and link that instead.
  • Check whether this PR disables tests. If it legitimately needs to disable a test, create a new ticket to re-enable it in a specific milestone.

@sonarcloud
Copy link

sonarcloud bot commented May 31, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@kathy-t
Copy link
Contributor Author

kathy-t commented May 31, 2023

Re-requesting reviews because I moved the audit and accessibility tests from CircleCI to GitHub Actions. Read the updated PR description for more details

runs-on: ubuntu-20.04
services:
postgres:
image: postgres:13.3
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't provide DockerHub credentials for the postgres and elasticsearch image, but the CircleCI config does:

auth:
username: dockstoretestuser
password: $DOCKERHUB_PASSWORD

I assume it was added because of rate-limiting? Do we need it here, and if so, do these credentials already exist in our GitHub secrets (I can't see them)?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At one point there was a concern that DockerHub was going to limit pulls; so we started to add credentials. But then I believe CircleCI made some sort of deal with DockerHub so that pulls from CircleCI wouldn't be rate-limited (or have a higher limit).

My guess is we might be OK; the unauthenticated limit is 100 per 6 hours per IP, and presumably not all our actions run from the same IP.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd hope that github has some form of Docker image caching and/or a similar deal.

runs-on: ubuntu-20.04
services:
postgres:
image: postgres:13.3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At one point there was a concern that DockerHub was going to limit pulls; so we started to add credentials. But then I believe CircleCI made some sort of deal with DockerHub so that pulls from CircleCI wouldn't be rate-limited (or have a higher limit).

My guess is we might be OK; the unauthenticated limit is 100 per 6 hours per IP, and presumably not all our actions run from the same IP.

@denis-yuen
Copy link
Member

The CircleCI audit job previously was not a required check in GitHub. Should the GitHub Action audit job be a required job? Tagging @denis-yuen because I think it has to be configured via the repo settings

Let's see over the next couple of months how it performs

Copy link
Member

@denis-yuen denis-yuen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this ended up being a lot more work than expected, appreciate powering through it

runs-on: ubuntu-20.04
services:
postgres:
image: postgres:13.3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd hope that github has some form of Docker image caching and/or a similar deal.

@denis-yuen
Copy link
Member

Noting that in moving the audit test to GitHub Actions, the CircleCI upload_to_s3 job will happen even if the audit fails. It previously required that the unit tests and audit pass.

Think this is generally ok

@kathy-t kathy-t merged commit b4fa975 into develop Jun 1, 2023
@kathy-t kathy-t deleted the feature/seab-5531/determine-base-branch branch June 1, 2023 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants