Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOCK-2471: Fix flaky UI integration tests #1857

Merged
merged 10 commits into from
Oct 24, 2023

Conversation

svonworl
Copy link
Contributor

@svonworl svonworl commented Oct 23, 2023

Description
This PR fixes some of the flakiness in the CircleCI UI2 tests, specifically related to failures due to problems in the "my tools" interface. Since it was related, the fix to https://ucsc-cgl.atlassian.net/browse/SEAB-5970 is included in this PR.

After I fixed the "my tools" issues, the CircleCI runs started consistently failing because of a browser crash in myworkflows.ts. This was weird, because I didn't change anything that should have made it more frequent. However, it needed to be fixed, and typically, per the internets, the crashes happen because the browser runs out of memory. One of the describes in myworkflows.ts was huge, so I split it into pieces, in the hope that maybe it would reduce the memory load somewhere in the CircleCI/Chromium internals (maybe in the recording logic?). Now, the tests seem to pass, although it's very possible I didn't address the root issue and it is not actually fixed. Potentially Heisenbuggy. Let's run it this way for a bit, and if the crashes recur, we can dig deeper...

So, in total, there are four significant changes:

  1. In MyToolComponent, we now wait for both the "my" tools and "my" apptools responses to arrive before processing them. combineLatest produces a series of values like so [(null, null), (A, null), (A, B)], and thus, previously, to display the entry denoted by current URL, the code was selecting/requesting an entry from the "my" entries based on incomplete information (from the list of old-style tools or apptools, whichever was updated in the combined observable, but not both), then another entry based on full information (the old-style tools + apptools). My best theory is that, occasionally, the response for the wrong entry arrived after the response for the correct entry, causing the wrong entry to be displayed, which triggered the test failures.
  2. We now observe workflow$ and set the template flag that causes delegation to the apptool template.
  3. We now delegate correctly to the workflow template from the tools template.
  4. Split large describe block in myworkflows.ts.

Review Instructions
Check the CircleCI runs and confirm that the UI2 tests are not failing due to problems in the "my tools" interface Primarily, this will be failures in mytools.ts, deletion.ts, and archival.ts (forthcoming). Also, confirm that myworkflows.ts is consistently running fine.

Issues
https://ucsc-cgl.atlassian.net/browse/DOCK-2471
#5696
https://ucsc-cgl.atlassian.net/browse/SEAB-5970

Security
If there are any concerns that require extra attention from the security team, highlight them here.

Please make sure that you've checked the following before submitting your pull request. Thanks!

  • Check that your code compiles by running npm run build
  • Ensure that the PR targets the correct branch. Check the milestone or fix version of the ticket.
  • If this is the first time you're submitting a PR or even if you just need a refresher, consider reviewing our style guide
  • Do not bypass Angular sanitization (bypassSecurityTrustHtml, etc.), or justify why you need to do so
  • If displaying markdown, use the markdown-wrapper component, which does extra sanitization
  • Do not use cookies, although this may change in the future
  • Run npm audit and ensure you are not introducing new vulnerabilities
  • Do due diligence on new 3rd party libraries, checking for CVEs
  • Don't allow user-uploaded images to be served from the Dockstore domain
  • If this PR is for a user-facing feature, create and link a documentation ticket for this feature (usually in the same milestone as the linked issue). Style points if you create a documentation PR directly and link that instead.
  • Check whether this PR disables tests. If it legitimately needs to disable a test, create a new ticket to re-enable it in a specific milestone.

@@ -13,7 +13,8 @@
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<div *ngIf="tool || !displayAppTool; else appTool">
<app-workflow *ngIf="displayAppTool" [isWorkflowPublic]="isToolPublic"></app-workflow>
<div *ngIf="tool && !displayAppTool">
Copy link
Contributor Author

@svonworl svonworl Oct 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous logic that controlled whether to use the tool template (for old-style tools) or the workflow template (for apptools) was incorrect. It's fixed here, and I moved all of the code that conditionally delelegates to the workflow template to the top, to make it more clear what's happening without having to scan the entire template.

@codecov
Copy link

codecov bot commented Oct 23, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (0cbefeb) 40.49% compared to head (c67bb5b) 40.53%.
Report is 1 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1857      +/-   ##
===========================================
+ Coverage    40.49%   40.53%   +0.04%     
===========================================
  Files          364      365       +1     
  Lines        11268    11283      +15     
  Branches      2891     2896       +5     
===========================================
+ Hits          4563     4574      +11     
- Misses        4405     4407       +2     
- Partials      2300     2302       +2     

see 5 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@sonarcloud
Copy link

sonarcloud bot commented Oct 23, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

@@ -43,7 +43,7 @@ describe('Entry Deletion', () => {
function goToPrivatePage(entry: Entry): void {
cy.visit('/');
cy.visit(`${entry.myPrefix}/${entry.path}`);
cy.wait(2000);
cy.contains(entry.path);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 This is indeed better than a hard-coded wait.

@svonworl svonworl merged commit ca76a6b into develop Oct 24, 2023
20 checks passed
@svonworl svonworl deleted the feature/dock-2471/fix-flaky-ui-integration-tests branch October 24, 2023 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants