Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System tests should cleanup after themselves #2216

Open
wants to merge 50 commits into
base: next
Choose a base branch
from

Conversation

ATorrise
Copy link
Contributor

@ATorrise ATorrise commented Aug 6, 2024

What It Does
This PR enhances the test framework to simplify and automate the cleanup of test resources, such as datasets, USS files, jobs, and local files. The TestEnvironment, ITestEnvironment, and TestUtils classes have been updated to track and delete resources after system tests, preventing any leftover artifacts on the mainframe or local filesystem.

How to Test : Run System Tests

  • Execute the system tests as usual. Verify that all created resources (datasets, USS files, jobs, local files) are automatically cleaned up after the tests complete.
  • Ensure that no traces of jobs, USS files, or datasets are left behind on the mainframe.

Review Checklist
I certify that I have:

Additional Comments

  • Test Stability
    • Occasionally, environment-specific issues may arise, especially with test datasets like IEFBR14 files. I'm not sure how to rectify this and definitely need areas using these files double-checked.
  • Unchanged Files:
    • zowe-cli\packages\zosjobs\__tests__\__system__\MonitorJobs.system.test.ts only had minor indentation changes and was left intact as it leaves no jobs behind on the mainframe.
    • If a test file cleans up all resources, it was left without changes to avoid unnecessary modifications since most of these test files are large and would require many lines of renovations.

Signed-off-by: ATorrise <amber.torrise@broadcom.com>
@ATorrise ATorrise changed the base branch from master to next August 6, 2024 13:33
@ATorrise ATorrise linked an issue Aug 6, 2024 that may be closed by this pull request
Copy link

codecov bot commented Aug 6, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.26%. Comparing base (937889e) to head (201ecc1).
Report is 2 commits behind head on next.

Additional details and impacted files
@@           Coverage Diff           @@
##             next    #2216   +/-   ##
=======================================
  Coverage   91.26%   91.26%           
=======================================
  Files         628      628           
  Lines       17821    17821           
  Branches     3820     3785   -35     
=======================================
  Hits        16264    16264           
  Misses       1556     1556           
  Partials        1        1           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
…iles from permanent residency, ,instead creating them from scratch everytime

Signed-off-by: ATorrise <amber.torrise@broadcom.com>
…as matching for this test. also took a while to identify which shell script was populating 2 jobs

Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
…ests for cliZosJobs

Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
@ATorrise ATorrise marked this pull request as ready for review August 26, 2024 14:58
@awharn
Copy link
Member

awharn commented Aug 28, 2024

Checks are currently failing due to failed imports. I think the package.json file for the cli-test-utils package needs to be updated to reflect the new dependencies on the Jobs and Files SDKs.

ATorrise and others added 7 commits August 28, 2024 11:30
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
ATorrise and others added 9 commits August 30, 2024 09:59
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
…d not IPLed to test

Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Copy link
Member

@zFernand0 zFernand0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the PR feedback.
I believe there are a couple of things were we could improve on this one.
Left them in the comments. 😋

* @memberof ITestEnvironment
*/
systemTestProperties: TestPropertiesSchema;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When using generics, it's advisable to keep the name short, and ideally something different to an existing interface.

you could get the same benefits by doing <T extends ITestPropertiesSchema> and use the value of T as the desired type for this properti (e.g. systemTestProperties: T)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

__tests__/__src__/environment/TestEnvironment.ts Outdated Show resolved Hide resolved
Signed-off-by: ATorrise <amber.torrise@broadcom.com>
Copy link

sonarcloud bot commented Sep 5, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Review/QA
Development

Successfully merging this pull request may close these issues.

TD(tests): Make sure that system tests cleanup after themselves
4 participants