Simplify git logic in regression tests #4725
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolved issues:
This PR addresses a discussion in #4701: #4701 (comment) to simplify the git logic in the regression tests.
Description of changes:
This change removes the
is_older_commit
andis_mainline
functions of the git module to allow more flexibility in how the tests are run. Previously, two commits being compared would either have to be on the same log or one of them would have to be mainline for the auto-detection to work. This change introduces environment variables which the user can invoke to identify which commit is the "baseline" and which commit is the "altered" code.These changes also modify the test storage scheme to still include a git commit identifier so build artifacts for a baseline commit are now stored by
tests/regression_artifacts/baseline/$commit_id_$test_name
. This enables the DiffProfile to access old and new commits in a simpler fashion than the previous git logic while also providing the commit id in the filename which could be useful as a reference to the change that the profile is associated with.Additionally, the github actions workflow is updated to reflect these changes when invoking the test through CI, so that mainline sets the environment variable to baseline and the PR branch is set to altered.
Call-outs:
Since these changes modify the regression test along with the workflow, the workflow will run on this PR and fail. This is because the mainline run through the regression tests in CI will not store files in the same way that the revised approach to the tests does.
Testing:
I have run the CI workflow locally and it works as intended.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.