-
Notifications
You must be signed in to change notification settings - Fork 146
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Integration tests are not upgrading to the actual build being tested #3461
Comments
This affects more than the test I referenced in the description of the issue. Seems that lots of tests are affected by this.
https://github.com/elastic/elastic-agent/blob/main/testing/integration/upgrade_test.go#L159 Calls |
Pinging @elastic/elastic-agent (Team:Elastic-Agent) |
@pierrehilbert I am going to take this issue as it is blocking my ability to continue with Windows integration runner. This is a critical problem that we need to focus on for the testing framework. If we are not testing the actual build then the tests are useless. With multiple tests not upgrading to the actual built version of Elastic Agent it makes it impossible to test new code (as we are not really testing the code we expect), which prevents me from continuing to work on Windows. |
Fine for me! Thx! |
I have been digging into why tests are still failing to install on Windows. While I did lots of manual testing of PR #3384 confirming that it was indeed stopping and killing the watcher, I was shocked to find similar issues still occurring on Windows with my windows integration runner.
After spending too much time thinking that it could be the code to find the watcher, and too much time pausing the tests and inspecting running processes I identified the issue.
The issue is that some tests are not even upgrading to the build of the Elastic Agent that is under test. It is instead upgrading to the SNAPSHOT build from the artifacts. This can be seen clearly here:
TestStandaloneUpgrade/Upgrade_7.17.13_to_8.11.0-SNAPSHOT
The interesting part is this line (I removed the prefix), which is after the upgrade has completed.
upgrade_test.go:530: current agent state: {Info:{ID:c2870809-3718-4ccc-b334-49b07f424cd3 Version:8.11.0 Commit:b1d2e6b04062b8572718e583590afe678579bf9d BuildTime:2023-09-21 15:18:03 +0000 UTC Snapshot:true} State:HEALTHY Message:Running Components:[] FleetState:STOPPED FleetMessage:Not enrolled into Fleet}
The
Commit
andBuildTime
does not match by build that was placed on the host under test (aka. the instance):This means that the test is not even really testing the build of Elastic Agent we are expecting and that is the reason that the watcher is not being killed. I would not see the benefit of #3384 until a new snapshot is built and published.
The text was updated successfully, but these errors were encountered: