Uniquify the stand-alone step for checking agent status #993

adam-stokes · 2021-04-06T15:16:42Z

There were 2 steps identical in both the stand-alone and fleet test suites.
Running the stand-alone test suite was picking up the step from the fleet test
suite and trying to reference the FleetTestSuite structure which did not hold
any of the agent information (like the hostname) for the stand alone tests.

This fixes it so that the standalone test step is being referenced in the
correct test suite.

Signed-off-by: Adam Stokes 51892+adam-stokes@users.noreply.github.com

What does this PR do?

Makes the steps unique between stand-alone and fleet test suites

Why is it important?

The stand-alone tests were incorrectly picking up steps from the fleet test suite

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have run the Unit tests for the CLI, and they are passing locally
I have run the End-2-End tests for the suite I'm working on, and they are passing locally
I have noticed new Go dependencies (run make notice in the proper directory)

Author's Checklist

[ ]

How to test this PR locally

e2e/_suites/fleet/ > godog -t '@run_fleet_server'

Related issues

Closes unable to reference sats.Hostname when checking standalone agent status in fleet #984

There were 2 steps identical in both the stand-alone and fleet test suites. Running the stand-alone test suite was picking up the step from the fleet test suite and trying to reference the FleetTestSuite structure which did not hold any of the agent information (like the hostname) for the stand alone tests. This fixes it so that the standalone test step is being referenced in the correct test suite. Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

mdelapenya · 2021-04-06T15:19:04Z

Caused by #978

jalvz · 2021-04-06T15:37:41Z

he, you pushed 15 minutes faster than me ;P #994 :)
I believe this is not sufficient, thou

jalvz · 2021-04-06T15:42:24Z

btw, I think ideally as a next follow up we should unify standalone and fleet code, not able to use the same phrase for the same thing is a bit finicky: to add an step in one place you need to make sure that the same step does not exist in the other.
wdyt?

elasticmachine · 2021-04-06T15:54:25Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #993 opened
Start Time: 2021-04-06T15:16:59.215+0000
Duration: 37 min 18 sec
Commit: 9f708be

Test stats 🧪

Test	Results
Failed	0
Passed	138
Skipped	0
Total	138

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	138
Skipped	0
Total	138

There were 2 steps identical in both the stand-alone and fleet test suites. Running the stand-alone test suite was picking up the step from the fleet test suite and trying to reference the FleetTestSuite structure which did not hold any of the agent information (like the hostname) for the stand alone tests. This fixes it so that the standalone test step is being referenced in the correct test suite. Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> (cherry picked from commit d9e3b51)

mdelapenya · 2021-04-07T05:32:39Z

btw, I think ideally as a next follow up we should unify standalone and fleet code, not able to use the same phrase for the same thing is a bit finicky: to add an step in one place you need to make sure that the same step does not exist in the other.
wdyt?

You are totally right. See my comment in #994 (comment). We'd need to merge both structs to avoid that duplicity in steps, which was in place from the beginning as a separation of concerns for both modes. But it totally makes sense to unify them now.

mdelapenya · 2021-04-07T05:36:04Z

I believe this is not sufficient, thou

We discussed yesterday about removing the volume in a follow-up PR, but not only that, because we discovered that the code to download the configuration file rom the original Beats PR/branch is not needed anymore, as the downloaded binary (from GCP) already contains that file. PTAL at #981 (comment)

There were 2 steps identical in both the stand-alone and fleet test suites. Running the stand-alone test suite was picking up the step from the fleet test suite and trying to reference the FleetTestSuite structure which did not hold any of the agent information (like the hostname) for the stand alone tests. This fixes it so that the standalone test step is being referenced in the correct test suite. Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> (cherry picked from commit d9e3b51) Co-authored-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

…tic#993) (elastic#996)" This reverts commit 9818d67.

…tic#993) (elastic#997)" This reverts commit 435c45b.

* Revert "Uniquify the stand-alone step for checking agent status (#993) (#996)" This reverts commit 9818d67. * Revert "Add a standalone test scenario for fleet server mode (#978) (#987)" This reverts commit a42c0ce.

* Revert "Uniquify the stand-alone step for checking agent status (#993) (#997)" This reverts commit 435c45b. * Revert "Add a standalone test scenario for fleet server mode (#978) (#988)" This reverts commit 601064a.

* master: Remove the agent config file parameters for stand alone (elastic#983) Uniquify the stand-alone step for checking agent status (elastic#993)

There were 2 steps identical in both the stand-alone and fleet test suites. Running the stand-alone test suite was picking up the step from the fleet test suite and trying to reference the FleetTestSuite structure which did not hold any of the agent information (like the hostname) for the stand alone tests. This fixes it so that the standalone test step is being referenced in the correct test suite. Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

* master: chore: add debug info for the payload (elastic#1044) chore: add debug traces for the webhook payload (elastic#1043) fix: wrong interpolation (elastic#1042) Update Elastic Agent to not use Kibana (elastic#1036) fix: apply X version for non-master branches (elastic#1037) fix: add NodeJS to PATH (elastic#1035) fix: use an agent when building kibana (elastic#1030) fix(jjb): use a branch that exists (elastic#1029) remove uninstall step (elastic#1017) fix: delay checking stale agent version until it's used (elastic#1016) fix: use same JJB than in custom kibana (elastic#1010) chore: simplify PR template (elastic#1011) feat: support passing KIBANA_VERSION (elastic#905) [mergify] assign the original author (elastic#1009) Remove the agent config file parameters for stand alone (elastic#983) Uniquify the stand-alone step for checking agent status (elastic#993)

* Move kibana into internals, update fleet test suite * migrate docker-compose related code to internal layout * move docker related code to internal layout * move git related code to internal layout * move common attributes into internal common file system layout * move elasticsearch specifics into its own filesystem layout * move installer based code to internal layout * move shell related code to internal layout * move sanitizer code to internal layout * move io related code to internal layout * move utils into internal layout * Update package integration querying/altering * move curl to internal layout * move helm to internal layout * move kubectl into internal layout * move state internal filesystem * cleanup config in stand-alone * remove unused files * Uniquify the stand-alone step for checking agent status (#993) There were 2 steps identical in both the stand-alone and fleet test suites. Running the stand-alone test suite was picking up the step from the fleet test suite and trying to reference the FleetTestSuite structure which did not hold any of the agent information (like the hostname) for the stand alone tests. This fixes it so that the standalone test step is being referenced in the correct test suite. * Remove the agent config file parameters for stand alone (#983) * Update helm/metricbeat tests to use new layout * Fix policy endpoint update * fix panic on helm init * Fix step reference as this being merged seperately * Update function call to correct standalone step * Fix merge conflict * update ProfileEnv query/set for KibanaVersion * More fixes to agent endpoint security checks * update backend feature to call out endpoint in step * use common.TimeoutFactor in docker checkprocess state * Update adding endpoint integration * enable features for fleet server * not necessary to enroll after install * wait for filebeat/metricbeat before restarts * clear out fts.CurrentToken during beforeScenario * attach system integration on deploy * enroll if rpm * dont store fleet policy * update kibana config for latest fleet server * Update e2e/_suites/fleet/fleet.go * Update e2e/_suites/fleet/fleet.go * Update e2e/_suites/fleet/fleet.go * Update .pre-commit-config.yaml * Update e2e/Makefile * rename apt -> deb for installer type * execute docker start/stop with timeout between * fixes fleet_server scenario * Utilize fleet server in all tests * Fix enrollment url for fleet server * Query elasticsearch logs for endpoint security event changes * Increase search result size for ES * Fix issue with fleet server restarting continuously * unpin kibana pr now that most major breakage is resolved * force unenroll * for new fleet bootstrap on re-enrollment * Fix unenrollment * Add timeout safeguard to elastic-agent execution In some cases such as attempting to re-enroll with a revoked token, the elastic-agent will retry indefinitely. This fix adds a safeguard utilizing 'timeout' command prepended to the elastic-agent command so that it will timeout after TimeoutFactor Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

* Move kibana into internals, update fleet test suite * migrate docker-compose related code to internal layout * move docker related code to internal layout * move git related code to internal layout * move common attributes into internal common file system layout * move elasticsearch specifics into its own filesystem layout * move installer based code to internal layout * move shell related code to internal layout * move sanitizer code to internal layout * move io related code to internal layout * move utils into internal layout * Update package integration querying/altering * move curl to internal layout * move helm to internal layout * move kubectl into internal layout * move state internal filesystem * cleanup config in stand-alone * remove unused files * Uniquify the stand-alone step for checking agent status (#993) There were 2 steps identical in both the stand-alone and fleet test suites. Running the stand-alone test suite was picking up the step from the fleet test suite and trying to reference the FleetTestSuite structure which did not hold any of the agent information (like the hostname) for the stand alone tests. This fixes it so that the standalone test step is being referenced in the correct test suite. * Remove the agent config file parameters for stand alone (#983) * Update helm/metricbeat tests to use new layout * Fix policy endpoint update * fix panic on helm init * Fix step reference as this being merged seperately * Update function call to correct standalone step * Fix merge conflict * update ProfileEnv query/set for KibanaVersion * More fixes to agent endpoint security checks * update backend feature to call out endpoint in step * use common.TimeoutFactor in docker checkprocess state * Update adding endpoint integration * enable features for fleet server * not necessary to enroll after install * wait for filebeat/metricbeat before restarts * clear out fts.CurrentToken during beforeScenario * attach system integration on deploy * enroll if rpm * dont store fleet policy * update kibana config for latest fleet server * Update e2e/_suites/fleet/fleet.go * Update e2e/_suites/fleet/fleet.go * Update e2e/_suites/fleet/fleet.go * Update .pre-commit-config.yaml * Update e2e/Makefile * rename apt -> deb for installer type * execute docker start/stop with timeout between * fixes fleet_server scenario * Utilize fleet server in all tests * Fix enrollment url for fleet server * Query elasticsearch logs for endpoint security event changes * Increase search result size for ES * Fix issue with fleet server restarting continuously * unpin kibana pr now that most major breakage is resolved * force unenroll * for new fleet bootstrap on re-enrollment * Fix unenrollment * Add timeout safeguard to elastic-agent execution In some cases such as attempting to re-enroll with a revoked token, the elastic-agent will retry indefinitely. This fix adds a safeguard utilizing 'timeout' command prepended to the elastic-agent command so that it will timeout after TimeoutFactor Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> (cherry picked from commit 5f59670) # Conflicts: # e2e/Makefile # e2e/_suites/fleet/ingest_manager_test.go # e2e/_suites/fleet/stand-alone.go # e2e/_suites/fleet/world.go

* Move kibana into internals, update fleet test suite * migrate docker-compose related code to internal layout * move docker related code to internal layout * move git related code to internal layout * move common attributes into internal common file system layout * move elasticsearch specifics into its own filesystem layout * move installer based code to internal layout * move shell related code to internal layout * move sanitizer code to internal layout * move io related code to internal layout * move utils into internal layout * Update package integration querying/altering * move curl to internal layout * move helm to internal layout * move kubectl into internal layout * move state internal filesystem * cleanup config in stand-alone * remove unused files * Uniquify the stand-alone step for checking agent status (#993) There were 2 steps identical in both the stand-alone and fleet test suites. Running the stand-alone test suite was picking up the step from the fleet test suite and trying to reference the FleetTestSuite structure which did not hold any of the agent information (like the hostname) for the stand alone tests. This fixes it so that the standalone test step is being referenced in the correct test suite. * Remove the agent config file parameters for stand alone (#983) * Update helm/metricbeat tests to use new layout * Fix policy endpoint update * fix panic on helm init * Fix step reference as this being merged seperately * Update function call to correct standalone step * Fix merge conflict * update ProfileEnv query/set for KibanaVersion * More fixes to agent endpoint security checks * update backend feature to call out endpoint in step * use common.TimeoutFactor in docker checkprocess state * Update adding endpoint integration * enable features for fleet server * not necessary to enroll after install * wait for filebeat/metricbeat before restarts * clear out fts.CurrentToken during beforeScenario * attach system integration on deploy * enroll if rpm * dont store fleet policy * update kibana config for latest fleet server * Update e2e/_suites/fleet/fleet.go * Update e2e/_suites/fleet/fleet.go * Update e2e/_suites/fleet/fleet.go * Update .pre-commit-config.yaml * Update e2e/Makefile * rename apt -> deb for installer type * execute docker start/stop with timeout between * fixes fleet_server scenario * Utilize fleet server in all tests * Fix enrollment url for fleet server * Query elasticsearch logs for endpoint security event changes * Increase search result size for ES * Fix issue with fleet server restarting continuously * unpin kibana pr now that most major breakage is resolved * force unenroll * for new fleet bootstrap on re-enrollment * Fix unenrollment * Add timeout safeguard to elastic-agent execution In some cases such as attempting to re-enroll with a revoked token, the elastic-agent will retry indefinitely. This fix adds a safeguard utilizing 'timeout' command prepended to the elastic-agent command so that it will timeout after TimeoutFactor Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> (cherry picked from commit 5f59670) # Conflicts: # e2e/Makefile # e2e/_suites/fleet/ingest_manager_test.go # e2e/_suites/fleet/stand-alone.go # e2e/_suites/fleet/world.go Co-authored-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

* Move kibana into internals, update fleet test suite * migrate docker-compose related code to internal layout * move docker related code to internal layout * move git related code to internal layout * move common attributes into internal common file system layout * move elasticsearch specifics into its own filesystem layout * move installer based code to internal layout * move shell related code to internal layout * move sanitizer code to internal layout * move io related code to internal layout * move utils into internal layout * Update package integration querying/altering * move curl to internal layout * move helm to internal layout * move kubectl into internal layout * move state internal filesystem * cleanup config in stand-alone * remove unused files * Uniquify the stand-alone step for checking agent status (#993) There were 2 steps identical in both the stand-alone and fleet test suites. Running the stand-alone test suite was picking up the step from the fleet test suite and trying to reference the FleetTestSuite structure which did not hold any of the agent information (like the hostname) for the stand alone tests. This fixes it so that the standalone test step is being referenced in the correct test suite. * Remove the agent config file parameters for stand alone (#983) * Update helm/metricbeat tests to use new layout * Fix policy endpoint update * fix panic on helm init * Fix step reference as this being merged seperately * Update function call to correct standalone step * Fix merge conflict * update ProfileEnv query/set for KibanaVersion * More fixes to agent endpoint security checks * update backend feature to call out endpoint in step * use common.TimeoutFactor in docker checkprocess state * Update adding endpoint integration * enable features for fleet server * not necessary to enroll after install * wait for filebeat/metricbeat before restarts * clear out fts.CurrentToken during beforeScenario * attach system integration on deploy * enroll if rpm * dont store fleet policy * update kibana config for latest fleet server * Update e2e/_suites/fleet/fleet.go * Update e2e/_suites/fleet/fleet.go * Update e2e/_suites/fleet/fleet.go * Update .pre-commit-config.yaml * Update e2e/Makefile * rename apt -> deb for installer type * execute docker start/stop with timeout between * fixes fleet_server scenario * Utilize fleet server in all tests * Fix enrollment url for fleet server * Query elasticsearch logs for endpoint security event changes * Increase search result size for ES * Fix issue with fleet server restarting continuously * unpin kibana pr now that most major breakage is resolved * force unenroll * for new fleet bootstrap on re-enrollment * Fix unenrollment * Add timeout safeguard to elastic-agent execution In some cases such as attempting to re-enroll with a revoked token, the elastic-agent will retry indefinitely. This fix adds a safeguard utilizing 'timeout' command prepended to the elastic-agent command so that it will timeout after TimeoutFactor Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> Signed-off-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>

adam-stokes requested review from jalvz and a team April 6, 2021 15:16

mdelapenya approved these changes Apr 6, 2021

View reviewed changes

adam-stokes added v7.11.0 v7.12.0 v7.13.0 labels Apr 6, 2021

adam-stokes merged commit d9e3b51 into master Apr 6, 2021

adam-stokes deleted the gh-fix-984 branch April 6, 2021 15:55

mergify bot mentioned this pull request Apr 6, 2021

Uniquify the stand-alone step for checking agent status (bp #993) #995

Merged

mergify bot mentioned this pull request Apr 6, 2021

Uniquify the stand-alone step for checking agent status (bp #993) #996

Merged

mergify bot mentioned this pull request Apr 6, 2021

Uniquify the stand-alone step for checking agent status (bp #993) #997

Merged

mdelapenya mentioned this pull request Apr 7, 2021

Fix fleet server test #994

Closed

8 tasks

mdelapenya added a commit to mdelapenya/e2e-testing that referenced this pull request Apr 8, 2021

Revert "Uniquify the stand-alone step for checking agent status (elas…

a442410

…tic#993) (elastic#996)" This reverts commit 9818d67.

mdelapenya mentioned this pull request Apr 8, 2021

chore: revert standalone scenario for fleet server in 7.12 #1006

Merged

9 tasks

mdelapenya added a commit to mdelapenya/e2e-testing that referenced this pull request Apr 8, 2021

Revert "Uniquify the stand-alone step for checking agent status (elas…

4e5a412

…tic#993) (elastic#997)" This reverts commit 435c45b.

mdelapenya mentioned this pull request Apr 8, 2021

chore: revert standalone scenario for fleet server in 7.11 #1007

Merged

9 tasks

mdelapenya mentioned this pull request Oct 26, 2021

chore: bump elastic-package to v0.25.0 #1721

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uniquify the stand-alone step for checking agent status #993

Uniquify the stand-alone step for checking agent status #993

adam-stokes commented Apr 6, 2021

mdelapenya commented Apr 6, 2021

jalvz commented Apr 6, 2021

jalvz commented Apr 6, 2021

elasticmachine commented Apr 6, 2021

Build stats

Test stats 🧪

Trends 🧪

Test stats 🧪

mdelapenya commented Apr 7, 2021

mdelapenya commented Apr 7, 2021

Uniquify the stand-alone step for checking agent status #993

Uniquify the stand-alone step for checking agent status #993

Conversation

adam-stokes commented Apr 6, 2021

What does this PR do?

Why is it important?

Checklist

Author's Checklist

How to test this PR locally

Related issues

mdelapenya commented Apr 6, 2021

jalvz commented Apr 6, 2021

jalvz commented Apr 6, 2021

elasticmachine commented Apr 6, 2021

💚 Build Succeeded

Build stats

Test stats 🧪

Trends 🧪

💚 Flaky test report

Test stats 🧪

mdelapenya commented Apr 7, 2021

mdelapenya commented Apr 7, 2021