Skip to content
This repository has been archived by the owner on Sep 17, 2024. It is now read-only.

chore: use preconfigured policy for fleet-server's default policy #2111

Merged
merged 3 commits into from
Feb 8, 2022
Merged

chore: use preconfigured policy for fleet-server's default policy #2111

merged 3 commits into from
Feb 8, 2022

Conversation

mdelapenya
Copy link
Contributor

What does this PR do?

This PR adds preconfigured policies for Fleet Server in all kibana.config.yml files used in the tests (the default and the one for the pre-configured policies scenario)

Because we are setting the fleet-server policy from the config, we are now hardcoding the policy name and ID in the bootstrap process of fleet-server (that happens right after the stack starts), not getting it from Kibana API for a default policy.

We are also adding a deprecation comment in the GetDefaultPolicy method, as it's not used anymore. We could simply remove it, but I prefer doing it in a separate PR.

Why is it important?

Fleet team is removing default policies

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have run the Unit tests (make unit-test), and they are passing locally
  • I have run the End-2-End tests for the suite I'm working on, and they are passing locally
  • I have noticed new Go dependencies (run make notice in the proper directory)

Author's Checklist

  • [ ]

How to test this PR locally

Because the current stack version is not updated, we need to set a more recent one (at least 4 days ago, where a bug was fixed on Kibana side elastic/kibana#124363):

$ TAGS="blah" TIMEOUT_FACTOR=3 LOG_LEVEL=TRACE DEVELOPER_MODE=true ELASTIC_APM_ACTIVE=false STACK_VERSION=8.1.0-98e72485-SNAPSHOT BEAT_VERSION=8.1.0-98e72485-SNAPSHOT PROVIDER=docker make -C e2e/_suites/fleet functional-test^C

The blah tag is a non-existing one to avoid launching any scenario, but instead simply bootstrapping the stack, which happens in the before suite hook. The result would be the 3 containers for the stack (es, kbn and fleet-server) without log errors, and after checking the Kibana UI, verify that fleet-server is enrolled.

Related issues

@mdelapenya mdelapenya added Team:Automation Label for the Observability productivity team Team:Fleet Label for the Fleet team backport-v8.0.0 Automated backport with mergify labels Feb 8, 2022
@mdelapenya mdelapenya self-assigned this Feb 8, 2022
@mdelapenya mdelapenya requested review from juliaElastic and a team February 8, 2022 10:42
@mdelapenya
Copy link
Contributor Author

We'd probably need to merge this one and merge the latest automated bump

juliaElastic
juliaElastic approved these changes Feb 8, 2022
View reviewed changes
Copy link
Contributor

@juliaElastic juliaElastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@elasticmachine
Copy link
Contributor

elasticmachine commented Feb 8, 2022 •
edited
Loading

❕ Build Aborted

There is a new build on-going so the previous on-going builds have been aborted.

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Start Time: 2022-02-08T14:38:59.646+0000

  • Duration: 22 min 34 sec

Test stats 🧪

Test Results
Failed 0
Passed 152
Skipped 0
Total 152

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@mdelapenya
Copy link
Contributor Author

Let's merge this one, backport to 8.0 and, if needed, backport to 7.17 and 7.16

@mdelapenya mdelapenya merged commit 5ed1aff into elastic:main Feb 8, 2022
mergify bot pushed a commit that referenced this pull request Feb 8, 2022
@mdelapenya
chore: use preconfigured policy for fleet-server's default policy (#2111
743ca3f 
)

(cherry picked from commit 5ed1aff)

# Conflicts:
#	e2e/_suites/fleet/fleet.go
@mergify mergify bot mentioned this pull request Feb 8, 2022
Merged
@elasticmachine
Copy link
Contributor

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-02-08T14:53:03.265+0000

  • Duration: 67 min 23 sec

Test stats 🧪

Test Results
Failed 72
Passed 177
Skipped 0
Total 249

Test errors 72

Expand to view the tests failures

> Show only the first 10 test failures

Initializing / End-To-End Tests / fleet_debian_amd64_linux_integration / Adding the Linux Integration to an Agent ... – Linux Integration
    Expand to view the error details

     Step the agent is listed in Fleet as "online"

  • no stacktrace
Initializing / End-To-End Tests / fleet_sles15_linux_integration / Adding the Linux Integration to an Agent ... – Linux Integration
    Expand to view the error details

     Step the agent is listed in Fleet as "online"

  • no stacktrace
Initializing / End-To-End Tests / fleet_debian_arm64_linux_integration / Adding the Linux Integration to an Agent ... – Linux Integration
    Expand to view the error details

     Step the agent is listed in Fleet as "online"

  • no stacktrace
Initializing / End-To-End Tests / fleet_debian_amd64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of filebeat #1 – Running on top of Beats
    Expand to view the error details

     Step the "elastic-agent" process is in the "started" state on the host

  • no stacktrace
Initializing / End-To-End Tests / fleet_debian_amd64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of metricbeat #1 – Running on top of Beats
    Expand to view the error details

     Step the "elastic-agent" process is in the "started" state on the host

  • no stacktrace
Initializing / End-To-End Tests / fleet_debian_amd64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of filebeat #2 – Running on top of Beats
    Expand to view the error details

     Step the "elastic-agent" process is in the "started" state on the host

  • no stacktrace
Initializing / End-To-End Tests / fleet_debian_amd64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of metricbeat #2 – Running on top of Beats
    Expand to view the error details

     Step the "elastic-agent" process is in the "started" state on the host

  • no stacktrace
Initializing / End-To-End Tests / fleet_debian_amd64_fleet_mode_agent / Deploying the agent – Fleet Mode Agent
    Expand to view the error details

     Step the "elastic-agent" process is in the "started" state on the host

  • no stacktrace
Initializing / End-To-End Tests / fleet_debian_amd64_fleet_mode_agent / Restarting the installed agent – Fleet Mode Agent
    Expand to view the error details

     Step the "elastic-agent" process is "restarted" on the host

  • no stacktrace
Initializing / End-To-End Tests / fleet_debian_amd64_fleet_mode_agent / Un-enrolling the agent deactivates the agent – Fleet Mode Agent
    Expand to view the error details

     Step the agent is un-enrolled

  • no stacktrace

Steps errors 30

Expand to view the steps failures

Show only the first 10 steps failures

Shell Script
  • Took 6 min 23 sec . View more details here
  • Description: ssh -tt -o TCPKeepAlive=yes -o ServerAliveInterval=60 -o ServerAliveCountMax=10 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /var/lib/jenkins/workspace/PR-2111-4-3697fea8-c974-4ac2-9dce-91bb51816630/e2essh admin@18.188.247.213 -- 'sudo bash /home/admin/e2e-testing/.ci/scripts/functional-test.sh "backend_processes && ~@nightly && ~@skip:amd64" '
Archive the artifacts
  • Took 0 min 0 sec . View more details here
  • Description: [2022-02-08T15:34:46.320Z] Archiving artifacts script returned exit code 2
Shell Script
  • Took 6 min 38 sec . View more details here
  • Description: ssh -tt -o TCPKeepAlive=yes -o ServerAliveInterval=60 -o ServerAliveCountMax=10 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /var/lib/jenkins/workspace/PR-2111-4-a91fd16f-c676-4254-9ac1-419fc1624e9b/e2essh ec2-user@3.144.149.116 -- 'sudo bash /home/ec2-user/e2e-testing/.ci/scripts/functional-test.sh "backend_processes && ~@nightly && ~@skip:amd64" '
Archive the artifacts
  • Took 0 min 0 sec . View more details here
  • Description: [2022-02-08T15:35:48.597Z] Archiving artifacts script returned exit code 2
Shell Script
  • Took 5 min 7 sec . View more details here
  • Description: ssh -tt -o TCPKeepAlive=yes -o ServerAliveInterval=60 -o ServerAliveCountMax=10 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /var/lib/jenkins/workspace/PR-2111-4-5e071db5-d506-46bd-b57f-a7990e70ffef/e2essh admin@18.117.81.77 -- 'sudo bash /home/admin/e2e-testing/.ci/scripts/functional-test.sh "running_on_beats && ~@nightly && ~@skip:arm64" '
Archive the artifacts
  • Took 0 min 0 sec . View more details here
  • Description: [2022-02-08T15:33:47.829Z] Archiving artifacts script returned exit code 2
Shell Script
  • Took 4 min 2 sec . View more details here
  • Description: ssh -tt -o TCPKeepAlive=yes -o ServerAliveInterval=60 -o ServerAliveCountMax=10 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /var/lib/jenkins/workspace/PR-2111-4-00c19e24-f6c9-4b14-8ce3-c53a114c56bf/e2essh admin@18.191.134.228 -- 'sudo bash /home/admin/e2e-testing/.ci/scripts/functional-test.sh "running_on_beats && ~@nightly && ~@skip:amd64" '
Archive the artifacts
  • Took 0 min 0 sec . View more details here
  • Description: [2022-02-08T15:32:39.245Z] Archiving artifacts script returned exit code 2
Shell Script
  • Took 4 min 40 sec . View more details here
  • Description: ssh -tt -o TCPKeepAlive=yes -o ServerAliveInterval=60 -o ServerAliveCountMax=10 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i /var/lib/jenkins/workspace/PR-2111-4-5f3fe597-5fe9-467f-a311-1334e0f7629c/e2essh ec2-user@18.218.179.116 -- 'sudo bash /home/ec2-user/e2e-testing/.ci/scripts/functional-test.sh "running_on_beats && ~@nightly && ~@skip:amd64" '
Archive the artifacts
  • Took 0 min 0 sec . View more details here
  • Description: [2022-02-08T15:34:07.618Z] Archiving artifacts script returned exit code 2

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Genuine test errors 72

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_linux_integration / Adding the Linux Integration to an Agent ... – Linux Integration
  • Name: Initializing / End-To-End Tests / fleet_sles15_linux_integration / Adding the Linux Integration to an Agent ... – Linux Integration
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_linux_integration / Adding the Linux Integration to an Agent ... – Linux Integration
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of filebeat #1 – Running on top of Beats
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of metricbeat #1 – Running on top of Beats
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of filebeat #2 – Running on top of Beats
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of metricbeat #2 – Running on top of Beats
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_fleet_mode_agent / Deploying the agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_fleet_mode_agent / Restarting the installed agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_fleet_mode_agent / Un-enrolling the agent deactivates the agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_fleet_mode_agent / Re-enrolling the agent activates the agent in Fleet – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_fleet_mode_agent / Revoking the enrollment token for the agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_fleet_mode_agent / Un-installing the installed agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of filebeat #1 – Running on top of Beats
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of metricbeat #1 – Running on top of Beats
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of filebeat #2 – Running on top of Beats
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of metricbeat #2 – Running on top of Beats
  • Name: Initializing / End-To-End Tests / fleet_sles15_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of filebeat #1 – Running on top of Beats
  • Name: Initializing / End-To-End Tests / fleet_sles15_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of metricbeat #1 – Running on top of Beats
  • Name: Initializing / End-To-End Tests / fleet_sles15_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of filebeat #2 – Running on top of Beats
  • Name: Initializing / End-To-End Tests / fleet_sles15_running_on_beats / Deploying the Elastic-Agent with enroll and then run on top of metricbeat #2 – Running on top of Beats
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_fleet_mode_agent / Deploying the agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_fleet_mode_agent / Restarting the installed agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_fleet_mode_agent / Un-enrolling the agent deactivates the agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_fleet_mode_agent / Re-enrolling the agent activates the agent in Fleet – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_fleet_mode_agent / Revoking the enrollment token for the agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_fleet_mode_agent / Un-installing the installed agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_sles15_fleet_mode_agent / Deploying the agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_sles15_fleet_mode_agent / Restarting the installed agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_sles15_fleet_mode_agent / Un-enrolling the agent deactivates the agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_sles15_fleet_mode_agent / Re-enrolling the agent activates the agent in Fleet – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_sles15_fleet_mode_agent / Revoking the enrollment token for the agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_sles15_fleet_mode_agent / Un-installing the installed agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_backend_processes / Deploying the agent – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_backend_processes / Stopping the agent stops backend processes – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_backend_processes / Restarting the installed agent – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_backend_processes / Un-enrolling the agent stops backend processes – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_backend_processes / Re-enrolling the agent starts the elastic-agent process – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_backend_processes / Un-installing the installed agent – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_backend_processes / Un-enrolling Elastic Agent stops Elastic Endpoint – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_backend_processes / Removing Endpoint from Agent policy stops the connected Endpoint – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_backend_processes / Deploying the agent – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_backend_processes / Stopping the agent stops backend processes – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_backend_processes / Restarting the installed agent – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_backend_processes / Un-enrolling the agent stops backend processes – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_backend_processes / Re-enrolling the agent starts the elastic-agent process – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_backend_processes / Un-installing the installed agent – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_backend_processes / Un-enrolling Elastic Agent stops Elastic Endpoint – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_debian_arm64_backend_processes / Removing Endpoint from Agent policy stops the connected Endpoint – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_sles15_backend_processes / Deploying the agent – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_sles15_backend_processes / Stopping the agent stops backend processes – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_sles15_backend_processes / Restarting the installed agent – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_sles15_backend_processes / Un-enrolling the agent stops backend processes – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_sles15_backend_processes / Re-enrolling the agent starts the elastic-agent process – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_sles15_backend_processes / Un-installing the installed agent – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_sles15_backend_processes / Un-enrolling Elastic Agent stops Elastic Endpoint – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_sles15_backend_processes / Removing Endpoint from Agent policy stops the connected Endpoint – Backend Processes
  • Name: Initializing / End-To-End Tests / fleet_centos8_amd64_fleet_mode_agent / Deploying the agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_centos8_amd64_fleet_mode_agent / Restarting the installed agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_centos8_amd64_fleet_mode_agent / Un-enrolling the agent deactivates the agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_centos8_amd64_fleet_mode_agent / Re-enrolling the agent activates the agent in Fleet – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_centos8_amd64_fleet_mode_agent / Revoking the enrollment token for the agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_centos8_amd64_fleet_mode_agent / Un-installing the installed agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_centos8_arm64_fleet_mode_agent / Deploying the agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_centos8_arm64_fleet_mode_agent / Restarting the installed agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_centos8_arm64_fleet_mode_agent / Un-enrolling the agent deactivates the agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_centos8_arm64_fleet_mode_agent / Re-enrolling the agent activates the agent in Fleet – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_centos8_arm64_fleet_mode_agent / Revoking the enrollment token for the agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_centos8_arm64_fleet_mode_agent / Un-installing the installed agent – Fleet Mode Agent
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_apm_server / Deploying a default stand-alone agent with the Elastic APM integration – APM Integration
  • Name: Initializing / End-To-End Tests / fleet_debian_amd64_apm_server / Deploying a ubi8 stand-alone agent with the Elastic APM integration – APM Integration
  • Name: Initializing / End-To-End Tests / kubernetes-autodiscover_debian_amd64_elastic-agent / [empty] – TEST-x86_64-kubernetes-autodiscover-2edd4b82-2022-02-08-15:28:14.xml

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

mdelapenya added a commit that referenced this pull request Feb 8, 2022
@mergify @mdelapenya
[8.0](backport #2111) chore: use preconfigured policy for fleet-serve…
9b00e41 
…r's default policy (#2116)

* chore: use preconfigured policy for fleet-server's default policy (#2111)

(cherry picked from commit 5ed1aff)

# Conflicts:
#	e2e/_suites/fleet/fleet.go

* fix: resolve conflicts

Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
mdelapenya added a commit to mdelapenya/e2e-testing that referenced this pull request Feb 10, 2022
@mdelapenya
Revert "[8.0](backport elastic#2111) chore: use preconfigured policy …
7b876ba 
…for fleet-server's default policy (elastic#2116)"

This reverts commit 9b00e41.
mdelapenya added a commit that referenced this pull request Feb 10, 2022
@mdelapenya
Revert "[8.0](backport #2111) chore: use preconfigured policy for fle…
36680c4 
…et-server's default policy (#2116)" (#2133)

This reverts commit 9b00e41.
@mdelapenya mdelapenya deleted the 2039-remove-default-policies branch March 9, 2022 06:44
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@juliaElastic juliaElastic juliaElastic approved these changes

@adam-stokes adam-stokes adam-stokes approved these changes

Assignees

@mdelapenya mdelapenya

Labels
backport-v8.0.0 Automated backport with mergify Team:Automation Label for the Observability productivity team Team:Fleet Label for the Fleet team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Remove dependency on Fleet Default policy
4 participants
@mdelapenya @elasticmachine @adam-stokes @juliaElastic