Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Ingest Manager] Make agent retry values for bootstraping configurable #25163

Merged
merged 6 commits into from
Apr 20, 2021
Merged

[Ingest Manager] Make agent retry values for bootstraping configurable #25163

merged 6 commits into from
Apr 20, 2021

Conversation

michalpristas
Copy link
Contributor

@michalpristas michalpristas commented Apr 20, 2021 •
edited
Loading

What does this PR do?

This PR adds an option to configure values used in container command for GET/POST request to kibana.

Why is it important?

#25063

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

cc @simitt

@michalpristas michalpristas added enhancement needs_backport PR is waiting to be backported to other branches. v7.13.0 Team:Elastic-Agent Label for the Agent team labels Apr 20, 2021
@michalpristas michalpristas self-assigned this Apr 20, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/agent (Team:Agent)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Apr 20, 2021
@elasticmachine
Copy link
Collaborator

elasticmachine commented Apr 20, 2021 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #25163 updated

  • Start Time: 2021-04-20T11:32:29.677+0000

  • Duration: 79 min 46 sec

  • Commit: f580370

Test stats 🧪

Test Results
Failed 0
Passed 6788
Skipped 16
Total 6804

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 6788
Skipped 16
Total 6804

ruflin
ruflin reviewed Apr 20, 2021
View reviewed changes
Copy link
Member

@ruflin ruflin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor change request.

At one point we should discuss if these things should also be exposed through configs.

x-pack/elastic-agent/pkg/agent/cmd/container.go Outdated
defaultStateDirectory = "/usr/share/elastic-agent/state" // directory that will hold the state data
requestRetrySleepEnv = "KIBANA_REQUEST_RETRY_SLEEP"
maxRequestRetriesEnv = "KIBANA_REQUEST_RETRY_COUNT"
defaultRequestRetrySleep = "1" // sleep 1 sec between retries for HTTP requests
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make this 1s instead and have time parsing? This makes it possible to pass in any time values if needed and makes it more explicity.

x-pack/elastic-agent/pkg/agent/cmd/container.go Outdated
@@ -70,6 +75,8 @@ The following actions are possible and grouped based on the actions.
KIBANA_FLEET_USERNAME - kibana username to enable Fleet [$KIBANA_USERNAME]
KIBANA_FLEET_PASSWORD - kibana password to enable Fleet [$KIBANA_PASSWORD]
KIBANA_FLEET_CA - path to certificate authority to use with communicate with Kibana [$KIBANA_CA]
KIBANA_REQUEST_RETRY_SLEEP - specifies number of seconds of sleep taken when agent performs a request to kibana [default 1]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above

Suggested change
KIBANA_REQUEST_RETRY_SLEEP - specifies number of seconds of sleep taken when agent performs a request to kibana [default 1]
KIBANA_REQUEST_RETRY_SLEEP - specifies number of seconds of sleep taken when agent performs a request to kibana [default 1s]

simitt
simitt approved these changes Apr 20, 2021
View reviewed changes
Copy link
Contributor

@simitt simitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Havent't tested yet, but LGTM

@urso
Copy link

urso commented Apr 20, 2021

In general LGTM.

Should we consider exponential backoff (see: libbeat/common/backoff)?

@simitt
Copy link
Contributor

simitt commented Apr 20, 2021

Should we consider exponential backoff (see: libbeat/common/backoff)?

Can this be done in a follow up PR?

@michalpristas
Copy link
Contributor Author

i think blake did not go with exponential because we're just waiting for kibana to be online. we want result as soon as possible and we're not spamming kibana with request as it does not perform any work in this case.

@michalpristas michalpristas merged commit b9e4022 into elastic:master Apr 20, 2021
michalpristas added a commit to michalpristas/beats that referenced this pull request Apr 20, 2021
@michalpristas
[Ingest Manager] Make agent retry values for bootstraping configurable (
bf1cbb6 
elastic#25163)

[Ingest Manager] Make agent retry values for bootstraping configurable (elastic#25163)
@michalpristas michalpristas mentioned this pull request Apr 20, 2021
Merged
6 tasks
michalpristas added a commit that referenced this pull request Apr 20, 2021
@michalpristas
Cherry-pick #25163 to 7.x: Make agent retry values for bootstraping c…
34c48a2 
…onfigurable (#25174)

Cherry-pick #25163 to 7.x: Make agent retry values for bootstraping configurable  (#25174)
v1v added a commit to v1v/beats that referenced this pull request Apr 21, 2021
@v1v
Merge remote-tracking branch 'upstream/master' into feature/customise…
d6c9e5c 
…-github-pr-comment-template

* upstream/master:
  Check native environment before starting (elastic#25186)
  Change event.code and winlog.event_id type (elastic#25176)
  [Ingest Manager] Proxy processes/elastic-agent to stats (elastic#25193)
  Update mergify backporting to 7.x and 7.13 (elastic#25196)
  [Heartbeat]: ensure synthetics version co* [Heartbeat]: ensure synthetics version compatability for suites  * address review and fix notice  * fix lowercase struct  * fix version conflict and rebase  * update go.* stuff to master  * fix notice.txt  * move validate inside sourcempatability for suites (elastic#24777)
  [Filebeat] Ensure Kibana audit `event.category` and `event.type` are still processed as strings. (elastic#25101)
  Update replace.asciidoc (elastic#25055)
  Fix nil panic when overwriting metadata (elastic#24741)
  [Filebeat] Add Malware Bazaar to Threat Intel Module (elastic#24570)
  Fix k8s svc selectors mapping (elastic#25169)
  [Ingest Manager] Make agent retry values for bootstraping configurable (elastic#25163)
  [Metricbeat] Remove elasticsearc.index.created from the SM code (elastic#25113)
v1v added a commit to v1v/beats that referenced this pull request Apr 22, 2021
@v1v
Merge remote-tracking branch 'upstream/master' into feature/pin-testi…
82ffbd4 
…ng-versions-stack

* upstream/master: (28 commits)
  Add support for parsers in filestream input (elastic#24763)
  Skip flaky test TestFilestreamTruncate (elastic#25218)
  backport: Add 7.13 branch (elastic#25189)
  Update decode_json_fields.asciidoc (elastic#25056)
  [Elastic Agent] Fix status and inspect command to work inside running container (elastic#25204)
  Check native environment before starting (elastic#25186)
  Change event.code and winlog.event_id type (elastic#25176)
  [Ingest Manager] Proxy processes/elastic-agent to stats (elastic#25193)
  Update mergify backporting to 7.x and 7.13 (elastic#25196)
  [Heartbeat]: ensure synthetics version co* [Heartbeat]: ensure synthetics version compatability for suites  * address review and fix notice  * fix lowercase struct  * fix version conflict and rebase  * update go.* stuff to master  * fix notice.txt  * move validate inside sourcempatability for suites (elastic#24777)
  [Filebeat] Ensure Kibana audit `event.category` and `event.type` are still processed as strings. (elastic#25101)
  Update replace.asciidoc (elastic#25055)
  Fix nil panic when overwriting metadata (elastic#24741)
  [Filebeat] Add Malware Bazaar to Threat Intel Module (elastic#24570)
  Fix k8s svc selectors mapping (elastic#25169)
  [Ingest Manager] Make agent retry values for bootstraping configurable (elastic#25163)
  [Metricbeat] Remove elasticsearc.index.created from the SM code (elastic#25113)
  [Ingest Manager] Keep http and logging config during enroll (elastic#25132)
  Refactor kubernetes autodiscover to avoid skipping short-living pods (elastic#24742)
  [libbeat] New decode xml wineventlog processor (elastic#25115)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ruflin ruflin ruflin left review comments

@simitt simitt simitt approved these changes

Assignees

@michalpristas michalpristas

Labels
enhancement needs_backport PR is waiting to be backported to other branches. Team:Elastic-Agent Label for the Agent team v7.13.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@michalpristas @elasticmachine @urso @simitt @ruflin