Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ingest Management] Fix reloading of log level for services #24055

Merged
merged 5 commits into from
Feb 16, 2021

Conversation

michalpristas
Copy link
Contributor

@michalpristas michalpristas commented Feb 16, 2021

What does this PR do?

This PR fixes 2 things.

First one is an issue introduced in #23886, this caused that VarSkipKey removed key from Action which was then stored using Ack and Store. This action was input-less so after restart agent was not running anything and it thought it;s ok because preserved state id is same as the one from fleet.
This one I discovered without filing an issue

Second one is bug described in #23720. The issue here is that agent creates a server with own counter, set to 1. But service already has a config with state counter. So if we change log level and restart an agent counter is 1 which == to the counter in a service and agent thinks everything is up to date.
Fix is that we take counter from a service and if it's > 0, which means services already retrieved some configuration we force reload of config by incrementing our local counter to what_service_has + 1

Why is it important?

Fixes bugs

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

@michalpristas michalpristas added bug Team:Elastic-Agent Label for the Agent team labels Feb 16, 2021
@michalpristas michalpristas self-assigned this Feb 16, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/agent (Team:Agent)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label Team:Ingest Management labels Feb 16, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ingest-management (Team:Ingest Management)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Feb 16, 2021
@michalpristas michalpristas changed the title Fix/endpoint logging [Ingest Management] Fix reloading of log level for services Feb 16, 2021
Copy link
Contributor

@blakerouse blakerouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Looks good.

// we're modifying object
for k, v := range skippedKeys {
data[k] = v
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, because the code removed it from above they needed to be added back in.

I think maybe making the comment a little clearer and with some reference to the delete(...) about would be better.

// force config reload
if observedConfigStateIdx > 0 {
appState.expectedConfigIdx = observedConfigStateIdx + 1
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it is a service, this makes perfect sense. Good catch!

@elasticmachine
Copy link
Collaborator

elasticmachine commented Feb 16, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #24055 updated

  • Start Time: 2021-02-16T13:38:27.929+0000

  • Duration: 53 min 4 sec

  • Commit: 94d2cc6

Test stats 🧪

Test Results
Failed 0
Passed 6542
Skipped 16
Total 6558

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 6542
Skipped 16
Total 6558

@michalpristas michalpristas merged commit 55b1e2a into elastic:master Feb 16, 2021
michalpristas added a commit to michalpristas/beats that referenced this pull request Feb 16, 2021
…24055)

[Ingest Management] Fix reloading of log level for services (elastic#24055)
michalpristas added a commit to michalpristas/beats that referenced this pull request Feb 16, 2021
…24055)

[Ingest Management] Fix reloading of log level for services (elastic#24055)
michalpristas added a commit that referenced this pull request Feb 16, 2021
…24062)

[Ingest Management] Fix reloading of log level for services (#24055)
michalpristas added a commit that referenced this pull request Feb 16, 2021
…24061)

[Ingest Management] Fix reloading of log level for services (#24055)
v1v added a commit to v1v/beats that referenced this pull request Feb 17, 2021
…-arm

* upstream/master:
  [CI] install docker-compose with retry (elastic#24069)
  Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes elastic#24051 (elastic#24052)
  updating manifest files for filebeat threatintel module (elastic#24074)
  Add Zeek Signatures (elastic#23772)
  Update Beats to ECS 1.8.0 (elastic#23465)
  Support running Docker logging plugin on ARM64 (elastic#24034)
  Fix ec2 metricset fields.yml and add integration test (elastic#23726)
  Only build targz and zip versions of Beats if PACKAGES is set in agent (elastic#24060)
  [Filebeat] Add field definitions for known Netflow/IPFIX vendor fields (elastic#23773)
  [Elastic Agent] Enroll with Fleet Server (elastic#23865)
  [Filebeat] Convert logstash logEvent.action objects to strings (elastic#23944)
  [Ingest Management] Fix reloading of log level for services (elastic#24055)
  Add Agent standalone k8s manifest (elastic#23679)
v1v added a commit to v1v/beats that referenced this pull request Feb 17, 2021
…dows-7

* upstream/master: (332 commits)
  Use ECS v1.8.0 (elastic#24086)
  Add support for postgresql csv logs (elastic#23334)
  [Heartbeat] Refactor config system (elastic#23467)
  [CI] install docker-compose with retry (elastic#24069)
  Add nodes to filebeat-kubernetes.yaml ClusterRole - fixes elastic#24051 (elastic#24052)
  updating manifest files for filebeat threatintel module (elastic#24074)
  Add Zeek Signatures (elastic#23772)
  Update Beats to ECS 1.8.0 (elastic#23465)
  Support running Docker logging plugin on ARM64 (elastic#24034)
  Fix ec2 metricset fields.yml and add integration test (elastic#23726)
  Only build targz and zip versions of Beats if PACKAGES is set in agent (elastic#24060)
  [Filebeat] Add field definitions for known Netflow/IPFIX vendor fields (elastic#23773)
  [Elastic Agent] Enroll with Fleet Server (elastic#23865)
  [Filebeat] Convert logstash logEvent.action objects to strings (elastic#23944)
  [Ingest Management] Fix reloading of log level for services (elastic#24055)
  Add Agent standalone k8s manifest (elastic#23679)
  [Metricbeat][Kubernetes] Extend state_node with more conditions (elastic#23905)
  [CI] googleStorageUploadExt step (elastic#24048)
  Check fields are documented for aws metricsets (elastic#23887)
  Update go-concert to 0.1.0 (elastic#23770)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Team:Elastic-Agent Label for the Agent team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants