Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ingest Manager] Syncing unpacked files #21706

Merged
merged 1 commit into from
Oct 13, 2020

Conversation

michalpristas
Copy link
Contributor

What does this PR do?

This PR adds a sync call after files are unpacked from archives to disk. This seems to help with issues we've seen in tests where agent reported text file busy when executing beat. On second try it was able to execute binary.

Why is it important?

Gets rid of errors when executing beats and makes spinning beats up faster avoiding unnecessary retries.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

@elasticmachine
Copy link
Collaborator

Pinging @elastic/ingest-management (Team:Ingest Management)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Oct 12, 2020
@michalpristas

This comment has been minimized.

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #21706 opened]

  • Start Time: 2020-10-12T08:55:56.778+0000

  • Duration: 37 min 18 sec

Test stats 🧪

Test Results
Failed 0
Passed 1386
Skipped 4
Total 1390

@michalpristas
Copy link
Contributor Author

/packaging

@michalpristas
Copy link
Contributor Author

/package

1 similar comment
@michalpristas
Copy link
Contributor Author

/package

Copy link
Contributor

@blakerouse blakerouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting discovery. Change looks good, hope this fixes the e2e test!

@michalpristas michalpristas merged commit 5e786b3 into elastic:master Oct 13, 2020
v1v added a commit to v1v/beats that referenced this pull request Oct 13, 2020
* upstream/master: (127 commits)
  Update obs app links (elastic#21682)
  fix: update fleet test suite name (elastic#21738)
  Remove dot from file.extension value in Auditbeat FIM (elastic#21644)
  Fix leaks with metadata processors (elastic#16349)
  Add istiod metricset (elastic#21519)
  [Ingest Manager] Change Sync/Close call order (elastic#21735)
  [Ingest Manager] Syncing unpacked files (elastic#21706)
  Fix concurrent map read and write in socket dataset (elastic#21690)
  Fix conditional coding to remove seccomp info from Winlogbeat (elastic#21652)
  [Elastic Agent] Fix issue where inputs without processors defined would panic (elastic#21628)
  Add configuration of filestream input (elastic#21565)
  libbeat/logp: introduce Logger.WithOptions (elastic#21671)
  Make o365audit input cancellable (elastic#21647)
  fix: remove extra curly brace in script (elastic#21692)
  [Winlogbeat] Remove brittle configuration validation from wineventlog (elastic#21593)
  Fix function that parses from/to/contact headers (elastic#21672)
  [CI] Support Windows-2016 in pipeline 2.0 (elastic#21337)
  Skip publisher flaky tests (elastic#21657)
  backport: add 7.10 branch (elastic#21635)
  [CI: Packaging] fix: push ubi8 images too (elastic#21621)
  ...
michalpristas added a commit to michalpristas/beats that referenced this pull request Oct 14, 2020
michalpristas added a commit to michalpristas/beats that referenced this pull request Oct 14, 2020
michalpristas added a commit that referenced this pull request Oct 14, 2020
* [Ingest Manager] Syncing unpacked files (#21706)

[Ingest Manager] Syncing unpacked files (#21706)

* [Ingest Manager] Change Sync/Close call order (#21735)

[Ingest Manager] Change Sync/Close call order (#21735)

* [Ingest Manager] Agent atomic installer (#21745)

[Ingest Manager] Agent atomic installer (#21745)
michalpristas added a commit that referenced this pull request Oct 14, 2020
* [Ingest Manager] Syncing unpacked files (#21706)

[Ingest Manager] Syncing unpacked files (#21706)

* [Ingest Manager] Change Sync/Close call order (#21735)

[Ingest Manager] Change Sync/Close call order (#21735)

* [Ingest Manager] Agent atomic installer (#21745)

[Ingest Manager] Agent atomic installer (#21745)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants