Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge feature arch v2 #33630

Merged
merged 17 commits into from
Nov 9, 2022
Merged

Merge feature arch v2 #33630

merged 17 commits into from
Nov 9, 2022

Conversation

fearful-symmetry
Copy link
Contributor

What does this PR do?

This merges the feature-arch-v2 branch into main.

This should not be merged until elastic-agent V2 changes have merged.

fearful-symmetry and others added 17 commits September 6, 2022 09:44
…32673)

* basic framework

* continued tinkering

* move away from ast code, use a struct

* get metricbeat working, starting on filebeat

* add notice update

* add basic config register

* move over processors to individual beats

* remove comments

* start to integrate V2 client changes

* finishing touches

* lint

* cleanup merge

* remove V1 controller

* stil tinkering with linter

* still fixing linter

* plz linter

* fmt x-pack files

* notice update

* fix output test

* refactor stop functions, refactor tests, some misc cleanup

* fix client version string

* add devguide

* linter

* expand filebeat test

* cleanup test

* fix docs, add tests, debuggin

* add signal handler

* fix mutex issue in register

* Fix osquerybeat configuration for V2

* clean up component registration

* spelling

* remove workaround for filebeat types

* try to fix filebeat tests

* add nil checks, fix test, fix unit stop

* continue tinkering with nil type checks

* add test for missing config datastreams, clean up nil handling

* change nil protections, use getter methods

* fix config access in output code

Co-authored-by: Aleksandr Maus <aleksandr.maus@elastic.co>
* first attempt at auditbeat support

* add license header

* initial packetbeat support

* fix bad branch

* cleanup

* typo in comment

* clean up, move around files

* add new processors to streams
* first attempt at auditbeat support

* add license header

* cleanup

* move files around
* add v2 config

* fix name

* fix doc
* Fix errors for non-synth capable instances (#33310)

Fixes #32694 by making sure we use the lightweight wrapper code always when monitors cannot be initialized.

This also fixes an unrelated bug, where errors attached to non-summary events would not be indexed.

* [Automation] Update elastic stack version to 8.6.0-5a8d757d for testing (#33323)

Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co>

* add pid awareness to file locking (#33169)

* add pid awareness to file locking

* cleanup, logic for handling restarts with the same PID

* add zombie-state awareness

* fix file naming

* add retry for unlock

* was confused by unlock code, fix, cleanup

* update notice

* fix race with file creation, update deps

* clean up tests, spelling

* hack for cgo

* add lic headers

* notice

* try to fix windows issues

* fix typos

* small fixes

* use exclusive locks

* remove feature to start with a specially named pidfile

* clean up some error handling, fix test cleanup

* forgot changelog

* Fix sample config in log rotation docs (#33306)

* Add banner to deprecate functionbeat (#33297)

* fix unchecked stream_id

* packetbeat/protos/dns: clean up package (#33286)

* avoid magic numbers
* fix hashableDNSTuple size and offsets
* avoid use of String and Error methods in formatted print calls
* remove redundant conversions
* quieten linter
* use plugin-owned logp.Logger

* update elastic-agent-libs

* Revert "fix unchecked stream_id"

This reverts commit 26ef6da.

* [Automation] Update elastic stack version to 8.6.0-40086bc7 for testing (#33339)

Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co>

Co-authored-by: Andrew Cholakian <andrewvc@elastic.co>
Co-authored-by: apmmachine <58790750+apmmachine@users.noreply.github.com>
Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co>
Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
Co-authored-by: DeDe Morton <dede.morton@elastic.co>
Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>
@fearful-symmetry fearful-symmetry added the Team:Elastic-Agent Label for the Agent team label Nov 9, 2022
@fearful-symmetry fearful-symmetry self-assigned this Nov 9, 2022
@fearful-symmetry fearful-symmetry requested review from a team as code owners November 9, 2022 16:45
@fearful-symmetry fearful-symmetry requested review from belimawr and leehinman and removed request for a team November 9, 2022 16:45
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Nov 9, 2022
@mergify
Copy link
Contributor

mergify bot commented Nov 9, 2022

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @fearful-symmetry? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

Copy link
Member

@cmacknz cmacknz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚢

@cmacknz
Copy link
Member

cmacknz commented Nov 9, 2022

The agent PR is merged elastic/elastic-agent#1701

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-11-09T16:45:34.450+0000

  • Duration: 148 min 9 sec

Test stats 🧪

Test Results
Failed 0
Passed 23769
Skipped 1951
Total 25720

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@cmacknz cmacknz merged commit a6ce455 into main Nov 9, 2022
@cmacknz cmacknz deleted the feature-arch-v2 branch November 9, 2022 19:35
@amitkanfer
Copy link
Collaborator

💥

chrisberkhout pushed a commit that referenced this pull request Jun 1, 2023
* Update Metricbeat, Filebeat, libbeat with elastic-agent V2 support (#32673)

* basic framework

* continued tinkering

* move away from ast code, use a struct

* get metricbeat working, starting on filebeat

* add notice update

* add basic config register

* move over processors to individual beats

* remove comments

* start to integrate V2 client changes

* finishing touches

* lint

* cleanup merge

* remove V1 controller

* stil tinkering with linter

* still fixing linter

* plz linter

* fmt x-pack files

* notice update

* fix output test

* refactor stop functions, refactor tests, some misc cleanup

* fix client version string

* add devguide

* linter

* expand filebeat test

* cleanup test

* fix docs, add tests, debuggin

* add signal handler

* fix mutex issue in register

* Fix osquerybeat configuration for V2

* clean up component registration

* spelling

* remove workaround for filebeat types

* try to fix filebeat tests

* add nil checks, fix test, fix unit stop

* continue tinkering with nil type checks

* add test for missing config datastreams, clean up nil handling

* change nil protections, use getter methods

* fix config access in output code

Co-authored-by: Aleksandr Maus <aleksandr.maus@elastic.co>

* V2 packetbeat support (#33041)

* first attempt at auditbeat support

* add license header

* initial packetbeat support

* fix bad branch

* cleanup

* typo in comment

* clean up, move around files

* add new processors to streams

* First pass at auditbeat support (#33026)

* first attempt at auditbeat support

* add license header

* cleanup

* move files around

* Add heartbeat support for V2 (#33157)

* add v2 config

* fix name

* fix doc

* fix go.mod

* fix unchecked stream_id

* fix unchecked stream_id (#33335)

* Update elastic-agent-libs for output panic fix (#33336)

* Fix errors for non-synth capable instances (#33310)

Fixes #32694 by making sure we use the lightweight wrapper code always when monitors cannot be initialized.

This also fixes an unrelated bug, where errors attached to non-summary events would not be indexed.

* [Automation] Update elastic stack version to 8.6.0-5a8d757d for testing (#33323)

Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co>

* add pid awareness to file locking (#33169)

* add pid awareness to file locking

* cleanup, logic for handling restarts with the same PID

* add zombie-state awareness

* fix file naming

* add retry for unlock

* was confused by unlock code, fix, cleanup

* update notice

* fix race with file creation, update deps

* clean up tests, spelling

* hack for cgo

* add lic headers

* notice

* try to fix windows issues

* fix typos

* small fixes

* use exclusive locks

* remove feature to start with a specially named pidfile

* clean up some error handling, fix test cleanup

* forgot changelog

* Fix sample config in log rotation docs (#33306)

* Add banner to deprecate functionbeat (#33297)

* fix unchecked stream_id

* packetbeat/protos/dns: clean up package (#33286)

* avoid magic numbers
* fix hashableDNSTuple size and offsets
* avoid use of String and Error methods in formatted print calls
* remove redundant conversions
* quieten linter
* use plugin-owned logp.Logger

* update elastic-agent-libs

* Revert "fix unchecked stream_id"

This reverts commit 26ef6da.

* [Automation] Update elastic stack version to 8.6.0-40086bc7 for testing (#33339)

Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co>

Co-authored-by: Andrew Cholakian <andrewvc@elastic.co>
Co-authored-by: apmmachine <58790750+apmmachine@users.noreply.github.com>
Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co>
Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
Co-authored-by: DeDe Morton <dede.morton@elastic.co>
Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>

* update elastic-agent-client (#33552)

Co-authored-by: Aleksandr Maus <aleksandr.maus@elastic.co>
Co-authored-by: Andrew Cholakian <andrewvc@elastic.co>
Co-authored-by: apmmachine <58790750+apmmachine@users.noreply.github.com>
Co-authored-by: apmmachine <infra-root-apmmachine@elastic.co>
Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
Co-authored-by: DeDe Morton <dede.morton@elastic.co>
Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Elastic-Agent Label for the Agent team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants