Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Packetbeat] Create x-pack magefile #21979

Merged
merged 9 commits into from
Oct 27, 2020
Merged

[Packetbeat] Create x-pack magefile #21979

merged 9 commits into from
Oct 27, 2020

Conversation

andrewstucki
Copy link

@andrewstucki andrewstucki commented Oct 19, 2020 •
edited
Loading

What does this PR do?

This PR modifies the way that x-pack packetbeat builds in order to support the build process used in elastic-agent. I'm not entirely sure if there are any other places required for me to modify in order to have CI run mage package from the x-pack directory rather than OSS, but this should be all we need to get elastic-agent to know how to build packetbeat itself.

The elastic-agent change and packetbeat modifications for config files for elastic-agent configuration syntax are coming in subsequent PRs.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

@andrewstucki andrewstucki requested a review from a team as a code owner October 19, 2020 14:39
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Oct 19, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented Oct 19, 2020 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [v1v commented: Jenkins run the tests please]

  • Start Time: 2020-10-27T13:10:03.636+0000

  • Duration: 47 min 28 sec

Test stats 🧪

Test Results
Failed 0
Passed 3234
Skipped 30
Total 3264

@andrewstucki andrewstucki requested a review from a team October 20, 2020 02:40
andrewkroh
andrewkroh reviewed Oct 22, 2020
View reviewed changes
Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The top-level Makefile should be update to reflect that there is now an independent build for x-pack/packetbeat.

packetbeat/magefile.go Outdated Show resolved Hide resolved
packetbeat/magefile.go Outdated Show resolved Hide resolved
@andrewstucki andrewstucki mentioned this pull request Oct 23, 2020
Merged
6 tasks
@andrewstucki andrewstucki mentioned this pull request Oct 26, 2020
Merged
5 tasks
@andrewstucki andrewstucki requested a review from a team October 26, 2020 13:24
@andrewkroh
Copy link
Member

andrewkroh commented Oct 26, 2020 •
edited
Loading

To get this tested with CI before merging can you please

  • Uncomment packaging for x-pack/packetbeat then leave a GH comment with "/package"
  • Add a Jenkinsfile.yml into x-pack/packetbeat (@v1v can help out if there are any questions).

@andrewstucki andrewstucki requested a review from a team as a code owner October 26, 2020 14:03
@botelastic botelastic bot added the Team:Automation Label for the Observability productivity team label Oct 26, 2020
@v1v
Copy link
Member

v1v commented Oct 26, 2020

Add a Jenkinsfile.yml into x-pack/packetbeat (@v1v can help out if there are any questions).

So far the x-pack/packetbeat/Jenkinsfile.yml file was more a template, so it might be required to add support for:

  • linux
  • arm

It only runs:

  • "mage build unitTest" with the IntTesting for Windows-2019 by default
  • "mage build unitTest" for windows-2016 and 2012 only when merging to master/7.x

v1v
v1v previously requested changes Oct 26, 2020
View reviewed changes
Copy link
Member

@v1v v1v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my previous comment regarding supporting arm/linux builds/tests, since it's only building/unit testing for Windows-2019 on a PR basis and windows-2012/2016 for everything.

@andrewstucki andrewstucki requested a review from v1v October 26, 2020 14:27
@andrewstucki andrewstucki dismissed v1v’s stale review October 26, 2020 14:50

Updated with arm, mac, and linux targets

@elastic elastic deleted a comment from andrewkroh Oct 26, 2020
andrewkroh
andrewkroh reviewed Oct 26, 2020
View reviewed changes
x-pack/packetbeat/magefile.go Show resolved Hide resolved
x-pack/packetbeat/magefile.go Outdated Show resolved Hide resolved
x-pack/packetbeat/magefile.go Outdated Show resolved Hide resolved
@andrewstucki
Copy link
Author

/package

@v1v v1v added arm Enable builds in the CI for ARM testing macOS Enable builds in the CI for darwin testing windows-2012 Enable builds in the CI for windows-2012 windows-2016 Enable builds in the CI for windows-2016 labels Oct 27, 2020
@v1v
Copy link
Member

v1v commented Oct 27, 2020

/test

v1v
v1v reviewed Oct 27, 2020
View reviewed changes
Copy link
Member

@v1v v1v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default stages are:

  • build
  • windows

I just added a few labels to test the arm, windows-2016, macos and windows-2012 builds within this PR.

Let's wait for the outcome

@v1v
Copy link
Member

v1v commented Oct 27, 2020

Jenkins run the tests please

v1v
v1v approved these changes Oct 27, 2020
View reviewed changes
Copy link
Member

@v1v v1v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the CI point of view everything looks fine. 👍

@andrewstucki andrewstucki merged commit 3e73c99 into elastic:master Oct 27, 2020
@v1v
Copy link
Member

v1v commented Oct 29, 2020

I've just found the main Jenkinsfile.yml didn't have uncommented the entry for the x-pack/packetbeat, I'll raise a PR to enable this

@v1v v1v mentioned this pull request Oct 29, 2020
Merged
v1v added a commit to v1v/beats that referenced this pull request Oct 29, 2020
@v1v
Merge remote-tracking branch 'upstream/master' into feature/windows-7-64
3f9b862 
* upstream/master: (93 commits)
  Update commands used in the quick start (elastic#22248)
  Add interval documentation to `monitor` metricset (elastic#22152)
  [CI] enable x-pack/packetbeat in the CI (elastic#22252)
  Fix awscloudwatch input documentation (elastic#22247)
  Add support for different Azure Cloud environments in the metricbeat azure module (elastic#21044)
  [CI] support windows-2008-r2 (elastic#19791)
  protect against accessing undefined variables in sysmon module (elastic#22236)
  [CI] archive only if failed steps (elastic#22220)
  Add pe fields to Sysmon module (elastic#22217)
  [CI][flaky] Support 7.x branches and PRs (elastic#22197)
  Perfmon - Fix regular expressions to comply to multiple parentheses in instance name and object (elastic#22146)
  ci: improve linting speed (elastic#22103)
  Move cloudfoundry tags with metadata to common metadata fields (elastic#22150)
  [Docs] Update custom beat docs (elastic#22194)
  [Ingest Manager] Agent fix snapshot download for upgrade (elastic#22175)
  Update shared-autodiscover.asciidoc (elastic#21827)
  [DOCS] Warn about compression and Azure Event Hub for Kafka (elastic#21578)
  [CI][flaky] reporting for PRs in GitHub (elastic#21853)
  [Packetbeat] Create x-pack magefile (elastic#21979)
  [Elastic Agent] Fix deb/rpm installation (elastic#22153)
  ...
v1v added a commit to v1v/beats that referenced this pull request Oct 29, 2020
@v1v
Merge remote-tracking branch 'upstream/master' into feature/windows-7-32
a776d4d 
* upstream/master: (93 commits)
  Update commands used in the quick start (elastic#22248)
  Add interval documentation to `monitor` metricset (elastic#22152)
  [CI] enable x-pack/packetbeat in the CI (elastic#22252)
  Fix awscloudwatch input documentation (elastic#22247)
  Add support for different Azure Cloud environments in the metricbeat azure module (elastic#21044)
  [CI] support windows-2008-r2 (elastic#19791)
  protect against accessing undefined variables in sysmon module (elastic#22236)
  [CI] archive only if failed steps (elastic#22220)
  Add pe fields to Sysmon module (elastic#22217)
  [CI][flaky] Support 7.x branches and PRs (elastic#22197)
  Perfmon - Fix regular expressions to comply to multiple parentheses in instance name and object (elastic#22146)
  ci: improve linting speed (elastic#22103)
  Move cloudfoundry tags with metadata to common metadata fields (elastic#22150)
  [Docs] Update custom beat docs (elastic#22194)
  [Ingest Manager] Agent fix snapshot download for upgrade (elastic#22175)
  Update shared-autodiscover.asciidoc (elastic#21827)
  [DOCS] Warn about compression and Azure Event Hub for Kafka (elastic#21578)
  [CI][flaky] reporting for PRs in GitHub (elastic#21853)
  [Packetbeat] Create x-pack magefile (elastic#21979)
  [Elastic Agent] Fix deb/rpm installation (elastic#22153)
  ...
@andrewstucki andrewstucki mentioned this pull request Nov 11, 2020
Merged
6 tasks
andrewstucki pushed a commit to andrewstucki/beats that referenced this pull request Nov 11, 2020
[Packetbeat] Create x-pack magefile (elastic#21979)
32bc901 
* Create packetbeat x-pack magefile

* add changelog entry

* Fix up packetbeat magefile

* Add makefile for x-pack directory

* Uncomment packaging

* Add desired platforms to Jenkinsfile

* Add back in fields yml and kibana dashboards on package

* Add generated configs in x-pack/packetbeat

* Suggested changes

(cherry picked from commit 3e73c99)
@andrewstucki andrewstucki mentioned this pull request Nov 11, 2020
Merged
6 tasks
andrewstucki pushed a commit that referenced this pull request Nov 11, 2020
Cherry-pick #21979 to 7.x: [Packetbeat] Create x-pack magefile (#22545)
bcbef06 
* [Packetbeat] Create x-pack magefile (#21979)

* Create packetbeat x-pack magefile

* add changelog entry

* Fix up packetbeat magefile

* Add makefile for x-pack directory

* Uncomment packaging

* Add desired platforms to Jenkinsfile

* Add back in fields yml and kibana dashboards on package

* Add generated configs in x-pack/packetbeat

* Suggested changes

(cherry picked from commit 3e73c99)

* Fix up changelog
@andrewstucki andrewstucki mentioned this pull request Nov 16, 2020
Merged
5 tasks
@jsoriano jsoriano mentioned this pull request Jan 15, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrewkroh andrewkroh andrewkroh left review comments

@v1v v1v v1v approved these changes

Assignees
No one assigned
Labels
arm Enable builds in the CI for ARM testing macOS Enable builds in the CI for darwin testing Packaging Packetbeat Team:Automation Label for the Observability productivity team v7.11.0 windows-2012 Enable builds in the CI for windows-2012 windows-2016 Enable builds in the CI for windows-2016
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@andrewstucki @elasticmachine @andrewkroh @v1v