Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PoC: use pipeline generator #38162

Merged
merged 45 commits into from
Mar 20, 2024
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
55e3666
buildkite: python generator
v1v Mar 1, 2024
172c2bb
for testing purposes
v1v Mar 4, 2024
59de509
use intermediate script to run the pipeline generation with the requi…
v1v Mar 4, 2024
306b2ff
debug traces
v1v Mar 4, 2024
05fa8c4
fix typo
v1v Mar 4, 2024
340f6d1
support for PRs, labels and GH commands
v1v Mar 4, 2024
4b20d19
support category in the steps so github comments for mandatory stages…
v1v Mar 4, 2024
596890d
avoid empty groups
v1v Mar 4, 2024
df55062
refactored
v1v Mar 4, 2024
0f4df5d
changeset
pazone Mar 13, 2024
d006b37
Changesets
pazone Mar 13, 2024
e51daa0
fix global variable
pazone Mar 14, 2024
b92ffd7
Use beats images
pazone Mar 14, 2024
3449137
added tool versions and enabled pre-command for beats pipeline
pazone Mar 14, 2024
ab9eecb
Use beets VM images for filebeat
pazone Mar 14, 2024
b5cdc4c
AWS And Orka support
pazone Mar 15, 2024
2835ba1
Update auditbeat/buildkite.yml
v1v Mar 15, 2024
9c42231
Use native yaml marshalling
pazone Mar 18, 2024
6bfdd74
Merge branch 'feature/buildkite' of github.com:elastic/beats into fea…
pazone Mar 18, 2024
99dffd7
fix image
pazone Mar 18, 2024
9dc603a
comments pipeline generation: /test ...
pazone Mar 19, 2024
a74a5b6
Cleanup
alexsapran Mar 19, 2024
921ac8a
Fix typo
alexsapran Mar 19, 2024
b15fc29
Use mage commands inside the steps
alexsapran Mar 19, 2024
a528142
Create artifacts_path
alexsapran Mar 19, 2024
8b0b92a
Remove more wrapper scripts
alexsapran Mar 19, 2024
79d7a48
fixup! Remove more wrapper scripts
alexsapran Mar 19, 2024
049404a
Cleanup and removed doublicate code
alexsapran Mar 19, 2024
239f60a
Add missing parameter
alexsapran Mar 19, 2024
d49d8c0
fixup! Create artifacts_path
alexsapran Mar 19, 2024
c9e50c4
Remove print
alexsapran Mar 19, 2024
6c52899
Set magefile version as env var
alexsapran Mar 19, 2024
9ef55b2
Remove the usage of wrapper scripts
alexsapran Mar 19, 2024
bec043b
Remove more wrapper scripts
alexsapran Mar 19, 2024
2cb2431
Remove filebeat crosscompile wrapper
alexsapran Mar 19, 2024
5c20f45
Rework the logical if when loading the projects
alexsapran Mar 19, 2024
996d322
Remove comments
alexsapran Mar 19, 2024
1dcce41
Show pipeline in expanded mode
alexsapran Mar 19, 2024
0ce5a62
Fix windows mage build targets
alexsapran Mar 19, 2024
c839a67
Make github context start with capital letter
alexsapran Mar 19, 2024
fc58c36
Merge remote-tracking branch 'upstream/main' into feature/buildkite
alexsapran Mar 20, 2024
3a1cc1b
Add crosscompile GOX_FLAGS
alexsapran Mar 20, 2024
48a7851
Fix mage command for Filebeat windows steps
alexsapran Mar 20, 2024
a090777
Make steps run from inside the proper project folder
alexsapran Mar 20, 2024
ca2c8fc
Use more compatible cd and restore missed mage
alexsapran Mar 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .buildkite/auditbeat/scripts/unit-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,5 @@ source .buildkite/env-scripts/linux-env.sh
echo "--- Running Unit Tests"
sudo chmod -R go-w auditbeat/

cd auditbeat
umask 0022
mage build unitTest
mage -d auditbeat build unitTest
44 changes: 44 additions & 0 deletions .buildkite/buildkite.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
projects:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What beats are onboarded to this automated pipeline generator.

These are folders from the top of the project. They should also host a similar buildkite.yml file.

- "auditbeat"
- "deploy/kubernetes"
- "filebeat"
- "heartbeat"
- "libbeat"
- "metricbeat"
- "packetbeat"
- "winlogbeat"
- "x-pack/auditbeat"
- "x-pack/dockerlogbeat"
- "x-pack/filebeat"
- "x-pack/functionbeat"
- "x-pack/heartbeat"
- "x-pack/libbeat"
- "x-pack/metricbeat"
- "x-pack/osquerybeat"
- "x-pack/packetbeat"
- "x-pack/winlogbeat"

## Changeset macros that are defined here and used in each specific 3.0 pipeline.
changeset:
ci:
- "^Jenkinsfile"
- "^\\.ci/scripts/.*"
oss:
- "^go.mod"
- "^pytest.ini"
- "^dev-tools/.*"
- "^libbeat/.*"
- "^testing/.*"
xpack:
- "^go.mod"
- "^pytest.ini"
- "^dev-tools/.*"
- "^libbeat/.*"
- "^testing/.*"
- "^x-pack/libbeat/.*"

disabled:
when:
labels: ## Skip the GitHub Pull Request builds if any of the given GitHub labels match with the assigned labels in the PR.
- skip-ci
draft: true ## Skip the GitHub Pull Request builds with Draft PRs.
2 changes: 1 addition & 1 deletion .buildkite/hooks/pre-command
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ retry() {
return 0
}

if [[ "$BUILDKITE_PIPELINE_SLUG" == "filebeat" || "$BUILDKITE_PIPELINE_SLUG" == "auditbeat" || "$BUILDKITE_PIPELINE_SLUG" == "heartbeat" ]]; then
if [[ "$BUILDKITE_PIPELINE_SLUG" == "beats" || "$BUILDKITE_PIPELINE_SLUG" == "filebeat" || "$BUILDKITE_PIPELINE_SLUG" == "auditbeat" || "$BUILDKITE_PIPELINE_SLUG" == "heartbeat" ]]; then
source .buildkite/env-scripts/env.sh
source .buildkite/env-scripts/util.sh
source .buildkite/env-scripts/win-env.sh
Expand Down
Loading
Loading