Skip to content

Commit

Permalink
Prepare repo for use (#4)
Browse files Browse the repository at this point in the history
Prepare the repo for use by:

* Add .go files currently in github.com/elastic/quark
* Adding license and notice files
* Set up submodule with github.com/elastic/quark
* Setting up build scripts

Note on the build system:
To build a usable Go module, which doesn't require users to compile C code before themselves, compiled libquark artifacts are included in the published Go module. These files must be compiled by CI machines. In order to enforce this, any commits with modified .a files will have an automated commit overwrite these changes with files compiled by CI machines, so any merged PRs only contain .a files created by build machines.
  • Loading branch information
mjwolf authored Oct 4, 2024
1 parent d3eaf61 commit 366748a
Show file tree
Hide file tree
Showing 19 changed files with 1,634 additions and 8 deletions.
33 changes: 25 additions & 8 deletions .buildkite/default-pipeline.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,31 @@
---
# $yaml-language-server: $schema=https://raw.githubusercontent.com/buildkite/pipeline-schema/main/schema.json

steps:
- label: "Build"
command: "true"

- label: "Test"
command: "true"
key: build
command: "./.buildkite/scripts/build.sh"
artifact_paths:
- "libquark_big_amd64.a"
- "libquark_big_arm64.a"
agents:
image: family/core-ubuntu-2004
provider: gcp
machineType: n2-standard-2

- label: "Deploy"
command: "true"
- label: "Commit"
depends_on: build
key: commit
command: "./.buildkite/scripts/commit.sh"
agents:
image: "docker.elastic.co/ci-agent-images/sec-linux-platform/cloud-defend-builder@sha256:623a0d998dd6ebde09c7b7bbb829e3ba95b5475a0702ab35a5c6049775226f37"
cpu: "1"
memory: "1Gi"

- label: "Publish"
command: "true"
- label: "Check NOTICE.txt"
key: check-notice
command: "make check-notice"
agents:
image: "docker.elastic.co/ci-agent-images/sec-linux-platform/cloud-defend-builder@sha256:623a0d998dd6ebde09c7b7bbb829e3ba95b5475a0702ab35a5c6049775226f37"
cpu: "1"
memory: "1Gi"
13 changes: 13 additions & 0 deletions .buildkite/scripts/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/bin/bash

set -euo pipefail

# Build libquark_big.a for all supported architectures

git submodule update --init --recursive

for ARCH in amd64
do
ARCH=$ARCH make -C src centos7
mv src/libquark_big.a libquark_big_${ARCH}.a
done
45 changes: 45 additions & 0 deletions .buildkite/scripts/commit.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
#!/bin/bash

set -euo pipefail

# Commit modified .a files to the same branch that was built

BOT_NAME=${BOT_NAME:-"buildkite-bot"}
BOT_EMAIL=${BOT_EMAIL:-"buildkite-bot@noreply.elastic.co"}

function download {
buildkite-agent artifact download "$1" "$2"
}

for ARCH in amd64; do
download libquark_big_${ARCH}.a .
done

# If there are no changes, don't commit
if ! git diff --name-only HEAD | grep -q -E '\.a$'; then
echo "No changes detected"
exit 0
fi

# Don't commit if the last commit is from the bot, to avoid infinite loop of builds
if test "$(git log -1 --pretty=format:'%ae')" = "${BOT_EMAIL}"; then
echo "last commit from bot"
exit 0
fi

if test -z "${BUILDKITE}"; then
echo "This script doesn't appear to be running in buildkite; refusing to commit"
exit 1
fi

git config --global user.name "${BOT_NAME}"
git config --global user.email "${BOT_EMAIL}"

git config --global credential.https://git.luolix.top.username token
git config --global credential.https://git.luolix.top.helper '!echo \"password=\$(cat /run/secrets/VAULT_GITHUB_TOKEN)\";'

git add libquark_big_amd64.a

git commit -m "Auto-update .a files by Buildkite"

git push origin HEAD:"${BUILDKITE_BRANCH}"
Empty file added .gitignore
Empty file.
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "quark"]
path = src
url = https://github.com/elastic/quark.git
2 changes: 2 additions & 0 deletions LICENSE.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Source code in this repository is licensed under the Apache License Version 2.0,
an Apache compatible license.
18 changes: 18 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
.PHONY: notice
notice:
@echo "Generating NOTICE"
go mod tidy
go mod download
go list -m -json all | go run go.elastic.co/go-licence-detector \
-includeIndirect \
-rules tools/notice/rules.json \
-overrides tools/notice/overrides.json \
-noticeTemplate tools/notice/NOTICE.txt.tmpl \
-noticeOut NOTICE.txt \
-depsOut ""

.PHONY: check-notice
check-notice: notice
@if git diff --name-only | grep "NOTICE.txt"; then \
echo "NOTICE.txt differs from committed version; regenerate and commit."; \
fi
Loading

0 comments on commit 366748a

Please sign in to comment.