Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Xpack Foo module on mage build system #9242

Merged
merged 9 commits into from
Nov 27, 2018
Merged

Xpack Foo module on mage build system #9242

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
48c5c17
Build, test, package x-pack/metricbeat with mage
andrewkroh Nov 21, 2018
d997a82
Add PROJECTS_XPACK_MAGE to top-level Makefile
andrewkroh Nov 26, 2018
a6de97a
Add docker to packages.yml for x-pack/metricbeat
andrewkroh Nov 26, 2018
cffa525
Added Foo module
sayden Nov 26, 2018
fd128f8
updates after mage fmt update command
sayden Nov 26, 2018
d7eda29
fix a couple of minor things within magefile
sayden Nov 26, 2018
213fe31
Add a near empty docker-compose.yml file
sayden Nov 26, 2018
d302141
Remove mssql env_file
andrewkroh Nov 27, 2018
a2b519f
Added metricbeat/metricbeat.reference.yml metricbeat/metricbeat.yml a…
sayden Nov 27, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,10 @@ jobs:
env: TARGETS="-C metricbeat crosscompile"
go: $GO_VERSION
stage: test
- os: linux
env: TARGETS="-C x-pack/metricbeat testsuite"
go: $GO_VERSION
stage: test

# Packetbeat
- os: linux
Expand Down
16 changes: 11 additions & 5 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,18 @@ REVIEWDOG_OPTIONS?=-diff "git diff master"
REVIEWDOG_REPO=github.com/haya14busa/reviewdog/cmd/reviewdog
XPACK_SUFFIX=x-pack/

# PROJECTS_XPACK_MAGE is a list of Beats whose primary build logic is based in
# Mage. For compatibility with CI testing these projects support a subset of the
# makefile targets. After all Beats converge to primarily using Mage we can
# remove this and treat all sub-projects the same.
PROJECTS_XPACK_MAGE=x-pack/filebeat x-pack/metricbeat

# Runs complete testsuites (unit, system, integration) for all beats with coverage and race detection.
# Also it builds the docs and the generators

.PHONY: testsuite
testsuite:
@$(foreach var,$(PROJECTS),$(MAKE) -C $(var) testsuite || exit 1;)
@$(foreach var,$(PROJECTS) $(PROJECTS_XPACK_MAGE),$(MAKE) -C $(var) testsuite || exit 1;)

.PHONY: setup-commit-hook
setup-commit-hook:
Expand Down Expand Up @@ -54,13 +60,13 @@ coverage-report:

.PHONY: update
update: notice
@$(foreach var,$(PROJECTS),$(MAKE) -C $(var) update || exit 1;)
@$(foreach var,$(PROJECTS) $(PROJECTS_XPACK_MAGE),$(MAKE) -C $(var) update || exit 1;)
@$(MAKE) -C deploy/kubernetes all

.PHONY: clean
clean:
@rm -rf build
@$(foreach var,$(PROJECTS),$(MAKE) -C $(var) clean || exit 1;)
@$(foreach var,$(PROJECTS) $(PROJECTS_XPACK_MAGE),$(MAKE) -C $(var) clean || exit 1;)
@$(MAKE) -C generator clean
@-mage -clean 2> /dev/null

Expand All @@ -72,7 +78,7 @@ clean-vendor:

.PHONY: check
check: python-env
@$(foreach var,$(PROJECTS) dev-tools x-pack/filebeat,$(MAKE) -C $(var) check || exit 1;)
@$(foreach var,$(PROJECTS) dev-tools $(PROJECTS_XPACK_MAGE),$(MAKE) -C $(var) check || exit 1;)
@# Checks also python files which are not part of the beats
@$(FIND) -name *.py -exec $(PYTHON_ENV)/bin/autopep8 -d --max-line-length 120 {} \; | (! grep . -q) || (echo "Code differs from autopep8's style" && false)
@# Validate that all updates were committed
Expand Down Expand Up @@ -107,7 +113,7 @@ misspell:

.PHONY: fmt
fmt: add-headers python-env
@$(foreach var,$(PROJECTS) dev-tools x-pack/filebeat,$(MAKE) -C $(var) fmt || exit 1;)
@$(foreach var,$(PROJECTS) dev-tools $(PROJECTS_XPACK_MAGE),$(MAKE) -C $(var) fmt || exit 1;)
@# Cleans also python files which are not part of the beats
@$(FIND) -name "*.py" -exec $(PYTHON_ENV)/bin/autopep8 --in-place --max-line-length 120 {} \;

Expand Down
2 changes: 1 addition & 1 deletion dev-tools/mage/integtest.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -274,7 +274,7 @@ func dockerComposeBuildImages() error {
return err
}

args := []string{"build", "--pull", "--force-rm"}
args := []string{"-p", dockerComposeProjectName(), "build", "--pull", "--force-rm"}
if _, noCache := os.LookupEnv("DOCKER_NOCACHE"); noCache {
args = append(args, "--no-cache")
}
Expand Down
25 changes: 15 additions & 10 deletions dev-tools/make/xpack.mk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,21 @@
#
# Variables
#
PWD := $(CURDIR)
.DEFAULT_GOAL := help
PWD := $(CURDIR)

#
# Includes
#
include $(ES_BEATS)/dev-tools/make/mage.mk

#
# Targets
# Targets (alphabetically sorted).
#
.PHONY: check
check: mage
mage check

.PHONY: clean
clean: mage
mage clean
Expand All @@ -23,16 +27,17 @@ clean: mage
fmt: mage
mage fmt

.PHONY: check
check: mage
mage check

.PHONY: testsuite
testsuite: mage
mage update build unitTest integTest

# Default target.
.PHONY: help
help:
@echo Use mage rather than make. Here are the available mage targets:
@mage -l

.PHONY: testsuite
testsuite: mage
mage update build unitTest integTest

.PHONY: update
update: mage
mage update

23 changes: 9 additions & 14 deletions metricbeat/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,16 @@
FROM golang:1.11.2
MAINTAINER Nicolas Ruflin <ruflin@elastic.co>
FROM golang:1.10.3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sayden Thanks to #11330 we found this change. I wonder if any of the changes here were intended / needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I remember this we were using 1.10.3 because of needs in the CI (formatting if I remember well) and probably. I don't remember to directly change this, I think it was a co-authored PR with @andrewkroh who maybe can give us some more light here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be that things changed during the initial opening of the PR and when it get merged. Could you try a PR to get back the "old" Dockerfile again or have a look on what the actual differences and implications could be?


RUN set -x && \
apt-get update && \
apt-get install -y --no-install-recommends \
netcat python-pip virtualenv && \
apt-get clean
RUN \
apt-get update \
&& apt-get install -y --no-install-recommends \
netcat \
python-pip \
virtualenv \
&& rm -rf /var/lib/apt/lists/*

RUN pip install --upgrade pip
RUN pip install --upgrade setuptools
RUN pip install --upgrade docker-compose==1.21.0

# Setup work environment
ENV METRICBEAT_PATH /go/src/github.com/elastic/beats/metricbeat

RUN mkdir -p $METRICBEAT_PATH/build/coverage
WORKDIR $METRICBEAT_PATH

# Add healthcheck for docker/healthcheck metricset to check during testing
# Add healthcheck for the docker/healthcheck metricset to check during testing.
HEALTHCHECK CMD exit 0
3 changes: 3 additions & 0 deletions x-pack/metricbeat/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
ES_BEATS ?= ../..

include $(ES_BEATS)/dev-tools/make/xpack.mk
Loading