Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add build tags for cloud tests #16937

Merged
merged 14 commits into from
Mar 17, 2020
Merged

Conversation

jsoriano
Copy link
Member

@jsoriano jsoriano commented Mar 10, 2020

What does this PR do?

Add build tags to AWS and Azure integration tests, and make tests fail if they cannot get a working configuration.

Add support for a TEST_TAGS environment variable in mage targets that add additional build tags to go tests calls.
When TEST_TAGS includes a cloud provider, it is expected that valid credentials to run these tests are available in the environment, if not, tests will fail. This is intentional, to avoid skipping tests by mistake due to some misconfiguration in CI.

Why is it important?

To add support of integration testing of cloud features in CI we need some way to select what tests to run in a way that can be easily defined in jenkins configuration files. With TEST_TAGS we can do this selection of tests.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (We need to review documenation about running tests in general, we also need to review how to document mage targets and supported environment variables)
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works

How to test this PR locally

Run integration tests for cloud features with go test:

  • Without corresponding tags, tests shouldn't be run.
  • With go test --tags=integration,aws, AWS integration tests should be run, with credentials they should work, without credentials they should fail.

Repeat the steps with mage goIntegTest:

  • Without TEST_TAGS, cloud tests shouldn't be run.
  • With TEST_TAGS=aws, AWS integration tests should be run, with credentials they should work, without credentials they should fail.

Related issues

@jsoriano jsoriano self-assigned this Mar 10, 2020
@andresrc andresrc added [zube]: Inbox [zube]: In Progress Team:Platforms Label for the Integrations - Platforms team and removed [zube]: Inbox labels Mar 11, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-platforms (Team:Platforms)

@jsoriano
Copy link
Member Author

@kaiyan-sheng @narph I would like to have your feedback about this change before opening for review.

Copy link
Contributor

@exekias exekias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, love to see this!

@jsoriano jsoriano added needs_backport PR is waiting to be backported to other branches. v7.7.0 labels Mar 13, 2020
Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@jsoriano jsoriano merged commit 1345a97 into elastic:master Mar 17, 2020
@jsoriano jsoriano deleted the cloud-tests-tags branch March 17, 2020 12:49
jsoriano added a commit to jsoriano/beats that referenced this pull request Mar 17, 2020
Add build tags to AWS and Azure integration tests, and
make tests fail if they cannot get a working configuration.

Add support for a `TEST_TAGS` environment variable in
mage targets that add additional build tags to go tests calls.
When `TEST_TAGS` includes a cloud provider, it is expected
that valid credentials to run these tests are available in the
environment, if not, tests will fail. This is intentional, to avoid
skipping tests by mistake due to some misconfiguration in CI.

(cherry picked from commit 1345a97)
@jsoriano jsoriano removed the needs_backport PR is waiting to be backported to other branches. label Mar 17, 2020
jsoriano added a commit that referenced this pull request Mar 17, 2020
Add build tags to AWS and Azure integration tests, and
make tests fail if they cannot get a working configuration.

Add support for a `TEST_TAGS` environment variable in
mage targets that add additional build tags to go tests calls.
When `TEST_TAGS` includes a cloud provider, it is expected
that valid credentials to run these tests are available in the
environment, if not, tests will fail. This is intentional, to avoid
skipping tests by mistake due to some misconfiguration in CI.

(cherry picked from commit 1345a97)
jsoriano added a commit to jsoriano/beats that referenced this pull request Mar 18, 2020
Add a `tag(tag)` decorator that skips a test if the tag is not included
in the comma-separated list of `TEST_TAGS` environment variable.
This offers an initial support for the similar implementation added for
mage in elastic#16937.
jsoriano added a commit that referenced this pull request Mar 19, 2020
Add a `tag(tag)` decorator that skips a test if the tag is not included
in the comma-separated list of `TEST_TAGS` environment variable.
This offers an initial support for the similar implementation added for
mage in #16937.
jsoriano added a commit to jsoriano/beats that referenced this pull request Mar 19, 2020
Add a `tag(tag)` decorator that skips a test if the tag is not included
in the comma-separated list of `TEST_TAGS` environment variable.
This offers an initial support for the similar implementation added for
mage in elastic#16937.

(cherry picked from commit 3bf05f5)
jsoriano added a commit that referenced this pull request Mar 19, 2020
Add a `tag(tag)` decorator that skips a test if the tag is not included
in the comma-separated list of `TEST_TAGS` environment variable.
This offers an initial support for the similar implementation added for
mage in #16937.

(cherry picked from commit 3bf05f5)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
review Team:Platforms Label for the Integrations - Platforms team :Testing v7.7.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants