Skip to content
This repository has been archived by the owner on Sep 17, 2024. It is now read-only.

chore: move skip responsibility to the automation, not the CI #528

Merged
merged 2 commits into from
Dec 3, 2020

Conversation

mdelapenya
Copy link
Contributor

What does this PR do?

It moves the ability to skip scenarios tagged as @skip from the CI to the automation.

Why is it important?

Now it's possible to decide whether to skip scenarios or not, setting it to true (do skip) by default.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have run the Unit tests for the CLI, and they are passing locally
  • I have run the End-2-End tests for the suite I'm working on, and they are passing locally
  • I have noticed new Go dependencies (run make notice in the proper directory)

Author's Checklist

  • Variable expansion in Makefile

How to test this PR locally

Do not skip scenarios:

$ SUITE="metricbeat" \
    SKIP_SCENARIOS=false \
    TAGS="integrations && apache" DEVELOPER_MODE=true \
    TIMEOUT_FACTOR=3 LOG_LEVEL=TRACE \
    make -C e2e functional-test

go get -v github.com/cucumber/godog/cmd/godog@v0.10.0
go: found github.com/cucumber/godog/cmd/godog in github.com/cucumber/godog v0.10.0
cd _suites/metricbeat && \
	OP_LOG_LEVEL=TRACE \
	OP_LOG_INCLUDE_TIMESTAMP=TRUE \
	TIMEOUT_FACTOR=3 \
	STACK_VERSION= \
	DEVELOPER_MODE=true \
	godog --format=pretty --tags "integrations && apache"

Skip scenarios (default):

$ SUITE="metricbeat" \
    TAGS="integrations && apache" DEVELOPER_MODE=true \
    TIMEOUT_FACTOR=3 LOG_LEVEL=TRACE \
    make -C e2e functional-test

go get -v github.com/cucumber/godog/cmd/godog@v0.10.0
go: found github.com/cucumber/godog/cmd/godog in github.com/cucumber/godog v0.10.0
cd _suites/metricbeat && \
	OP_LOG_LEVEL=TRACE \
	OP_LOG_INCLUDE_TIMESTAMP=TRUE \
	TIMEOUT_FACTOR=3 \
	STACK_VERSION= \
	DEVELOPER_MODE=true \
	godog --format=pretty --tags "integrations && apache && ~skip"

Related issues

Follow-ups

@mdelapenya mdelapenya self-assigned this Dec 3, 2020
@mdelapenya mdelapenya requested review from a team and EricDavisX December 3, 2020 07:04
@mdelapenya mdelapenya marked this pull request as ready for review December 3, 2020 08:57
@mdelapenya
Copy link
Contributor Author

The CI logs reflect the changes are positive. Will be able to merge if the CI is green

@elasticmachine
Copy link
Contributor

elasticmachine commented Dec 3, 2020

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: mdelapenya commented: jenkins run the tests please

  • Start Time: 2020-12-03T11:44:26.265+0000

  • Duration: 26 min 41 sec

Test stats 🧪

Test Results
Failed 0
Passed 78
Skipped 12
Total 90

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 78
Skipped 12
Total 90

@mdelapenya
Copy link
Contributor Author

I ran the CI job setting the new Jenkins input parameter to false, so it won't skip the desired scenarios, causing the two errors (the scenario is skipped).

I'm gonna trigger the CI for the PR again, and if it passes, will merge.

@mdelapenya
Copy link
Contributor Author

jenkins run the tests please

STACK_VERSION?=
PICKLES_VERSION?="2.20.1"
VERSION_VALUE=`cat ../cli/VERSION.txt`

ifneq ($(TAGS),)
TAGS_FLAG=--tags
ifeq ($(SKIP_SCENARIOS),true)
## We always want to skip scenarios tagged with @skip
TAGS+= && ~skip
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I quoted the value here, but it ended up adding them to the value instead

@mdelapenya mdelapenya merged commit f98f802 into elastic:master Dec 3, 2020
@mdelapenya mdelapenya deleted the hide-skip branch December 3, 2020 16:02
mdelapenya added a commit to mdelapenya/e2e-testing that referenced this pull request Dec 4, 2020
…c#528)

* chore: move skip responsibility to the automation, not the CI

* chore: support skipping scenarios from Jenkins UI
mdelapenya added a commit to mdelapenya/e2e-testing that referenced this pull request Dec 4, 2020
…c#528)

* chore: move skip responsibility to the automation, not the CI

* chore: support skipping scenarios from Jenkins UI
mdelapenya added a commit that referenced this pull request Dec 4, 2020
* chore: move skip responsibility to the automation, not the CI

* chore: support skipping scenarios from Jenkins UI
mdelapenya added a commit that referenced this pull request Dec 4, 2020
* chore: move skip responsibility to the automation, not the CI

* chore: support skipping scenarios from Jenkins UI
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants