feature file to support tests on 'install' subcommand

[EricDavisX]

This is feature test support for Agent.

#326 (comment)

• tests noted and discussed in this issue.

in support of this Beats ticket: elastic/beats#21019

What does this PR do?

The current WIP adds a feature file that I think is fairly well defined, with only 3 new steps - and fairly clear expectation for what the new steps should do.

Why is it important?

We have a new feature in Agent and have no e2e coverage for it yet.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have run the Unit tests for the CLI, and they are passing locally
• I have run the End-2-End tests for the suite I'm working on, and they are passing locally
• I have noticed new Go dependencies (run make notice in the proper directory)

[EricDavisX]

@mdelapenya I took a minor look at the Go implementation that would support the new steps and... didn't make any quick progress. Would you want to help do it? I can ping @blakerouse to see if he wants to work this now that the feature side is laid out.

[elasticmachine]

💔 Tests Failed

Expand to view the summary

Build stats

• Build Cause: [Pull request #334 opened]

• Start Time: 2020-09-29T21:57:37.021+0000

• Duration: 5 min 20 sec

Test stats 🧪

Test	Results
Failed	2
Passed	26
Skipped	12
Total	40

Test errors

Expand to view the tests failures

• Name: Initializing / Pre-Submit / Sanity checks / Fix End of Files – pre_commit.lint

  • Age: 1
  • Duration: 0
  • Error Details: error

• Name: Initializing / Pre-Submit / Sanity checks / checkgherkinlint – pre_commit.lint

  • Age: 1
  • Duration: 0
  • Error Details: error

Steps errors

Expand to view the steps failures

• Name: Build docs

  • Description: make build-docs

  • Duration: 0 min 16 sec

  • Start Time: 2020-09-29T22:02:37.874+0000

  • log

Log output

Expand to view the last 100 lines of log output

[2020-09-29T22:02:32.594Z] --- PASS: TestBuildWithWellFormedRemote (0.00s)
[2020-09-29T22:02:32.594Z] === RUN   TestClone
[2020-09-29T22:02:32.594Z] time="2020-09-29T22:02:32Z" level=info msg="Cloning project. This process could take long depending on its size" directory=/tmp/dir650785514/git/Hello-World url="https://github.com/octocat/Hello-World"
[2020-09-29T22:02:32.594Z] Enumerating objects: 7, done.
[2020-09-29T22:02:32.594Z] Total 7 (delta 0), reused 0 (delta 0), pack-reused 7
[2020-09-29T22:02:32.594Z] time="2020-09-29T22:02:32Z" level=info msg="Git clone succeed" url="https://github.com/octocat/Hello-World"
[2020-09-29T22:02:32.594Z] --- PASS: TestClone (0.29s)
[2020-09-29T22:02:32.594Z] === RUN   TestMkdirAll
[2020-09-29T22:02:32.594Z] --- PASS: TestMkdirAll (0.00s)
[2020-09-29T22:02:32.594Z] === RUN   TestRecover
[2020-09-29T22:02:32.594Z] --- PASS: TestRecover (0.00s)
[2020-09-29T22:02:32.594Z] === RUN   TestUpdateCreatesStateFile
[2020-09-29T22:02:32.594Z] time="2020-09-29T22:02:32Z" level=error msg="Cannot write file" error="open /tmp/dir408294683/.op/myprofile-profile.run: is a directory" target=/tmp/dir408294683/.op/myprofile-profile.run
[2020-09-29T22:02:32.594Z] time="2020-09-29T22:02:32Z" level=error msg="Could not create state file" stateFile=/tmp/dir408294683/.op/myprofile-profile.run
[2020-09-29T22:02:32.594Z] --- PASS: TestUpdateCreatesStateFile (0.00s)
[2020-09-29T22:02:32.594Z] PASS
[2020-09-29T22:02:32.594Z] ok  	github.com/elastic/e2e-testing/cli/internal	0.301s
[2020-09-29T22:02:32.594Z] ?   	github.com/elastic/e2e-testing/cli/services	[no test files]
[2020-09-29T22:02:32.594Z] === RUN   TestGetEnvBool
[2020-09-29T22:02:32.594Z] --- PASS: TestGetEnvBool (0.00s)
[2020-09-29T22:02:32.594Z] PASS
[2020-09-29T22:02:32.594Z] ok  	github.com/elastic/e2e-testing/cli/shell	0.006s
[2020-09-29T22:02:32.594Z] make: Leaving directory '/var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-334/src/github.com/elastic/e2e-testing/cli'
[2020-09-29T22:02:32.594Z] + go get -v -u github.com/jstemmer/go-junit-report
[2020-09-29T22:02:32.594Z] go: finding github.com/jstemmer/go-junit-report v0.9.1
[2020-09-29T22:02:32.594Z] go: downloading github.com/jstemmer/go-junit-report v0.9.1
[2020-09-29T22:02:32.594Z] go: extracting github.com/jstemmer/go-junit-report v0.9.1
[2020-09-29T22:02:32.858Z] github.com/jstemmer/go-junit-report/parser
[2020-09-29T22:02:32.858Z] github.com/jstemmer/go-junit-report/formatter
[2020-09-29T22:02:32.858Z] github.com/jstemmer/go-junit-report
[2020-09-29T22:02:33.122Z] + go-junit-report
[2020-09-29T22:02:34.255Z] Post stage
[2020-09-29T22:02:34.277Z] Recording test results
[2020-09-29T22:02:34.895Z] Archiving artifacts
[2020-09-29T22:02:35.919Z] [INFO] getVaultSecret: Getting secrets
[2020-09-29T22:02:36.006Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-09-29T22:02:37.098Z] + set +x
[2020-09-29T22:02:37.675Z] Login Succeeded
[2020-09-29T22:02:37.844Z] Running in /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-334/src/github.com/elastic/e2e-testing/e2e
[2020-09-29T22:02:38.173Z] + make build-docs
[2020-09-29T22:02:38.173Z] rm -fr docs
[2020-09-29T22:02:38.173Z] Unable to find image 'docker.elastic.co/observability-ci/picklesdoc:2.20.1' locally
[2020-09-29T22:02:38.750Z] 2.20.1: Pulling from observability-ci/picklesdoc
[2020-09-29T22:02:38.750Z] bf5952930446: Pulling fs layer
[2020-09-29T22:02:38.750Z] 8117f11b67da: Pulling fs layer
[2020-09-29T22:02:38.750Z] 13ced73c2532: Pulling fs layer
[2020-09-29T22:02:38.750Z] c571404027a0: Pulling fs layer
[2020-09-29T22:02:38.750Z] f1c55a76e11f: Pulling fs layer
[2020-09-29T22:02:38.750Z] c571404027a0: Waiting
[2020-09-29T22:02:38.750Z] f1c55a76e11f: Waiting
[2020-09-29T22:02:39.015Z] 8117f11b67da: Verifying Checksum
[2020-09-29T22:02:39.015Z] 8117f11b67da: Download complete
[2020-09-29T22:02:39.965Z] bf5952930446: Verifying Checksum
[2020-09-29T22:02:39.965Z] bf5952930446: Download complete
[2020-09-29T22:02:40.230Z] f1c55a76e11f: Verifying Checksum
[2020-09-29T22:02:40.230Z] f1c55a76e11f: Download complete
[2020-09-29T22:02:40.808Z] 13ced73c2532: Verifying Checksum
[2020-09-29T22:02:40.808Z] 13ced73c2532: Download complete
[2020-09-29T22:02:41.763Z] c571404027a0: Verifying Checksum
[2020-09-29T22:02:41.763Z] c571404027a0: Download complete
[2020-09-29T22:02:41.763Z] bf5952930446: Pull complete
[2020-09-29T22:02:42.032Z] 8117f11b67da: Pull complete
[2020-09-29T22:02:44.600Z] 13ced73c2532: Pull complete
[2020-09-29T22:02:49.925Z] c571404027a0: Pull complete
[2020-09-29T22:02:49.925Z] f1c55a76e11f: Pull complete
[2020-09-29T22:02:49.925Z] Digest: sha256:79be4e07a946eab28b28d9354e960b003a20dbf1c634f3cc700c566b429f9203
[2020-09-29T22:02:49.925Z] Status: Downloaded newer image for docker.elastic.co/observability-ci/picklesdoc:2.20.1
[2020-09-29T22:02:54.160Z] Pickles v.2.20.1.0
[2020-09-29T22:02:54.160Z]  
[2020-09-29T22:02:54.160Z] Generating documentation based on the following parameters 
[2020-09-29T22:02:54.160Z] ---------------------------------------------------------- 
[2020-09-29T22:02:54.160Z] Feature Directory              : /suites 
[2020-09-29T22:02:54.160Z] Output Directory               : /suites/docs 
[2020-09-29T22:02:54.160Z] Project Name                   : E2E Testing 
[2020-09-29T22:02:54.160Z] Project Version                : 1.0.0 
[2020-09-29T22:02:54.160Z] Language                       : en 
[2020-09-29T22:02:54.160Z] Incorporate Test Results?      : No 
[2020-09-29T22:02:54.160Z] Include Experimental Features? : No 
[2020-09-29T22:02:54.160Z] Exclude Tag                    :  
[2020-09-29T22:02:54.160Z] Technical Tag                    :  
[2020-09-29T22:02:54.160Z] There was an error parsing the feature file here: /suites/_suites/ingest-manager/features/agent_subcommands.feature
[2020-09-29T22:02:54.160Z] Errormessage was: 'Unable to parse feature' 
[2020-09-29T22:02:54.160Z] Some files (1) were not parsed correctly. 
[2020-09-29T22:02:54.160Z] Writing HTML to /suites/docs 
[2020-09-29T22:02:54.160Z] Pickles did not complete successfully Some files were not parsed correctly.
[2020-09-29T22:02:54.160Z] Makefile:23: recipe for target 'build-docs' failed
[2020-09-29T22:02:54.160Z] make: *** [build-docs] Error 1
[2020-09-29T22:02:54.240Z] Post stage
[2020-09-29T22:02:54.258Z] Running in /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-334/src/github.com/elastic/e2e-testing
[2020-09-29T22:02:54.304Z] Archiving artifacts
[2020-09-29T22:02:54.741Z] Stage "End-To-End Tests" skipped due to earlier failure(s)
[2020-09-29T22:02:54.837Z] Stage "Release" skipped due to earlier failure(s)
[2020-09-29T22:02:56.289Z] Running on worker-395930 in /var/lib/jenkins/workspace/e2e-tests_e2e-testing-mbp_PR-334
[2020-09-29T22:02:56.360Z] [INFO] getVaultSecret: Getting secrets
[2020-09-29T22:02:56.453Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-09-29T22:02:58.649Z] + chmod 755 generate-build-data.sh
[2020-09-29T22:02:58.649Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-334/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-334/runs/1 FAILURE 320162
[2020-09-29T22:02:58.649Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-334/runs/1/steps/?limit=10000 -o steps-info.json
[2020-09-29T22:02:59.372Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-334/runs/1/tests/?status=FAILED -o tests-errors.json
[2020-09-29T22:03:00.095Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-334/runs/1/log/ -o pipeline-log.txt

[EricDavisX]

the work here was picked up up in another branch and fixed - some work remains and is logged separately.
#326 - for initial work and
#379 for adding back in the .rpm/.deb tests we altered. i am closing this out!