Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate config before init NewInput to avoid uneccessory cleanup and kubernetes watcher leak #18630

Merged

Conversation

DanielQujun
Copy link
Contributor

@DanielQujun DanielQujun commented May 19, 2020

An alternative Fix #18629

When Filebeat using add_kubernetes_metadata processor and enabled input.reload, It will cause Kubernetes watcher goroutine leaks when the input config is wrong.

I looked through the code in filebeat/input/log/input.go, notice that there is a defer function to cleanup when executing NewInput failed.

func NewInput(

// NewInput instantiates a new Log
func NewInput(
	cfg *common.Config,
	outlet channel.Connector,
	context input.Context,
) (input.Input, error) {
	cleanupNeeded := true
	cleanupIfNeeded := func(f func() error) {
		if cleanupNeeded {
			f()
		}
	}

But it seems not easy to add Kubernetes watcher stopping function here, So I move the validate section to the top, then NewInput will return directly if the config not qualified.

// NewInput instantiates a new Log
func NewInput(
	cfg *common.Config,
	outlet channel.Connector,
	context input.Context,
) (input.Input, error) {
	cleanupNeeded := true
	cleanupIfNeeded := func(f func() error) {
		if cleanupNeeded {
			f()
		}
	}

	inputConfig := defaultConfig

	if err := cfg.Unpack(&inputConfig); err != nil {
		return nil, err
	}
	if err := inputConfig.resolveRecursiveGlobs(); err != nil {
		return nil, fmt.Errorf("Failed to resolve recursive globs in config: %v", err)
	}
	if err := inputConfig.normalizeGlobPatterns(); err != nil {
		return nil, fmt.Errorf("Failed to normalize globs patterns: %v", err)
	}

	if len(inputConfig.Paths) == 0 {
		return nil, fmt.Errorf("each input must have at least one path defined")
	}

@elasticmachine
Copy link
Collaborator

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

1 similar comment
@elasticmachine
Copy link
Collaborator

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 19, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented May 19, 2020

💔 Tests Failed

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

Test stats 🧪

Test Results
Failed 1
Passed 9214
Skipped 1541
Total 10756

Test errors

Expand to view the tests failures

  • Name: Build and Test / Metricbeat OSS Integration tests / TestData – database

    • Status: FAILED
    • Age: 1
    • Duration: 1.76
    • Error Details: Failed

Steps errors

Expand to view the steps failures

  • Name: Report to Codecov

    • Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done

    • Result: FAILURE

    • Duration: 2 min 22 sec

    • Start Time: 2020-05-20T16:40:34.496+0000

    • log

  • Name: Report to Codecov

    • Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done

    • Result: FAILURE

    • Duration: 2 min 22 sec

    • Start Time: 2020-05-20T16:29:20.971+0000

    • log

  • Name: Mage goIntegTest

    • Description: mage goIntegTest

    • Result: FAILURE

    • Duration: 36 min 47 sec

    • Start Time: 2020-05-20T16:17:44.716+0000

    • log

  • Name: Report to Codecov

    • Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done

    • Result: FAILURE

    • Duration: 2 min 22 sec

    • Start Time: 2020-05-20T16:50:44.343+0000

    • log

Log output

Expand to view the last 100 lines of log output

[2020-05-20T17:10:22.249Z] 
[2020-05-20T17:10:22.249Z] OK (SKIP=178)
[2020-05-20T17:10:22.249Z] >> python test: Unit Testing Complete
[2020-05-20T17:10:22.326Z] Recording test results
[2020-05-20T17:10:31.144Z] Stashed 2 file(s)
[2020-05-20T17:10:31.155Z] Archiving artifacts
[2020-05-20T17:10:33.985Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats
[2020-05-20T17:10:34.293Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-05-20T17:10:34.306Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Lint
[2020-05-20T17:10:34.382Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Elastic-Agent-Mac-OS-X
[2020-05-20T17:10:34.447Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Winlogbeat-oss
[2020-05-20T17:10:34.513Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Elastic-Agent-x-pack
[2020-05-20T17:10:34.585Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Dockerlogbeat
[2020-05-20T17:10:34.649Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Journalbeat-oss
[2020-05-20T17:10:34.714Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Generators-Metricbeat-Linux
[2020-05-20T17:10:34.780Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack-Mac-OS-X
[2020-05-20T17:10:34.852Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-Mac-OS-X
[2020-05-20T17:10:34.921Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Functionbeat-x-pack
[2020-05-20T17:10:34.987Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Filebeat-Mac-OS-X
[2020-05-20T17:10:35.055Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Elastic-Agent-x-pack-Windows
[2020-05-20T17:10:35.124Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-OSS-Unit-tests
[2020-05-20T17:10:35.192Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-crosscompile
[2020-05-20T17:10:35.272Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Heartbeat-oss
[2020-05-20T17:10:35.340Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Auditbeat-x-pack
[2020-05-20T17:10:35.453Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Libbeat-x-pack
[2020-05-20T17:10:35.519Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Functionbeat-Mac-OS-X-x-pack
[2020-05-20T17:10:35.587Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Auditbeat-Linux
[2020-05-20T17:10:35.655Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Heartbeat-Mac-OS-X
[2020-05-20T17:10:35.720Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Packetbeat-oss
[2020-05-20T17:10:35.786Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Filebeat-Windows
[2020-05-20T17:10:35.853Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Filebeat-x-pack-Windows
[2020-05-20T17:10:35.921Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Auditbeat-crosscompile
[2020-05-20T17:10:35.986Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Winlogbeat-Windows-x-pack
[2020-05-20T17:10:36.051Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Auditbeat-Mac-OS-X
[2020-05-20T17:10:36.118Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Filebeat-x-pack-Mac-OS-X
[2020-05-20T17:10:36.182Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Winlogbeat-Windows
[2020-05-20T17:10:36.247Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Generators-Beat-Linux
[2020-05-20T17:10:36.313Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack-Windows
[2020-05-20T17:10:36.377Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Filebeat-x-pack
[2020-05-20T17:10:36.451Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Heartbeat-Windows
[2020-05-20T17:10:36.516Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Filebeat-oss
[2020-05-20T17:10:36.580Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Functionbeat-Windows
[2020-05-20T17:10:36.645Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Libbeat-oss
[2020-05-20T17:10:36.709Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-Python-integration-tests
[2020-05-20T17:10:36.781Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Auditbeat-Windows
[2020-05-20T17:10:36.847Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests
[2020-05-20T17:10:36.919Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Libbeat-crosscompile
[2020-05-20T17:10:36.985Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Libbeat-stress-tests
[2020-05-20T17:10:37.054Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Generators-Metricbeat-Mac-OS-X
[2020-05-20T17:10:37.119Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Generators-Beat-Mac-OS-X
[2020-05-20T17:10:37.184Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack
[2020-05-20T17:10:37.251Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-Windows
[2020-05-20T17:10:37.602Z] + cat
[2020-05-20T17:10:37.602Z] + /usr/local/bin/runbld ./runbld-script
[2020-05-20T17:10:37.602Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-05-20T17:10:44.215Z] runbld>>> runbld started
[2020-05-20T17:10:44.215Z] runbld>>> 1.6.11/a66728ff8f4356963772e6e6d2069392fa06acbe
[2020-05-20T17:10:45.604Z] runbld>>> The following profiles matched the job 'Beats/beats-beats-mbp/PR-18630' in order of occurrence in the config (last value wins).
[2020-05-20T17:10:46.547Z] runbld>>> Debug logging enabled.
[2020-05-20T17:10:46.547Z] runbld>>> Storing result
[2020-05-20T17:10:46.808Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-05-20T17:10:46.808Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200520171046-1C8EB6DE
[2020-05-20T17:10:46.808Z] runbld>>> Adding system facts.
[2020-05-20T17:10:47.752Z] runbld>>> Adding vcs info for the latest commit:  d0d7d80018975912df198b9f8222c0a6568deb00
[2020-05-20T17:10:47.752Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-05-20T17:10:47.752Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-05-20T17:10:47.752Z] Processing JUnit reports with runbld...
[2020-05-20T17:10:47.752Z] + echo 'Processing JUnit reports with runbld...'
[2020-05-20T17:10:48.325Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-05-20T17:10:48.325Z] runbld>>> DURATION: 14ms
[2020-05-20T17:10:48.325Z] runbld>>> STDOUT: 40 bytes
[2020-05-20T17:10:48.325Z] runbld>>> STDERR: 49 bytes
[2020-05-20T17:10:48.325Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-05-20T17:10:48.325Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats
[2020-05-20T17:10:49.716Z] runbld>>> Storing build metadata: 
[2020-05-20T17:10:49.716Z] runbld>>> Adding test report.
[2020-05-20T17:10:49.716Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats
[2020-05-20T17:10:50.659Z] runbld>>> Found 111 test output files
[2020-05-20T17:10:51.638Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-openmetrics.xml
[2020-05-20T17:10:51.638Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-istio.xml
[2020-05-20T17:10:51.638Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-iis.xml
[2020-05-20T17:10:51.638Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-activemq.xml
[2020-05-20T17:10:51.638Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-tomcat.xml
[2020-05-20T17:10:51.639Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-cloudfoundry.xml
[2020-05-20T17:10:52.218Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-windows.xml
[2020-05-20T17:10:52.480Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-graphite.xml
[2020-05-20T17:10:53.054Z] runbld>>> Test output logs contained: Errors: 0 Failures: 1 Tests: 10606 Skipped: 1304
[2020-05-20T17:10:53.055Z] runbld>>> Storing result
[2020-05-20T17:10:53.055Z] runbld>>> FAILURES: 1
[2020-05-20T17:10:53.626Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-05-20T17:10:53.626Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200520171046-1C8EB6DE
[2020-05-20T17:10:53.626Z] runbld>>> Email notification disabled by environment variable.
[2020-05-20T17:10:53.626Z] runbld>>> Slack notification disabled by environment variable.
[2020-05-20T17:10:59.653Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630
[2020-05-20T17:10:59.883Z] [INFO] getVaultSecret: Getting secrets
[2020-05-20T17:10:59.936Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-05-20T17:11:00.732Z] + chmod 755 generate-build-data.sh
[2020-05-20T17:11:00.732Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18630/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18630/runs/3 FAILURE 4533132
[2020-05-20T17:11:01.283Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18630/runs/3/steps/?limit=10000 -o steps-info.json
[2020-05-20T17:11:02.626Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18630/runs/3/tests/?status=FAILED -o tests-errors.json

@DanielQujun
Copy link
Contributor Author

@exekias Hi, PTAL

@exekias
Copy link
Contributor

exekias commented May 20, 2020

Thank you so much for contributing @DanielQujun! This is looking good to me, pinging @urso just in case you see something I miss

@exekias exekias requested a review from urso May 20, 2020 13:13
@exekias exekias added review Team:Integrations Label for the Integrations team labels May 20, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 20, 2020
@exekias
Copy link
Contributor

exekias commented May 20, 2020

@DanielQujun could you please add a new line to the CHANGELOG.next.asciidoc?

@DanielQujun DanielQujun force-pushed the validate_input_config_before_init branch 2 times, most recently from e8aee9d to 3bd083c Compare May 20, 2020 15:55
@DanielQujun
Copy link
Contributor Author

DanielQujun commented May 20, 2020

@DanielQujun could you please add a new line to the CHANGELOG.next.asciidoc?

thanks for your review, updated done.

@DanielQujun
Copy link
Contributor Author

@urso Hi, would you please take a look at your convenience.

Copy link

@urso urso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Always +1 to validate input/configuration before creating objects/resources. Thanks for contributing.

@urso
Copy link

urso commented May 27, 2020

jenkins run tests

@elasticmachine
Copy link
Collaborator

elasticmachine commented May 27, 2020

💔 Build Failed

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #18630 updated]

  • Start Time: 2020-05-28T02:52:46.915+0000

  • Duration: 124 min 46 sec

Test stats 🧪

Test Results
Failed 0
Passed 9115
Skipped 1537
Total 10652

Steps errors

Expand to view the steps failures

  • Name: Make -C libbeat testsuite
    • Description: make -C libbeat testsuite

    • Duration: 26 min 30 sec

    • Start Time: 2020-05-28T03:16:50.293+0000

    • log

Log output

Expand to view the last 100 lines of log output

[2020-05-28T04:56:04.938Z] + FILE=packetbeat/build/coverage/full.cov
[2020-05-28T04:56:04.938Z] + [ -f packetbeat/build/coverage/full.cov ]
[2020-05-28T04:56:04.938Z] + FILE=winlogbeat/build/coverage/full.cov
[2020-05-28T04:56:04.938Z] + [ -f winlogbeat/build/coverage/full.cov ]
[2020-05-28T04:56:04.938Z] + FILE=journalbeat/build/coverage/full.cov
[2020-05-28T04:56:04.938Z] + [ -f journalbeat/build/coverage/full.cov ]
[2020-05-28T04:56:06.202Z] Failed in branch Auditbeat x-pack
[2020-05-28T04:56:06.314Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats
[2020-05-28T04:56:06.621Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-05-28T04:56:06.634Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Lint
[2020-05-28T04:56:06.705Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Elastic-Agent-Mac-OS-X
[2020-05-28T04:56:06.768Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Elastic-Agent-x-pack
[2020-05-28T04:56:06.832Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Filebeat-x-pack-Mac-OS-X
[2020-05-28T04:56:06.896Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Winlogbeat-oss
[2020-05-28T04:56:06.960Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Dockerlogbeat
[2020-05-28T04:56:07.024Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Journalbeat-oss
[2020-05-28T04:56:07.088Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Generators-Metricbeat-Linux
[2020-05-28T04:56:07.153Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Filebeat-Mac-OS-X
[2020-05-28T04:56:07.222Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-Mac-OS-X
[2020-05-28T04:56:07.295Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Functionbeat-x-pack
[2020-05-28T04:56:07.364Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Elastic-Agent-x-pack-Windows
[2020-05-28T04:56:07.434Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-OSS-Unit-tests
[2020-05-28T04:56:07.501Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-crosscompile
[2020-05-28T04:56:07.574Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Heartbeat-oss
[2020-05-28T04:56:07.643Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Functionbeat-Mac-OS-X-x-pack
[2020-05-28T04:56:07.712Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Libbeat-x-pack
[2020-05-28T04:56:07.784Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Winlogbeat-Windows-x-pack
[2020-05-28T04:56:07.853Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Filebeat-Windows
[2020-05-28T04:56:07.918Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Filebeat-x-pack-Windows
[2020-05-28T04:56:07.983Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Packetbeat-oss
[2020-05-28T04:56:08.049Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Heartbeat-Mac-OS-X
[2020-05-28T04:56:08.122Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-Windows
[2020-05-28T04:56:08.189Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Winlogbeat-Windows
[2020-05-28T04:56:08.257Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Generators-Beat-Linux
[2020-05-28T04:56:08.323Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack-Windows
[2020-05-28T04:56:08.391Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Libbeat-oss
[2020-05-28T04:56:08.456Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Auditbeat-Linux
[2020-05-28T04:56:08.528Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Filebeat-oss
[2020-05-28T04:56:08.593Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Functionbeat-Windows
[2020-05-28T04:56:08.659Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Auditbeat-crosscompile
[2020-05-28T04:56:08.727Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Heartbeat-Windows
[2020-05-28T04:56:08.794Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Filebeat-x-pack
[2020-05-28T04:56:08.858Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack-Mac-OS-X
[2020-05-28T04:56:08.928Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Auditbeat-Mac-OS-X
[2020-05-28T04:56:08.993Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Generators-Metricbeat-Mac-OS-X
[2020-05-28T04:56:09.060Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests
[2020-05-28T04:56:09.157Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-Python-integration-tests
[2020-05-28T04:56:09.227Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Generators-Beat-Mac-OS-X
[2020-05-28T04:56:09.297Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Auditbeat-Windows
[2020-05-28T04:56:09.372Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack
[2020-05-28T04:56:09.449Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Auditbeat-x-pack
[2020-05-28T04:56:09.807Z] + cat
[2020-05-28T04:56:09.807Z] + /usr/local/bin/runbld ./runbld-script
[2020-05-28T04:56:09.807Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-05-28T04:56:16.407Z] runbld>>> runbld started
[2020-05-28T04:56:16.407Z] runbld>>> 1.6.11/a66728ff8f4356963772e6e6d2069392fa06acbe
[2020-05-28T04:56:17.795Z] runbld>>> The following profiles matched the job 'Beats/beats-beats-mbp/PR-18630' in order of occurrence in the config (last value wins).
[2020-05-28T04:56:19.180Z] runbld>>> Debug logging enabled.
[2020-05-28T04:56:19.180Z] runbld>>> Storing result
[2020-05-28T04:56:19.180Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-05-28T04:56:19.180Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200528045618-A97000D2
[2020-05-28T04:56:19.180Z] runbld>>> Adding system facts.
[2020-05-28T04:56:20.126Z] runbld>>> Adding vcs info for the latest commit:  a93628bb1602f9bef93e3f382ab691dcecd72911
[2020-05-28T04:56:20.387Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-05-28T04:56:20.387Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-05-28T04:56:20.387Z] + echo 'Processing JUnit reports with runbld...'
[2020-05-28T04:56:20.387Z] Processing JUnit reports with runbld...
[2020-05-28T04:56:20.648Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-05-28T04:56:20.648Z] runbld>>> DURATION: 21ms
[2020-05-28T04:56:20.648Z] runbld>>> STDOUT: 40 bytes
[2020-05-28T04:56:20.648Z] runbld>>> STDERR: 49 bytes
[2020-05-28T04:56:20.648Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-05-28T04:56:20.648Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats
[2020-05-28T04:56:22.033Z] runbld>>> Storing build metadata: 
[2020-05-28T04:56:22.033Z] runbld>>> Adding test report.
[2020-05-28T04:56:22.033Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats
[2020-05-28T04:56:23.420Z] runbld>>> Found 109 test output files
[2020-05-28T04:56:23.420Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-graphite.xml
[2020-05-28T04:56:23.420Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-OSS-Integration-tests/metricbeat/build/TEST-go-integration-windows.xml
[2020-05-28T04:56:24.811Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-openmetrics.xml
[2020-05-28T04:56:24.811Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-tomcat.xml
[2020-05-28T04:56:24.811Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-istio.xml
[2020-05-28T04:56:24.811Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-cloudfoundry.xml
[2020-05-28T04:56:24.811Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-iis.xml
[2020-05-28T04:56:24.811Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630/src/github.com/elastic/beats/Metricbeat-x-pack/x-pack/metricbeat/build/TEST-go-integration-activemq.xml
[2020-05-28T04:56:25.384Z] runbld>>> Test output logs contained: Errors: 0 Failures: 0 Tests: 10502 Skipped: 1299
[2020-05-28T04:56:25.384Z] runbld>>> Storing result
[2020-05-28T04:56:25.384Z] runbld>>> FAILURES: 0
[2020-05-28T04:56:25.646Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-05-28T04:56:25.646Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200528045618-A97000D2
[2020-05-28T04:56:25.646Z] runbld>>> Email notification disabled by environment variable.
[2020-05-28T04:56:25.646Z] runbld>>> Slack notification disabled by environment variable.
[2020-05-28T04:56:31.688Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-18630
[2020-05-28T04:56:31.910Z] [INFO] getVaultSecret: Getting secrets
[2020-05-28T04:56:31.949Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-05-28T04:56:32.685Z] + chmod 755 generate-build-data.sh
[2020-05-28T04:56:32.685Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18630/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18630/runs/3 FAILURE 7425508
[2020-05-28T04:56:32.685Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18630/runs/3/steps/?limit=10000 -o steps-info.json
[2020-05-28T04:56:34.029Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18630/runs/3/tests/?status=FAILED -o tests-errors.json
[2020-05-28T04:56:34.580Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-18630/runs/3/log/ -o pipeline-log.txt

…t's misconfigured(elastic#18629)

fix log format missing in libbeat/common/kubernetes/util.go

Signed-off-by: 屈骏 <qujun@tiduyun.com>
@DanielQujun DanielQujun force-pushed the validate_input_config_before_init branch from 3bd083c to a93628b Compare May 28, 2020 02:52
@DanielQujun
Copy link
Contributor Author

@exekias Hi, I have rebased with master branch, Do I get a flaky test?

@exekias exekias added the needs_backport PR is waiting to be backported to other branches. label May 28, 2020
@exekias exekias merged commit 74d81c2 into elastic:master May 28, 2020
@exekias
Copy link
Contributor

exekias commented May 28, 2020

It seems the test was unrelated

Thank you for contributing!! 🎉

exekias pushed a commit to exekias/beats that referenced this pull request May 28, 2020
…t's misconfigured(elastic#18629) (elastic#18630)

fix log format missing in libbeat/common/kubernetes/util.go

Signed-off-by: 屈骏 <qujun@tiduyun.com>
(cherry picked from commit 74d81c2)
@exekias exekias added v7.9.0 and removed needs_backport PR is waiting to be backported to other branches. labels May 28, 2020
exekias pushed a commit that referenced this pull request May 28, 2020
…t's misconfigured(#18629) (#18630) (#18810)

fix log format missing in libbeat/common/kubernetes/util.go

Signed-off-by: 屈骏 <qujun@tiduyun.com>
(cherry picked from commit 74d81c2)

Co-authored-by: Daniel <qujunorz@163.com>
@DanielQujun
Copy link
Contributor Author

Great! Thanks~

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
review Team:Integrations Label for the Integrations team v7.9.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[filebeat] Kubernetes watcher leaks when filebeat reload with wrong input config
5 participants