Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[7.x](backport #25998) Add more ECS fields to logs #26266

Merged
merged 1 commit into from
Jun 16, 2021
Merged

[7.x](backport #25998) Add more ECS fields to logs #26266

merged 1 commit into from
Jun 16, 2021

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Jun 11, 2021 •
edited by zube bot
Loading

This is an automatic backport of pull request #25998 done by Mergify.

Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

  • @Mergifyio refresh will re-evaluate the rules
  • @Mergifyio rebase will rebase this PR on its base branch
  • @Mergifyio update will merge the base branch into this PR
  • @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

  • look at your merge queues
  • generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.io/

@jalvz
Add more ECS fields to logs (#25998)
96dbe30 
(cherry picked from commit bfbadaa)
@mergify mergify bot added the backport label Jun 11, 2021
@mergify mergify bot assigned jalvz Jun 11, 2021
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jun 11, 2021
@botelastic
Copy link

botelastic bot commented Jun 11, 2021

This pull request doesn't have a Team:<team> label.

@elasticmachine
Copy link
Collaborator

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #26266 opened

  • Start Time: 2021-06-11T20:47:03.951+0000

  • Duration: 94 min 56 sec

  • Commit: 96dbe30

Test stats 🧪

Test Results
Failed 1
Passed 47463
Skipped 5207
Total 52671

Trends 🧪

Image of Build Times

Image of Tests

Test errors 1

Expand to view the tests failures

Extended / filebeat-windows-2016-windows-2016 / test_rotating_file – filebeat.tests.system.test_registrar.Test
    Expand to view the error details

     beat.beat.TimeoutError: Timeout waiting for 'cond' to be true. Waited 10 seconds.

    Expand to view the stacktrace

     self = <test_registrar.Test testMethod=test_rotating_file>

    def test_rotating_file(self):
        """
        Checks that the registry is properly updated after a file is rotated
        """
        self.render_config_template(
            path=os.path.abspath(self.working_dir) + "/log/*",
            close_inactive="1s"
        )
    
        os.mkdir(self.working_dir + "/log/")
        testfile_path = self.working_dir + "/log/test.log"
    
        filebeat = self.start_beat()
    
        with open(testfile_path, 'w') as testfile:
            testfile.write("offset 9\n")
    
        self.wait_until(lambda: self.output_has(lines=1),
                        max_timeout=10)
    
        testfilerenamed = self.working_dir + "/log/test.1.log"
        os.rename(testfile_path, testfilerenamed)
    
        with open(testfile_path, 'w') as testfile:
            testfile.write("offset 10\n")
    
        self.wait_until(lambda: self.output_has(lines=2),
                        max_timeout=10)
    
        # Wait until rotation is detected
>       self.wait_until(
            lambda: self.log_contains(
                "Updating state for renamed file"),
            max_timeout=10)

tests\system\test_registrar.py:308: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <test_registrar.Test testMethod=test_rotating_file>
cond = <function Test.test_rotating_file.<locals>.<lambda> at 0x000002227253B430>
max_timeout = 10, poll_interval = 0.1, name = 'cond'

    def wait_until(self, cond, max_timeout=10, poll_interval=0.1, name="cond"):
        """
        Waits until the cond function returns true,
        or until the max_timeout is reached. Calls the cond
        function every poll_interval seconds.
    
        If the max_timeout is reached before cond() returns
        true, an exception is raised.
        """
        start = datetime.now()
        while not cond():
            if datetime.now() - start > timedelta(seconds=max_timeout):
>               raise TimeoutError("Timeout waiting for '{}' to be true. ".format(name) +
                                   "Waited {} seconds.".format(max_timeout))
E               beat.beat.TimeoutError: Timeout waiting for 'cond' to be true. Waited 10 seconds.

..\libbeat\tests\system\beat\beat.py:362: TimeoutError

Steps errors 3

Expand to view the steps failures

filebeat-windows-2016-windows-2016 - mage build unitTest
  • Took 5 min 27 sec . View more details on here
  • Description: mage build unitTest
gsutil -m -q cp -a public-read test-build-artifacts-filebeat-windows-2016-windows-2016-tgz gs://beat
  • Took 0 min 5 sec . View more details on here
  • Description: @echo off gsutil -m -q cp -a public-read test-build-artifacts-filebeat-windows-2016-windows-2016-tgz gs://beats-ci-temp/Beats/beats/PR-26266-1
Error signal
  • Took 0 min 0 sec . View more details on here
  • Description: Error 'hudson.AbortException: script returned exit code 1'

Log output

Expand to view the last 100 lines of log output 

[2021-06-11T22:18:03.358Z] Copy .\packetbeat\null\build into build\packetbeat\null\build
[2021-06-11T22:18:03.403Z] Running in C:\Users\jenkins\workspace\PR-26266-1-4270169e-dba2-4b86-bdf2-64c91c8f21ed\src\github.com\elastic\beats\build
[2021-06-11T22:18:03.466Z] Recording test results
[2021-06-11T22:18:04.112Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-06-11T22:18:04.486Z] + gcloud auth activate-service-account --key-file ****
[2021-06-11T22:18:05.054Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-06-11T22:18:06.089Z] [Checks API] No suitable checks publisher found.
[2021-06-11T22:18:06.469Z] 
[2021-06-11T22:18:06.469Z] C:\Users\jenkins\workspace\PR-26266-1-a9999d6f-d1d4-4433-8a40-e922ecf3ed5b\src\github.com\elastic\beats>python .ci/scripts/pre_archive_test.py 
[2021-06-11T22:18:06.518Z] + gsutil -m -q cp -a public-read ZmlsZWJlYXQtd2luZG93cy0xMC13aW5kb3dzLTEwOTZkYmUzMGEwNjEwMjM2NmY0ZTBmMjRjMzlmNWRmY2ZjNzUyNzQ3Ng gs://beats-ci-temp/ci/cache/
[2021-06-11T22:18:06.828Z] 
[2021-06-11T22:18:06.828Z] C:\Users\jenkins\workspace\PR-26266-1-4270169e-dba2-4b86-bdf2-64c91c8f21ed\src\github.com\elastic\beats>go clean -modcache 
[2021-06-11T22:18:09.769Z] Copy .\filebeat\build into build\filebeat\build
[2021-06-11T22:18:09.769Z] Copy .\filebeat\null\build into build\filebeat\null\build
[2021-06-11T22:18:09.813Z] Running in C:\Users\jenkins\workspace\PR-26266-1-a9999d6f-d1d4-4433-8a40-e922ecf3ed5b\src\github.com\elastic\beats\build
[2021-06-11T22:18:09.878Z] Recording test results
[2021-06-11T22:18:12.461Z] [Checks API] No suitable checks publisher found.
[2021-06-11T22:18:13.010Z] 
[2021-06-11T22:18:13.010Z] C:\Users\jenkins\workspace\PR-26266-1-a9999d6f-d1d4-4433-8a40-e922ecf3ed5b\src\github.com\elastic\beats>go clean -modcache 
[2021-06-11T22:18:38.098Z] + gsutil --version
[2021-06-11T22:18:39.616Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-06-11T22:18:39.970Z] + gcloud auth activate-service-account --key-file ****
[2021-06-11T22:18:40.543Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-06-11T22:18:40.973Z] + gsutil --version
[2021-06-11T22:18:41.212Z] + gsutil -m -q cp -a public-read cGFja2V0YmVhdC13aW5kb3dzLTEwLXdpbmRvd3MtMTA5NmRiZTMwYTA2MTAyMzY2ZjRlMGYyNGMzOWY1ZGZjZmM3NTI3NDc2 gs://beats-ci-temp/ci/cache/
[2021-06-11T22:18:42.933Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-06-11T22:18:43.299Z] + gcloud auth activate-service-account --key-file ****
[2021-06-11T22:18:43.868Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-06-11T22:18:44.577Z] + gsutil -m -q cp -a public-read cGFja2V0YmVhdC13aW5kb3dzLTIwMTYtd2luZG93cy0yMDE2OTZkYmUzMGEwNjEwMjM2NmY0ZTBmMjRjMzlmNWRmY2ZjNzUyNzQ3Ng gs://beats-ci-temp/ci/cache/
[2021-06-11T22:19:18.804Z] module\system\test_system.py .....s..s...s.s.                            [ 96%]
[2021-06-11T22:19:18.804Z] module\traefik\test_traefik.py s                                         [ 97%]
[2021-06-11T22:19:18.804Z] module\uwsgi\test_uwsgi.py ss                                            [ 97%]
[2021-06-11T22:19:18.804Z] module\vsphere\test_vsphere.py sss                                       [ 99%]
[2021-06-11T22:19:18.804Z] module\zookeeper\test_zookeeper.py ss                                    [100%]
[2021-06-11T22:19:18.804Z] 
[2021-06-11T22:19:18.804Z] ============================== warnings summary ===============================
[2021-06-11T22:19:18.804Z] tests\system\test_lightmodules.py:57
[2021-06-11T22:19:18.804Z]   C:\Users\jenkins\workspace\PR-26266-1-27bc42cb-71f4-4fd8-8e11-c1124b7e1cc3\src\github.com\elastic\beats\metricbeat\tests\system\test_lightmodules.py:57: PytestCollectionWarning: cannot collect test class 'TestHTTPHandler' because it has a __init__ constructor (from: metricbeat/tests/system/test_lightmodules.py)
[2021-06-11T22:19:18.804Z]     class TestHTTPHandler(http.server.BaseHTTPRequestHandler):
[2021-06-11T22:19:18.804Z] 
[2021-06-11T22:19:18.804Z] -- Docs: https://docs.pytest.org/en/stable/warnings.html
[2021-06-11T22:19:18.804Z] - generated xml file: C:\Users\jenkins\workspace\PR-26266-1-27bc42cb-71f4-4fd8-8e11-c1124b7e1cc3\src\github.com\elastic\beats\metricbeat\build\TEST-python-unit.xml -
[2021-06-11T22:19:18.804Z] ============================ slowest 20 durations =============================
[2021-06-11T22:19:18.804Z] 16.92s call     metricbeat/tests/system/test_cmd.py::TestCommands::test_modules_list
[2021-06-11T22:19:18.805Z] 16.89s call     metricbeat/tests/system/test_cmd.py::TestCommands::test_modules_disable
[2021-06-11T22:19:18.805Z] 16.88s call     metricbeat/tests/system/test_cmd.py::TestCommands::test_modules_enable
[2021-06-11T22:19:18.805Z] 13.04s call     metricbeat/tests/system/test_config.py::ConfigTest::test_service_name
[2021-06-11T22:19:18.805Z] 12.56s call     metricbeat/tests/system/test_reload.py::Test::test_start_stop
[2021-06-11T22:19:18.805Z] 10.81s call     metricbeat/tests/system/test_cmd.py::TestCommands::test_modules_test_error
[2021-06-11T22:19:18.805Z] 10.20s call     metricbeat/module/system/test_system.py::Test::test_process
[2021-06-11T22:19:18.805Z] 9.80s call     metricbeat/tests/system/test_processors.py::Test::test_dropevent_with_condition
[2021-06-11T22:19:18.805Z] 9.57s call     metricbeat/tests/system/test_reload.py::Test::test_reload
[2021-06-11T22:19:18.805Z] 8.90s call     metricbeat/tests/system/test_base.py::Test::test_export_index_pattern_migration
[2021-06-11T22:19:18.805Z] 8.90s call     metricbeat/tests/system/test_timeseries.py::TestTimeseries::test_enable_timeseries
[2021-06-11T22:19:18.805Z] 8.87s call     metricbeat/module/http/test_http.py::Test::test_server
[2021-06-11T22:19:18.805Z] 8.84s call     metricbeat/tests/system/test_base.py::Test::test_export_index_pattern
[2021-06-11T22:19:18.805Z] 8.77s call     metricbeat/tests/system/test_base.py::Test::test_export_template
[2021-06-11T22:19:18.805Z] 8.77s call     metricbeat/module/system/test_system.py::Test::test_memory
[2021-06-11T22:19:18.805Z] 8.73s call     metricbeat/module/system/test_system.py::Test::test_cpu
[2021-06-11T22:19:18.805Z] 8.73s call     metricbeat/module/system/test_system.py::Test::test_process_summary
[2021-06-11T22:19:18.805Z] 8.73s call     metricbeat/module/system/test_system.py::Test::test_network
[2021-06-11T22:19:18.805Z] 8.73s call     metricbeat/module/system/test_system.py::Test::test_process_metricbeat
[2021-06-11T22:19:18.805Z] 8.72s call     metricbeat/module/system/test_system.py::Test::test_core_with_cpu_ticks
[2021-06-11T22:19:18.805Z] =========== 40 passed, 206 skipped, 1 warning in 397.22s (0:06:37) ============
[2021-06-11T22:19:18.805Z] >> python test: Unit Testing Complete
[2021-06-11T22:19:19.331Z] 
[2021-06-11T22:19:19.331Z] C:\Users\jenkins\workspace\PR-26266-1-27bc42cb-71f4-4fd8-8e11-c1124b7e1cc3\src\github.com\elastic\beats>FOR / %d IN ("ve") DO @IF EXIST "%d" rmdir /s /q "%d" 
[2021-06-11T22:19:22.196Z] 
[2021-06-11T22:19:22.197Z] C:\Users\jenkins\workspace\PR-26266-1-27bc42cb-71f4-4fd8-8e11-c1124b7e1cc3\src\github.com\elastic\beats>python .ci/scripts/pre_archive_test.py 
[2021-06-11T22:19:22.812Z] + gsutil --version
[2021-06-11T22:19:24.101Z] Copy .\metricbeat\build into build\metricbeat\build
[2021-06-11T22:19:24.101Z] Copy .\metricbeat\null\build into build\metricbeat\null\build
[2021-06-11T22:19:24.147Z] Running in C:\Users\jenkins\workspace\PR-26266-1-27bc42cb-71f4-4fd8-8e11-c1124b7e1cc3\src\github.com\elastic\beats\build
[2021-06-11T22:19:24.206Z] Recording test results
[2021-06-11T22:19:24.289Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-06-11T22:19:24.662Z] + gcloud auth activate-service-account --key-file ****
[2021-06-11T22:19:25.232Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-06-11T22:19:26.140Z] [Checks API] No suitable checks publisher found.
[2021-06-11T22:19:26.517Z] + gsutil -m -q cp -a public-read ZmlsZWJlYXQtd2luZG93cy03LTMyLXdpbmRvd3MtNy0zMi1iaXQ5NmRiZTMwYTA2MTAyMzY2ZjRlMGYyNGMzOWY1ZGZjZmM3NTI3NDc2 gs://beats-ci-temp/ci/cache/
[2021-06-11T22:19:26.794Z] 
[2021-06-11T22:19:26.794Z] C:\Users\jenkins\workspace\PR-26266-1-27bc42cb-71f4-4fd8-8e11-c1124b7e1cc3\src\github.com\elastic\beats>go clean -modcache 
[2021-06-11T22:19:30.608Z] + gsutil --version
[2021-06-11T22:19:32.054Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-06-11T22:19:32.457Z] + gcloud auth activate-service-account --key-file ****
[2021-06-11T22:19:33.031Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-06-11T22:19:33.721Z] + gsutil -m -q cp -a public-read cGFja2V0YmVhdC13aW5kb3dzLTctMzItd2luZG93cy03LTMyLWJpdDk2ZGJlMzBhMDYxMDIzNjZmNGUwZjI0YzM5ZjVkZmNmYzc1Mjc0NzY gs://beats-ci-temp/ci/cache/
[2021-06-11T22:20:50.227Z] + gsutil --version
[2021-06-11T22:20:51.662Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-06-11T22:20:52.028Z] + gcloud auth activate-service-account --key-file ****
[2021-06-11T22:20:52.600Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-06-11T22:20:53.323Z] + gsutil -m -q cp -a public-read bWV0cmljYmVhdC13aW5kb3dzLTEwLXdpbmRvd3MtMTA5NmRiZTMwYTA2MTAyMzY2ZjRlMGYyNGMzOWY1ZGZjZmM3NTI3NDc2 gs://beats-ci-temp/ci/cache/
[2021-06-11T22:20:57.570Z] Stage "Packaging" skipped due to earlier failure(s)
[2021-06-11T22:20:57.674Z] Stage "Packaging-Pipeline" skipped due to earlier failure(s)
[2021-06-11T22:20:57.815Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-26266/src/github.com/elastic/beats
[2021-06-11T22:20:59.062Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-26266
[2021-06-11T22:20:59.176Z] [INFO] getVaultSecret: Getting secrets
[2021-06-11T22:20:59.233Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-06-11T22:21:00.259Z] + chmod 755 generate-build-data.sh
[2021-06-11T22:21:00.259Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-26266/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-26266/runs/1 FAILURE 5636034
[2021-06-11T22:21:00.260Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-26266/runs/1/steps/?limit=10000 -o steps-info.json

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Test stats 🧪

Test Results
Failed 1
Passed 47463
Skipped 5207
Total 52671

Genuine test errors 1

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

  • Name: Extended / filebeat-windows-2016-windows-2016 / test_rotating_file – filebeat.tests.system.test_registrar.Test

@jalvz
Copy link
Contributor

jalvz commented Jun 14, 2021

jenkins retest this please

@jalvz
Copy link
Contributor

jalvz commented Jun 16, 2021

@urso same thing here, can you merge this for me, please?

@urso urso merged commit c67e0d4 into 7.x Jun 16, 2021
@urso urso deleted the mergify/bp/7.x/pr-25998 branch June 16, 2021 14:09
@sayden sayden mentioned this pull request Jul 1, 2021
Closed
@sayden sayden added the v7.14.0 label Jul 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@jalvz jalvz

Labels
backport needs_team Indicates that the issue/PR needs a Team:* label v7.14.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@elasticmachine @jalvz @urso @sayden