Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] Add field limit check for AWS Cloudtrail flattened fields #21388

Merged
merged 1 commit into from
Sep 30, 2020

Conversation

leehinman
Copy link
Contributor

What does this PR do?

Adds a 32k length check for:

  • aws.cloudtrail.flattened.request_parameters
  • aws.cloudtrail.flattened.response_elements
  • aws.cloudtrail.flattened.additional_eventdata
  • aws.cloudtrail.flattened.service_event_details

Why is it important?

Elasticsearch will fail to index the document if a flattened field is
over 32k in length.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

TESTING_FILEBEAT_MODULES=aws TESTING_FILEBEAT_FILESETS=cloudtrail mage -v pythonIntegTest

Related issues

@leehinman leehinman added bug Filebeat Filebeat needs_backport PR is waiting to be backported to other branches. Team:SIEM labels Sep 29, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Sep 29, 2020
add 32k length check for
  - aws.cloudtrail.flattened.request_parameters
  - aws.cloudtrail.flattened.response_elements
  - aws.cloudtrail.flattened.additional_eventdata
  - aws.cloudtrail.flattened.service_event_details

Closes elastic#21382
@elasticmachine
Copy link
Collaborator

💔 Tests Failed

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #21388 opened]

  • Start Time: 2020-09-29T15:05:26.010+0000

  • Duration: 109 min 6 sec

Test stats 🧪

Test Results
Failed 2
Passed 20193
Skipped 1855
Total 22050

Test errors

Expand to view the tests failures

  • Name: Build&Test / libbeat-build / TestReplaceClientWorker – pipeline

    • Age: 1
    • Duration: 25.87
    • Error Details: Failed
  • Name: Build&Test / libbeat-build / TestReplaceClientWorker/network_client – pipeline

    • Age: 1
    • Duration: 22.39
    • Error Details: Failed

Steps errors

Expand to view the steps failures

  • Name: mage build test

    • Description: mage build test

    • Duration: 21 min 30 sec

    • Start Time: 2020-09-29T15:33:50.614+0000

    • log

  • Name: Notifies GitHub of the status of a Pull Request

    • Description: script returned exit code 1

    • Duration: 0 min 1 sec

    • Start Time: 2020-09-29T15:54:41.119+0000

    • log

Log output

Expand to view the last 100 lines of log output

[2020-09-29T16:53:16.537Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats
[2020-09-29T16:53:16.576Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/uncategorized-1601393367166
[2020-09-29T16:53:16.678Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/libbeat-stress-tests-1601393636873
[2020-09-29T16:53:16.785Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/winlogbeat-crosscompile-1601393708793
[2020-09-29T16:53:16.891Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-elastic-agent-build-1601393722018
[2020-09-29T16:53:16.994Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/auditbeat-crosscompile-1601393751343
[2020-09-29T16:53:17.103Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/journalbeat-unitTest-1601393751892
[2020-09-29T16:53:17.206Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-dockerlogbeat-build-1601393773076
[2020-09-29T16:53:17.310Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-functionbeat-build-1601393831144
[2020-09-29T16:53:17.406Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/libbeat-crosscompile-1601393899307
[2020-09-29T16:53:17.508Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/packetbeat-build-1601393933189
[2020-09-29T16:53:17.614Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/metricbeat-unitTest-1601393995984
[2020-09-29T16:53:17.717Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-elastic-agent-windows-windows-2019-1601394037405
[2020-09-29T16:53:17.816Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/auditbeat-windows-windows-2019-1601394070138
[2020-09-29T16:53:17.909Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/heartbeat-build-1601394083476
[2020-09-29T16:53:18.002Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-auditbeat-windows-windows-2019-1601394132401
[2020-09-29T16:53:18.102Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-auditbeat-build-1601394138507
[2020-09-29T16:53:18.207Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/auditbeat-build-1601394155668
[2020-09-29T16:53:18.311Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/heartbeat-windows-windows-2019-1601394161177
[2020-09-29T16:53:18.408Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/packetbeat-windows-windows-2019-1601394190771
[2020-09-29T16:53:18.504Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-winlogbeat-build-windows-2019-1601394200842
[2020-09-29T16:53:18.602Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/winlogbeat-windows-windows-2019-1601394220803
[2020-09-29T16:53:18.699Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-filebeat-windows-windows-2019-1601394228115
[2020-09-29T16:53:18.793Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/generator-metricbeat-test-1601394278622
[2020-09-29T16:53:18.887Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/generator-beat-test-1601394317447
[2020-09-29T16:53:18.983Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/metricbeat-crosscompile-1601394319499
[2020-09-29T16:53:19.075Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-functionbeat-windows-windows-2019-1601394320250
[2020-09-29T16:53:19.172Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-libbeat-build-1601394327730
[2020-09-29T16:53:19.267Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/filebeat-windows-windows-2019-1601394337019
[2020-09-29T16:53:19.364Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/metricbeat-windows-windows-2019-1601394431386
[2020-09-29T16:53:19.466Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-metricbeat-windows-windows-2019-1601394518405
[2020-09-29T16:53:19.564Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/libbeat-build-1601394870716
[2020-09-29T16:53:19.670Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/filebeat-build-1601395081443
[2020-09-29T16:53:19.772Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/metricbeat-goIntegTest-1601395547047
[2020-09-29T16:53:19.871Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-filebeat-build-1601395561197
[2020-09-29T16:53:19.964Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/metricbeat-pythonIntegTest-1601395869824
[2020-09-29T16:53:20.058Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-metricbeat-build-1601397004311
[2020-09-29T16:53:20.158Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/auditbeat-macos-macosx-1601397045229
[2020-09-29T16:53:20.260Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/generator-macos-beat-macosx-1601397218625
[2020-09-29T16:53:20.353Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/metricbeat-macos-macosx-1601397367467
[2020-09-29T16:53:20.458Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-elastic-agent-macos-macosx-1601397371560
[2020-09-29T16:53:20.561Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/packetbeat-macos-macosx-1601397386860
[2020-09-29T16:53:20.658Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-auditbeat-macos-macosx-1601397401876
[2020-09-29T16:53:20.750Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-filebeat-macos-macosx-1601397612699
[2020-09-29T16:53:20.845Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-functionbeat-macos-macosx-1601397616350
[2020-09-29T16:53:20.946Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-metricbeat-macos-macosx-1601397696101
[2020-09-29T16:53:21.050Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/heartbeat-macos-macosx-1601397724160
[2020-09-29T16:53:21.155Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/generator-macos-metricbeat-macosx-1601397735302
[2020-09-29T16:53:21.270Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/filebeat-macos-macosx-1601398315706
[2020-09-29T16:53:21.712Z] + cat
[2020-09-29T16:53:21.712Z] + /usr/local/bin/runbld ./runbld-test-reports --job-name elastic+beats+pull-request
[2020-09-29T16:53:21.712Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-09-29T16:53:28.317Z] runbld>>> runbld started
[2020-09-29T16:53:28.317Z] runbld>>> 1.6.12/f45d832f2ba0aa2722ab4ec1fda8ad140f027f8b
[2020-09-29T16:53:30.254Z] runbld>>> The following profiles matched the job 'elastic+beats+pull-request' in order of occurrence in the config (last value wins).
[2020-09-29T16:53:30.254Z] runbld>>> Matches in the system config:
[2020-09-29T16:53:30.254Z] runbld>>> - Matched ^elastic\+beats
[2020-09-29T16:53:30.254Z] runbld>>> - Matched ^elastic\+beats\+pull-request
[2020-09-29T16:53:31.780Z] runbld>>> Debug logging enabled.
[2020-09-29T16:53:31.780Z] runbld>>> Storing result
[2020-09-29T16:53:31.780Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-09-29T16:53:31.780Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1597739501209/t/20200929165331-109F66D5
[2020-09-29T16:53:31.780Z] runbld>>> Adding system facts.
[2020-09-29T16:53:32.735Z] runbld>>> Adding vcs info for the latest commit:  2c7c944418d053e80aca0428ea786849e2973c86
[2020-09-29T16:53:32.735Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-09-29T16:53:32.735Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-09-29T16:53:32.996Z] Processing JUnit reports with runbld...
[2020-09-29T16:53:32.996Z] + echo 'Processing JUnit reports with runbld...'
[2020-09-29T16:53:33.258Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-09-29T16:53:33.258Z] runbld>>> DURATION: 32ms
[2020-09-29T16:53:33.258Z] runbld>>> STDOUT: 40 bytes
[2020-09-29T16:53:33.258Z] runbld>>> STDERR: 49 bytes
[2020-09-29T16:53:33.258Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-09-29T16:53:33.258Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats_PR-21388
[2020-09-29T16:53:34.208Z] runbld>>> Storing build metadata: 
[2020-09-29T16:53:34.208Z] runbld>>> Adding test report.
[2020-09-29T16:53:34.208Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats
[2020-09-29T16:53:35.154Z] runbld>>> Found 141 test output files
[2020-09-29T16:54:31.477Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-metricbeat-build-1601397004311/x-pack/metricbeat/build/TEST-go-integration-openmetrics.xml
[2020-09-29T16:54:31.477Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-metricbeat-build-1601397004311/x-pack/metricbeat/build/TEST-go-integration-istio.xml
[2020-09-29T16:54:31.477Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-metricbeat-build-1601397004311/x-pack/metricbeat/build/TEST-go-integration-activemq.xml
[2020-09-29T16:54:31.478Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-metricbeat-build-1601397004311/x-pack/metricbeat/build/TEST-go-integration-iis.xml
[2020-09-29T16:54:31.478Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/x-pack-metricbeat-build-1601397004311/x-pack/metricbeat/build/TEST-go-integration-tomcat.xml
[2020-09-29T16:54:31.478Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/metricbeat-goIntegTest-1601395547047/metricbeat/build/TEST-go-integration-graphite.xml
[2020-09-29T16:54:31.478Z] runbld>>> No testsuite node found in /var/lib/jenkins/workspace/Beats_beats_PR-21388/src/github.com/elastic/beats/metricbeat-goIntegTest-1601395547047/metricbeat/build/TEST-go-integration-windows.xml
[2020-09-29T16:54:31.478Z] runbld>>> Test output logs contained: Errors: 0 Failures: 2 Tests: 22050 Skipped: 1561
[2020-09-29T16:54:31.478Z] runbld>>> Storing result
[2020-09-29T16:54:31.478Z] runbld>>> FAILURES: 2
[2020-09-29T16:54:31.478Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-09-29T16:54:31.478Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1597739501209/t/20200929165331-109F66D5
[2020-09-29T16:54:31.478Z] runbld>>> Email notification disabled by environment variable.
[2020-09-29T16:54:31.478Z] runbld>>> Slack notification disabled by environment variable.
[2020-09-29T16:54:31.674Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-21388
[2020-09-29T16:54:31.767Z] [INFO] getVaultSecret: Getting secrets
[2020-09-29T16:54:31.852Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-09-29T16:54:32.675Z] + chmod 755 generate-build-data.sh
[2020-09-29T16:54:32.675Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-21388/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-21388/runs/1 FAILURE 6546392
[2020-09-29T16:54:32.675Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-21388/runs/1/steps/?limit=10000 -o steps-info.json
[2020-09-29T16:54:36.798Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-21388/runs/1/tests/?status=FAILED -o tests-errors.json
[2020-09-29T16:54:37.048Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-21388/runs/1/log/ -o pipeline-log.txt

@leehinman leehinman merged commit eae9f5c into elastic:master Sep 30, 2020
leehinman added a commit to leehinman/beats that referenced this pull request Sep 30, 2020
)

add 32k length check for
  - aws.cloudtrail.flattened.request_parameters
  - aws.cloudtrail.flattened.response_elements
  - aws.cloudtrail.flattened.additional_eventdata
  - aws.cloudtrail.flattened.service_event_details

Closes elastic#21382

(cherry picked from commit eae9f5c)
@leehinman leehinman added v7.10.0 and removed needs_backport PR is waiting to be backported to other branches. labels Sep 30, 2020
leehinman added a commit to leehinman/beats that referenced this pull request Sep 30, 2020
)

add 32k length check for
  - aws.cloudtrail.flattened.request_parameters
  - aws.cloudtrail.flattened.response_elements
  - aws.cloudtrail.flattened.additional_eventdata
  - aws.cloudtrail.flattened.service_event_details

Closes elastic#21382

(cherry picked from commit eae9f5c)
leehinman added a commit that referenced this pull request Sep 30, 2020
…21432)

add 32k length check for
  - aws.cloudtrail.flattened.request_parameters
  - aws.cloudtrail.flattened.response_elements
  - aws.cloudtrail.flattened.additional_eventdata
  - aws.cloudtrail.flattened.service_event_details

Closes #21382

(cherry picked from commit eae9f5c)
leehinman added a commit that referenced this pull request Sep 30, 2020
…21431)

add 32k length check for
  - aws.cloudtrail.flattened.request_parameters
  - aws.cloudtrail.flattened.response_elements
  - aws.cloudtrail.flattened.additional_eventdata
  - aws.cloudtrail.flattened.service_event_details

Closes #21382

(cherry picked from commit eae9f5c)
v1v added a commit to v1v/beats that referenced this pull request Oct 2, 2020
* upstream/master: (27 commits)
  [Ingest Manager] Split index restrictions into type,dataset, namespace parts (elastic#21406)
  Update Filebeat module expected logs files (elastic#21454)
  Edit SQL module docs and fix broken doc structure (elastic#21233)
  [Ingest Manager] Send snapshot flag together with metadata (elastic#21285)
  Revert "[JJBB] Set shallow cloning to 10 (elastic#21409)" (elastic#21447)
  [JJBB] Use reference repo for fast checkouts (elastic#21410)
  Add initial skeleton of filestream input (elastic#21427)
  Initial spec file for apm-server (elastic#21225)
  [Ingest Manager] Upgrade Action: make source URI optional (elastic#21372)
  Add field limit check for AWS Cloudtrail flattened fields (elastic#21388)
  [Winlogbeat] Move winlogbeat javascript processor to libbeat (elastic#21402)
  ci: pipeline to generate the changelog (elastic#21426)
  [JJBB] Set shallow cloning to 10 (elastic#21409)
  docs: add link to release notes for 7.9.2 (elastic#21405) (elastic#21419)
  docs: Prepare Changelog for 7.9.2 (elastic#21229) (elastic#21403)
  fix: mark flaky tests (elastic#21300)
  fix: use a fixed version of setuptools (elastic#21393)
  Move Kubernetes events metricset to its own block in reference config (elastic#21407)
  [libbeat] Enable WriteAheadLimit in the disk queue (elastic#21391)
  docs: fix apt/yum formatting (elastic#21362)
  ...
v1v added a commit to v1v/beats that referenced this pull request Oct 2, 2020
…ne-2.0-arm

* upstream/master: (54 commits)
  [CI] Change x-pack/auditbeat build events (comments, labels) (elastic#21463)
  [CI] changeset from elastic#20603 was not added to CI2.0 (elastic#21464)
  Add new log file reader for filestream input (elastic#21450)
  [CI] Send slack message with build status (elastic#21428)
  Remove duplicated sources url in dependencies report (elastic#21462)
  Add implementation of FSWatcher and FSScanner for filestream (elastic#21444)
  [Ingest Manager] Split index restrictions into type,dataset, namespace parts (elastic#21406)
  Update Filebeat module expected logs files (elastic#21454)
  Edit SQL module docs and fix broken doc structure (elastic#21233)
  [Ingest Manager] Send snapshot flag together with metadata (elastic#21285)
  Revert "[JJBB] Set shallow cloning to 10 (elastic#21409)" (elastic#21447)
  [JJBB] Use reference repo for fast checkouts (elastic#21410)
  Add initial skeleton of filestream input (elastic#21427)
  Initial spec file for apm-server (elastic#21225)
  [Ingest Manager] Upgrade Action: make source URI optional (elastic#21372)
  Add field limit check for AWS Cloudtrail flattened fields (elastic#21388)
  [Winlogbeat] Move winlogbeat javascript processor to libbeat (elastic#21402)
  ci: pipeline to generate the changelog (elastic#21426)
  [JJBB] Set shallow cloning to 10 (elastic#21409)
  docs: add link to release notes for 7.9.2 (elastic#21405) (elastic#21419)
  ...
v1v added a commit to v1v/beats that referenced this pull request Oct 2, 2020
…ci-build-label-support

* upstream/master:
  [CI] Change x-pack/auditbeat build events (comments, labels) (elastic#21463)
  [CI] changeset from elastic#20603 was not added to CI2.0 (elastic#21464)
  Add new log file reader for filestream input (elastic#21450)
  [CI] Send slack message with build status (elastic#21428)
  Remove duplicated sources url in dependencies report (elastic#21462)
  Add implementation of FSWatcher and FSScanner for filestream (elastic#21444)
  [Ingest Manager] Split index restrictions into type,dataset, namespace parts (elastic#21406)
  Update Filebeat module expected logs files (elastic#21454)
  Edit SQL module docs and fix broken doc structure (elastic#21233)
  [Ingest Manager] Send snapshot flag together with metadata (elastic#21285)
  Revert "[JJBB] Set shallow cloning to 10 (elastic#21409)" (elastic#21447)
  [JJBB] Use reference repo for fast checkouts (elastic#21410)
  Add initial skeleton of filestream input (elastic#21427)
  Initial spec file for apm-server (elastic#21225)
  [Ingest Manager] Upgrade Action: make source URI optional (elastic#21372)
  Add field limit check for AWS Cloudtrail flattened fields (elastic#21388)
  [Winlogbeat] Move winlogbeat javascript processor to libbeat (elastic#21402)
  ci: pipeline to generate the changelog (elastic#21426)
@leehinman leehinman deleted the 21382_cloudtrail_flattened branch October 5, 2020 19:14
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
) (elastic#21432)

add 32k length check for
  - aws.cloudtrail.flattened.request_parameters
  - aws.cloudtrail.flattened.response_elements
  - aws.cloudtrail.flattened.additional_eventdata
  - aws.cloudtrail.flattened.service_event_details

Closes elastic#21382

(cherry picked from commit bfed554)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[filebeat][aws][cloudtrail] flattened.request_parameters field can exceed 32k limit
3 participants