Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Filebeat] fix m365_defender pipeline bug #31227

Merged
merged 1 commit into from
Apr 11, 2022
Merged

[Filebeat] fix m365_defender pipeline bug #31227

merged 1 commit into from
Apr 11, 2022

Conversation

leehinman
Copy link
Contributor

Closes 31223

What does this PR do?

In microsoft m365_defender fileset remove alerts.entities when it is an empty list. This prevents errors
where alerts.entities is assumed to be a Map.

Why is it important?

users reported events not ingesting that had alers.entities as empty lists.

Checklist

  • My code follows the style guidelines of this project
    - [ ] I have commented my code, particularly in hard-to-understand areas
    - [ ] I have made corresponding changes to the documentation
    - [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

TESTING_FILEBEAT_MODULES=microsoft TESTING_FILEBEAT_FILESETS=m365_defender mage pythonIntegTest

Related issues

@leehinman leehinman added bug Filebeat Filebeat Team:Security-External Integrations backport-v8.3.0 Automated backport with mergify backport-v8.2.0 Automated backport with mergify backport-7.17 Automated backport to the 7.17 branch with mergify labels Apr 7, 2022
@leehinman leehinman requested review from a team as code owners April 7, 2022 21:38
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Apr 7, 2022
@elasticmachine
Copy link
Collaborator

elasticmachine commented Apr 7, 2022 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-04-11T16:55:57.314+0000

  • Duration: 67 min 4 sec

Test stats 🧪

Test Results
Failed 0
Passed 2064
Skipped 159
Total 2223

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@mergify
Copy link
Contributor

mergify bot commented Apr 11, 2022

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 31223_m365_defender_empty_list upstream/31223_m365_defender_empty_list
git merge upstream/main
git push upstream 31223_m365_defender_empty_list

@leehinman
[Filebeat] fix m365_defender pipeline bug
f36e586 
- remove `alerts.entities` when it is an empty list.  Prevents errors
where `alerts.entities` is assumed to be a Map.

Closes 31223
@leehinman leehinman force-pushed the 31223_m365_defender_empty_list branch from 64bbbab to f36e586 Compare April 11, 2022 16:55
@leehinman leehinman merged commit 60fcf9d into elastic:main Apr 11, 2022
mergify bot pushed a commit that referenced this pull request Apr 11, 2022
@leehinman
[Filebeat] fix m365_defender pipeline bug (#31227)
0ca4ed7 
- remove `alerts.entities` when it is an empty list.  Prevents errors
where `alerts.entities` is assumed to be a Map.

Closes #31223

(cherry picked from commit 60fcf9d)

# Conflicts:
#	x-pack/filebeat/module/microsoft/m365_defender/test/m365_defender-test.ndjson.log-expected.json
@mergify mergify bot mentioned this pull request Apr 11, 2022
Merged
mergify bot pushed a commit that referenced this pull request Apr 11, 2022
@leehinman
[Filebeat] fix m365_defender pipeline bug (#31227)
739608f 
- remove `alerts.entities` when it is an empty list.  Prevents errors
where `alerts.entities` is assumed to be a Map.

Closes #31223

(cherry picked from commit 60fcf9d)
@mergify mergify bot mentioned this pull request Apr 11, 2022
Merged
leehinman added a commit that referenced this pull request Apr 12, 2022
@leehinman
[Filebeat] fix m365_defender pipeline bug (#31227)
cf2c984 
- remove `alerts.entities` when it is an empty list.  Prevents errors
where `alerts.entities` is assumed to be a Map.

Closes #31223

(cherry picked from commit 60fcf9d)
leehinman added a commit that referenced this pull request Apr 12, 2022
@mergify @leehinman
[Filebeat] fix m365_defender pipeline bug (#31227) (#31253)
886108a 
- remove `alerts.entities` when it is an empty list.  Prevents errors
where `alerts.entities` is assumed to be a Map.

Closes #31223

(cherry picked from commit 60fcf9d)

Co-authored-by: Lee E Hinman <57081003+leehinman@users.noreply.github.com>
@leehinman leehinman deleted the 31223_m365_defender_empty_list branch April 12, 2022 17:50
leehinman added a commit that referenced this pull request Apr 12, 2022
@leehinman
[Filebeat] fix m365_defender pipeline bug (#31227)
a19eafa 
- remove `alerts.entities` when it is an empty list.  Prevents errors
where `alerts.entities` is assumed to be a Map.

Closes #31223

(cherry picked from commit 60fcf9d)
leehinman added a commit that referenced this pull request Apr 13, 2022
@mergify @leehinman
[Filebeat] fix m365_defender pipeline bug (#31227) (#31252)
9526317 
- remove `alerts.entities` when it is an empty list.  Prevents errors
where `alerts.entities` is assumed to be a Map.

Closes #31223

(cherry picked from commit 60fcf9d)

Co-authored-by: Lee E Hinman <57081003+leehinman@users.noreply.github.com>
v1v added a commit to v1v/beats that referenced this pull request Apr 18, 2022
@v1v
Merge remote-tracking branch 'upstream/main' into feature/rename-dock…
eda0443 
…er-tar-gz

* upstream/main: (139 commits)
  [Automation] Update elastic stack version to 8.3.0-c655cda8 for testing (elastic#31322)
  Define a queue metrics reporter interface  (elastic#31289)
  [Oracle Module] Change tablespace metricset collection period (elastic#31259)
  libbeat/reader/syslog: relax timestamp parsing to allow leading zero (elastic#31254)
  [Automation] Update elastic stack version to 8.3.0-55ba6f37 for testing (elastic#31311)
  [libbeat] Remove unused fields and functions in the memory queue (elastic#31302)
  [libbeat] Cleaning up some unneeded helper types (elastic#31290)
  Readme for kibana module (elastic#31276)
  [Automation] Update elastic stack version to 8.3.0-4be61f32 for testing (elastic#31296)
  x-pack/winlogbeat/module/routing/ingest: fix typo for channel name (elastic#31291)
  Small pipeline cleanup removing some unused data fields (elastic#31288)
  removing info log (elastic#30971)
  Simplify TLS config deserialization (elastic#31168)
  Detect new files under known paths in filestream input (elastic#31268)
  Add support for port mapping in docker hints (elastic#31243)
  Update qa-labels.yml (elastic#31260)
  libbeat: log debug for `proxy_url` and fixed docs (elastic#31130)
  [heartbeat][docs] Add note about ensuring correct index settings for uptime (elastic#31146)
  [Automation] Update elastic stack version to 8.3.0-2c8f9574 for testing (elastic#31256)
  [Filebeat] fix m365_defender pipeline bug (elastic#31227)
  ...
kush-elastic pushed a commit to kush-elastic/beats that referenced this pull request May 2, 2022
@leehinman @kush-elastic
[Filebeat] fix m365_defender pipeline bug (elastic#31227)
4815785 
- remove `alerts.entities` when it is an empty list.  Prevents errors
where `alerts.entities` is assumed to be a Map.

Closes elastic#31223
chrisberkhout pushed a commit that referenced this pull request Jun 1, 2023
@leehinman @chrisberkhout
[Filebeat] fix m365_defender pipeline bug (#31227)
4cc5523 
- remove `alerts.entities` when it is an empty list.  Prevents errors
where `alerts.entities` is assumed to be a Map.

Closes #31223
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@efd6 efd6 efd6 approved these changes

Assignees

@leehinman leehinman

Labels
backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.2.0 Automated backport with mergify backport-v8.3.0 Automated backport with mergify bug Filebeat Filebeat
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Filebeat][microsoft][m365_defender] Error processing when alerts.entities is an empty list
3 participants
@leehinman @elasticmachine @efd6