Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[system] Add routing pipeline to security data_stream #2523

Merged
merged 1 commit into from
Jan 13, 2022

Conversation

taylor-swanson
Copy link
Contributor

What does this PR do?

Added a routing pipeline to the security data_stream and limited the providers to Microsoft-Windows-Eventlog and Microsoft-Windows-Security-Auditing. Events that do not have these providers will still be indexed, but won't receive additional enrichment (which would likely lead to invalid data).

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

cd packages/system && elastic-package test
  • While I'm not sure how to generate logs from non-supported providers, I did test to make sure that existing logs from the supported providers (namely Microsoft-Windows-Security-Auditing ) are still being collected properly.

Related issues

…fic providers

- Added a routing pipeline to the security data_stream and limited the providers to
Microsoft-Windows-Eventlog and Microsoft-Windows-Security-Auditing. Events that do
not have these providers will still be indexed, but won't receive additional
enrichment (which would likely lead to invalid data).
@taylor-swanson taylor-swanson added bug Something isn't working, use only for issues Team:Security-External Integrations labels Jan 12, 2022
@taylor-swanson
Copy link
Contributor Author

Github isn't displaying the diff very well on this one, so here's another way of looking at it:

 4 files changed, 64 insertions(+), 3420 deletions(-)
 rewrite packages/system/data_stream/security/elasticsearch/ingest_pipeline/default.yml (98%)
 copy packages/system/data_stream/security/elasticsearch/ingest_pipeline/{default.yml => standard.yml} (99%)

I copied the original pipeline to standard.yml, it contains all of the specific enrichment processors for the providers I mentioned. Anything generic was moved/kept in default.yml, along with the routing logic. I changed very few lines in the end, despite what the stats say.

@elasticmachine
Copy link

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-01-12T14:50:09.677+0000

  • Duration: 15 min 0 sec

  • Commit: 9c6a862

Test stats 🧪

Test Results
Failed 0
Passed 269
Skipped 0
Total 269

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@taylor-swanson taylor-swanson marked this pull request as ready for review January 12, 2022 15:05
@elasticmachine
Copy link

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@taylor-swanson taylor-swanson merged commit 175d4ff into elastic:master Jan 13, 2022
@taylor-swanson taylor-swanson deleted the win-eventid-reuse branch January 13, 2022 14:58
eyalkraft pushed a commit to build-security/integrations that referenced this pull request Mar 30, 2022
…fic providers (elastic#2523)

- Added a routing pipeline to the security data_stream and limited the providers to
Microsoft-Windows-Eventlog and Microsoft-Windows-Security-Auditing. Events that do
not have these providers will still be indexed, but won't receive additional
enrichment (which would likely lead to invalid data).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working, use only for issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants