Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add system and pipeline tests for Suricata EVE #457

Merged
merged 3 commits into from
Dec 14, 2020

Conversation

andrewkroh
Copy link
Member

What does this PR do?

This adds tests and update the Suricata pipeline.

  • Sync the pipeline from beats e9d12e2119ff58.
  • Convert suricata.eve.flow_id to string because the field is a keyword in the mapping.
  • Add missing ECS field definitions.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all datasets collect metrics or logs.

Related issues

This adds tests and update the Suricata pipeline.

- Sync the pipeline from beats e9d12e2119ff58.
- Convert suricata.eve.flow_id to string because the field is a keyword in the mapping.
- Add missing ECS field definitions.
@elasticmachine
Copy link

elasticmachine commented Dec 10, 2020

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #457 updated

  • Start Time: 2020-12-14T19:16:44.149+0000

  • Duration: 29 min 43 sec

Test stats 🧪

Test Results
Failed 0
Passed 136
Skipped 0
Total 136

@elasticmachine
Copy link

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@@ -0,0 +1,173 @@
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

each of these appear to only have 3 events in the output, that seems off to me given that the number of events/flows in each of the logs is more on the order of 8+?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the input for the pipeline test. These three events are run through the ES Ingest Node pipeline. Then the output of the pipeline is checked against the test-events.json-expected.json file.

The other log files in the deploy directory are for the e2e tests. Those don't have golden files. Those tests only check that there were no errors and that all fields are documented.

@andrewkroh andrewkroh merged commit e0b7cfa into elastic:master Dec 14, 2020
andrewkroh added a commit to andrewkroh/beats that referenced this pull request Jan 12, 2021
Convert suricata.eve.flow_id to string because the field is a keyword in the mapping.

elastic/integrations#457
andrewkroh added a commit to elastic/beats that referenced this pull request Jan 25, 2021
* Sync changes to AWS CloudTrail

elastic/integrations#408

* Sync changes to CheckPoint Firewall

Change type of event.severity.

elastic/integrations#409

* Sync changes from Cisco ASA / FTD

elastic/integrations#414

* Sync changes from Cisco IOS

Make icmp and igmp fields strings because they are keywords.

elastic/integrations#416

* Sync changes to CrowdStrike Falcon

Fix some field types.

elastic/integrations#377

* Sync changes to Fortinet Firewall

Drop assignip if the value is "N/A".

elastic/integrations#437

* Sync changes to Juniper SRX

Convert event.risk values to float
Protect against missing event.timezone
Convert event.severity to long.

elastic/integrations#443

* Sync changes to Suricata EVE

Convert suricata.eve.flow_id to string because the field is a keyword in the mapping.

elastic/integrations#457

* Sync changes to Zeek DNS

Fix usages of ignore_failure with convert processor.
Make DNS transaction ID a string.

elastic/integrations#448

* Add changelog
andrewkroh added a commit to andrewkroh/beats that referenced this pull request Feb 16, 2021
* Sync changes to AWS CloudTrail

elastic/integrations#408

* Sync changes to CheckPoint Firewall

Change type of event.severity.

elastic/integrations#409

* Sync changes from Cisco ASA / FTD

elastic/integrations#414

* Sync changes from Cisco IOS

Make icmp and igmp fields strings because they are keywords.

elastic/integrations#416

* Sync changes to CrowdStrike Falcon

Fix some field types.

elastic/integrations#377

* Sync changes to Fortinet Firewall

Drop assignip if the value is "N/A".

elastic/integrations#437

* Sync changes to Juniper SRX

Convert event.risk values to float
Protect against missing event.timezone
Convert event.severity to long.

elastic/integrations#443

* Sync changes to Suricata EVE

Convert suricata.eve.flow_id to string because the field is a keyword in the mapping.

elastic/integrations#457

* Sync changes to Zeek DNS

Fix usages of ignore_failure with convert processor.
Make DNS transaction ID a string.

elastic/integrations#448

* Add changelog

(cherry picked from commit bf46572)
adriansr pushed a commit to elastic/beats that referenced this pull request Feb 17, 2021
#24077)

* Sync fixes from Integration Package Testing (#23424)

* Sync changes to AWS CloudTrail

elastic/integrations#408

* Sync changes to CheckPoint Firewall

Change type of event.severity.

elastic/integrations#409

* Sync changes from Cisco ASA / FTD

elastic/integrations#414

* Sync changes from Cisco IOS

Make icmp and igmp fields strings because they are keywords.

elastic/integrations#416

* Sync changes to CrowdStrike Falcon

Fix some field types.

elastic/integrations#377

* Sync changes to Fortinet Firewall

Drop assignip if the value is "N/A".

elastic/integrations#437

* Sync changes to Juniper SRX

Convert event.risk values to float
Protect against missing event.timezone
Convert event.severity to long.

elastic/integrations#443

* Sync changes to Suricata EVE

Convert suricata.eve.flow_id to string because the field is a keyword in the mapping.

elastic/integrations#457

* Sync changes to Zeek DNS

Fix usages of ignore_failure with convert processor.
Make DNS transaction ID a string.

elastic/integrations#448

* Add changelog

(cherry picked from commit bf46572)
eyalkraft pushed a commit to build-security/integrations that referenced this pull request Mar 30, 2022
This adds tests and update the Suricata pipeline.

- Sync the pipeline from beats e9d12e2119ff58.
- Convert suricata.eve.flow_id to string because the field is a keyword in the mapping.
- Add missing ECS field definitions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants