Test export commands in all Beats #20016
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
botelastic
bot
added
the
needs_team
botelastic
bot
removed
the
needs_team
|
Pinging @elastic/siem (Team:SIEM) |
adriansr
added
review
and removed
in progress
💔 Tests Failed
Expand to view the summary
Build stats
Test stats 🧪
Test errorsExpand to view the tests failures
Steps errorsExpand to view the steps failures
Log outputExpand to view the last 100 lines of log output
|
|
Pinging @elastic/integrations (Team:Integrations) |
exekias
approved these changes
Jul 20, 2020
|
Merged with unrelated CI errors |
v1v
added a commit
to v1v/beats
that referenced
this pull request
Jul 20, 2020
…ngs-archive * upstream/master: (274 commits) Test export commands in all Beats (elastic#20016) [Ingest Manager] Allow using drop path for agent build (elastic#20019) [DOCS] Remove include for deleted file in monitoring docs (elastic#20038) Removing xpack.monitoring.* settings (elastic#18608) [Heartbeat] Add service_name option for APM integration (elastic#19932) [Elastic Agent] Fix merging of fleet.yml. Add --staging to enroll cmd. (elastic#20026) Stricter stalebot configuration for PRs (elastic#20004) [ci] Favor direct mage invocation on CI (elastic#19960) Add cloudwatch input into Filebeat configure inputs documentation (elastic#19973) [Filebeat] remove delimiter \n from log line in s3 input (elastic#19972) [Metricbeat] Update MySQL dashboard (elastic#19913) Packetbeat process monitor: Ignore missing /proc/net/tcp6 (elastic#19945) [CI] fix MODULE variable cornercases (elastic#19985) Ignore timestamp in fortinet/clientendpoint and netscout/sightline (elastic#19998) add 7.9 to .backportrc.json (elastic#19952) Update internal links to external (elastic#19947) Remove Dynamic Script Compilations warning in Cisco module (elastic#19840) [Elastic Agent] Fix RPM and DEB packaging for Elastic Agent (elastic#19959) [Ingest Manager] Do not compare err with custom type (elastic#19980) Fix nanocore sum for non default intervals on Kubernetes Overview Dashboard (elastic#19675) ...
v1v
added a commit
to v1v/beats
that referenced
this pull request
Jul 20, 2020
…nly-mod * upstream/master: (75 commits) Test export commands in all Beats (elastic#20016) [Ingest Manager] Allow using drop path for agent build (elastic#20019) [DOCS] Remove include for deleted file in monitoring docs (elastic#20038) Removing xpack.monitoring.* settings (elastic#18608) [Heartbeat] Add service_name option for APM integration (elastic#19932) [Elastic Agent] Fix merging of fleet.yml. Add --staging to enroll cmd. (elastic#20026) Stricter stalebot configuration for PRs (elastic#20004) [ci] Favor direct mage invocation on CI (elastic#19960) Add cloudwatch input into Filebeat configure inputs documentation (elastic#19973) [Filebeat] remove delimiter \n from log line in s3 input (elastic#19972) [Metricbeat] Update MySQL dashboard (elastic#19913) Packetbeat process monitor: Ignore missing /proc/net/tcp6 (elastic#19945) [CI] fix MODULE variable cornercases (elastic#19985) Ignore timestamp in fortinet/clientendpoint and netscout/sightline (elastic#19998) add 7.9 to .backportrc.json (elastic#19952) Update internal links to external (elastic#19947) Remove Dynamic Script Compilations warning in Cisco module (elastic#19840) [Elastic Agent] Fix RPM and DEB packaging for Elastic Agent (elastic#19959) [Ingest Manager] Do not compare err with custom type (elastic#19980) Fix nanocore sum for non default intervals on Kubernetes Overview Dashboard (elastic#19675) ...
Merged
3 tasks
adriansr
added a commit
to adriansr/beats
that referenced
this pull request
Aug 14, 2020
This adds 4 new integration tests to all Beats that test export commands: > beatname export ilm-policy > beatname export template > beatname export index-pattern > beatname export config
adriansr
added a commit
that referenced
this pull request
Aug 17, 2020
…igration.6_to_7.enabled (#20574) * Test export commands in all Beats (#20016) This adds 4 new integration tests to all Beats that test export commands: > beatname export ilm-policy > beatname export template > beatname export index-pattern > beatname export config * [Filebeat] Add export tests to x-pack/filebeat (#20156) * Add export tests to x-pack/filebeat Add export sub-command tests to x-pack/filebeat and add an assertion for the size the Kibana index-pattern due to the Kibana API limiting payloads to 1 MiB. * Assert size of index pattern document is less than 1 MiB * Fix Winlogbeat export index-pattern with migration.6_to_7.enabled (#20540) - Test export index-pattern with migration.6_to_7.enabled=true - Always run these tests, not only when integration testing - Remove conflicting Winlogbeat alias * Changelog entry Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
adriansr
added a commit
to adriansr/beats
that referenced
this pull request
Aug 17, 2020
… with migration.6_to_7.enabled (elastic#20574) * Test export commands in all Beats (elastic#20016) This adds 4 new integration tests to all Beats that test export commands: > beatname export ilm-policy > beatname export template > beatname export index-pattern > beatname export config * [Filebeat] Add export tests to x-pack/filebeat (elastic#20156) * Add export tests to x-pack/filebeat Add export sub-command tests to x-pack/filebeat and add an assertion for the size the Kibana index-pattern due to the Kibana API limiting payloads to 1 MiB. * Assert size of index pattern document is less than 1 MiB * Fix Winlogbeat export index-pattern with migration.6_to_7.enabled (elastic#20540) - Test export index-pattern with migration.6_to_7.enabled=true - Always run these tests, not only when integration testing - Remove conflicting Winlogbeat alias * Changelog entry Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co> (cherry picked from commit 9ec7549)
Merged
3 tasks
adriansr
added a commit
that referenced
this pull request
Aug 17, 2020
…igration.6_to_7.enabled (#20628) * Cherry-pick #20540 to 7.x: Fix Winlogbeat export index-pattern with migration.6_to_7.enabled (#20574) * Test export commands in all Beats (#20016) This adds 4 new integration tests to all Beats that test export commands: > beatname export ilm-policy > beatname export template > beatname export index-pattern > beatname export config * [Filebeat] Add export tests to x-pack/filebeat (#20156) * Add export tests to x-pack/filebeat Add export sub-command tests to x-pack/filebeat and add an assertion for the size the Kibana index-pattern due to the Kibana API limiting payloads to 1 MiB. * Assert size of index pattern document is less than 1 MiB * Fix Winlogbeat export index-pattern with migration.6_to_7.enabled (#20540) - Test export index-pattern with migration.6_to_7.enabled=true - Always run these tests, not only when integration testing - Remove conflicting Winlogbeat alias Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co> (cherry picked from commit 9ec7549)
melchiormoulin
pushed a commit
to melchiormoulin/beats
that referenced
this pull request
Oct 14, 2020
This adds 4 new integration tests to all Beats that test export commands: > beatname export ilm-policy > beatname export template > beatname export index-pattern > beatname export config
leweafan
pushed a commit
to leweafan/beats
that referenced
this pull request
Apr 28, 2023
… with migration.6_to_7.enabled (elastic#20628) * Cherry-pick elastic#20540 to 7.x: Fix Winlogbeat export index-pattern with migration.6_to_7.enabled (elastic#20574) * Test export commands in all Beats (elastic#20016) This adds 4 new integration tests to all Beats that test export commands: > beatname export ilm-policy > beatname export template > beatname export index-pattern > beatname export config * [Filebeat] Add export tests to x-pack/filebeat (elastic#20156) * Add export tests to x-pack/filebeat Add export sub-command tests to x-pack/filebeat and add an assertion for the size the Kibana index-pattern due to the Kibana API limiting payloads to 1 MiB. * Assert size of index pattern document is less than 1 MiB * Fix Winlogbeat export index-pattern with migration.6_to_7.enabled (elastic#20540) - Test export index-pattern with migration.6_to_7.enabled=true - Always run these tests, not only when integration testing - Remove conflicting Winlogbeat alias Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co> (cherry picked from commit 66cfa25)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This adds 4 new integration tests to all Beats that test export commands:
Why is it important?
In the past we've been hit by some of the export commands failing and there was no test to check that. There's generic tests in libbeat using mockbeat, but in the case of fields definitions, the export command might fail due to problems with the fields built into a particular Beat.
For example, if a Filebeat module defines a field that duplicates a field defined in another place, this was silently accepted but it would cause
filebeat export index-patternto fail with:These new tests will validate that the export commands work for each individual Beat. Actual validation is limited to checking that the command succeeds and that it returns a valid JSON (ilm-policy, template, index-pattern) or YAML (config).
Ideally we would like to catch these kind of errors (esp. fields-definitions related errors) at an earlier stage, but it's worth having this last line of validation.
Checklist
[ ] I have made corresponding changes to the documentation[ ] I have made corresponding change to the default configuration files[ ] I have added an entry inCHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.