Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Discover] Add context awareness telemetry tests for Observability profiles #201310

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

davismcphee
Copy link
Contributor

@davismcphee davismcphee commented Nov 22, 2024

Summary

As suggested in #199255 (comment), I've copied and modified the existing Discover context awareness telemetry tests to work for Observability profiles. This helps test that solution root profiles are picked up as expected, as well as giving us some serverless coverage.

@elastic/appex-qa It didn't seem like there were any config specific services for serverless tests yet, so I added the EBT to services to the main serverless config. If there's a better way to do this, please let me know and I can update it.

Checklist

  • Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
  • Documentation was added for features that require explanation or tutorials
  • Unit or functional tests were updated or added to match the most common scenarios
  • If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
  • This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
  • Flaky Test Runner was used on any tests changed
  • The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

@davismcphee davismcphee added Feature:Discover Discover Application release_note:skip Skip the PR/issue when compiling release notes Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) Team:obs-ux-logs Observability Logs User Experience Team Project:OneDiscover Enrich Discover with contextual awareness labels Nov 22, 2024
@davismcphee davismcphee self-assigned this Nov 22, 2024
@davismcphee davismcphee requested a review from a team November 22, 2024 04:56
@kibanamachine
Copy link
Contributor

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7468

[✅] x-pack/test_serverless/functional/test_suites/observability/config.ts: 25/25 tests passed.

see run history

@davismcphee davismcphee marked this pull request as ready for review November 25, 2024 04:00
@davismcphee davismcphee requested review from a team as code owners November 25, 2024 04:00
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

Copy link
Contributor

@jughosta jughosta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Copy link
Contributor

@tonyghiani tonyghiani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment on lines 44 to 47
`--plugin-path=${resolve(
__dirname,
'../../../test/analytics/plugins/analytics_ftr_helpers'
)}`,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you help me understand what this plugin is doing?
Asking because if this is something-test-only where the behavior is different to what we have in a real serverless environment, then tests based on this would fail when running against MKI. In this case, this config entry should move to a separate config (see e.g. here how it's done for the example plugins), which doesn't run against MKI.
If it simulates MKI behavior locally, then it can stay. Would probably be good to add a comment explaining it in that case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elastic/kibana-core could confirm for sure since they maintain the service, but I'm pretty sure this does alter the behaviour because it adds a custom analytics shipper to capture EBT events in tests. I added a separate config for the telemetry tests here: 67a5954.

I also wasn't really sure how to handle the custom services to make sure they aren't available where they won't work, so I tried to do something similar to what was done for API integration tests: 38b9306. Let me know if you think there's a better way to do this and I can update.

@davismcphee davismcphee requested a review from a team as a code owner November 25, 2024 22:29
Copy link
Member

@jbudz jbudz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.buildkite/ftr_oblt_serverless_configs.yml

@elasticmachine
Copy link
Contributor

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] Jest Tests #14 / FilterPopover renders empty label correctly

Metrics [docs]

✅ unchanged

History

cc @davismcphee

@@ -1131,6 +1131,7 @@ x-pack/test_serverless/api_integration/test_suites/common/platform_security @ela
/x-pack/test_serverless/functional/test_suites/common/examples/unified_field_list_examples @elastic/kibana-data-discovery
/x-pack/test_serverless/functional/test_suites/common/management/data_views @elastic/kibana-data-discovery
src/plugins/discover/public/context_awareness/profile_providers/security @elastic/kibana-data-discovery @elastic/security-threat-hunting-investigations
src/plugins/discover/public/context_awareness/profile_providers/observability @elastic/kibana-data-discovery @elastic/obs-ux-logs-team
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tonyghiani I noticed this wasn't updated when we added the observability folder but I figure it makes sense to share ownership here.

Comment on lines 44 to 47
`--plugin-path=${resolve(
__dirname,
'../../../test/analytics/plugins/analytics_ftr_helpers'
)}`,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elastic/kibana-core could confirm for sure since they maintain the service, but I'm pretty sure this does alter the behaviour because it adds a custom analytics shipper to capture EBT events in tests. I added a separate config for the telemetry tests here: 67a5954.

I also wasn't really sure how to handle the custom services to make sure they aren't available where they won't work, so I tried to do something similar to what was done for API integration tests: 38b9306. Let me know if you think there's a better way to do this and I can update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) Feature:Discover Discover Application Project:OneDiscover Enrich Discover with contextual awareness release_note:skip Skip the PR/issue when compiling release notes Team:DataDiscovery Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. Team:obs-ux-logs Observability Logs User Experience Team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants