Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[rollup index patterns] don't show fields not present in field caps response #105533

Merged
merged 3 commits into from
Jul 16, 2021
Merged

[rollup index patterns] don't show fields not present in field caps response #105533

merged 3 commits into from
Jul 16, 2021

Conversation

mattkime
Copy link
Contributor

@mattkime mattkime commented Jul 14, 2021 •
edited
Loading

Summary

The index pattern field list will show rollup fields that don't exist in the field caps response (due to lack of data) as existing BUT without a field type. Such fields are useless....they don't even really exist. This PR removes them from the field list. There's also some typescript improvements.

To test (do on master and then this branch to see the problem and then to verify its resolved)

  • Run metricbeats locally, it will create an index and a bunch of docs
  • Create a rollup job on the metricbeats index and specify some metrics on the first field in the provided list.
  • Create an index pattern for the newly created rollup index
  • look at the field list. You should see the field chosen for metrics in the list but without a type.

Closes: #104781

@mattkime mattkime changed the title remove fields not present in field caps response [rollup index patterns] don't show fields not present in field caps response Jul 14, 2021
@mattkime mattkime added 7.14.0 Feature:Data Views Data Views code and UI - index patterns before 8.0 Feature:Kibana Management Feature label for Data Views, Advanced Setting, Saved Object management pages Feature:Rollups v7.15.0 v8.0.0 bug Fixes for quality problems that affect the customer experience release_note:skip Skip the PR/issue when compiling release notes auto-backport Deprecated - use backport:version if exact versions are needed labels Jul 14, 2021
mattkime
mattkime commented Jul 14, 2021
View reviewed changes
src/plugins/data/server/index_patterns/fetcher/lib/merge_capabilities_with_fields.ts
@@ -56,16 +56,21 @@ export const mergeCapabilitiesWithFields = (
rollupFields.push(
...fields
.filter((field) => !rollupFieldNames.includes(field))
.map((field) => {
.reduce<FieldDescriptor[]>((collector, field) => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Swapping map for reduce since we don't want to return fields missing in the field caps info.

@mattkime mattkime marked this pull request as ready for review July 14, 2021 19:48
@mattkime mattkime requested a review from a team as a code owner July 14, 2021 19:48
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app-services (Team:AppServices)

mattkime
mattkime commented Jul 14, 2021
View reviewed changes
x-pack/test/api_integration/apis/management/rollup/index_patterns_extensions.js
const fieldsForWildcard = body.fields.map((field) => field.name);
expect(fieldsForWildcard.sort()).eql(propertiesWithMappings.sort());
expect(fieldsForWildcard.sort()).eql(['testCreatedField']);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing test relied on the broken behavior - returning fields that didn't have proper field caps entries. Simply adjusted the test to expect the field with complete info.

@mattkime mattkime requested a review from Dosant July 14, 2021 19:49
@mattkime
Copy link
Contributor Author

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
data 3273 3271 -2

Any counts in public APIs

Total count of every any typed public API. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats any for more detailed information.

id before after diff
data 106 104 -2
Unknown metric groups

API count

id before after diff
data 3839 3837 -2

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Dosant
Dosant approved these changes Jul 16, 2021
View reviewed changes
Copy link
Contributor

@Dosant Dosant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested before/after using steps in the description.
Nice small ts types improvements 👍

@mattkime mattkime merged commit 2dd3693 into elastic:master Jul 16, 2021
@kibanamachine kibanamachine mentioned this pull request Jul 16, 2021
Merged
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Jul 16, 2021
@mattkime @kibanamachine
[rollup index patterns] don't show fields not present in field caps r…
2fda18a 
…esponse (elastic#105533)

* remove fields not present in field caps response
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Jul 16, 2021
@mattkime @kibanamachine
[rollup index patterns] don't show fields not present in field caps r…
72768ef 
…esponse (elastic#105533)

* remove fields not present in field caps response
@kibanamachine kibanamachine mentioned this pull request Jul 16, 2021
Merged
@kibanamachine
Copy link
Contributor

💚 Backport successful

Status Branch Result
✅ 7.14
✅ 7.x

The backport PRs will be merged automatically after passing CI.

kibanamachine added a commit that referenced this pull request Jul 16, 2021
@kibanamachine @mattkime
[rollup index patterns] don't show fields not present in field caps r…
2207cca 
…esponse (#105533) (#105957)

* remove fields not present in field caps response

Co-authored-by: Matthew Kime <matt@mattki.me>
kibanamachine added a commit that referenced this pull request Jul 16, 2021
@kibanamachine @mattkime
[rollup index patterns] don't show fields not present in field caps r…
fb50f83 
…esponse (#105533) (#105958)

* remove fields not present in field caps response

Co-authored-by: Matthew Kime <matt@mattki.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Dosant Dosant Dosant approved these changes

Assignees
No one assigned
Labels
auto-backport Deprecated - use backport:version if exact versions are needed bug Fixes for quality problems that affect the customer experience Feature:Data Views Data Views code and UI - index patterns before 8.0 Feature:Kibana Management Feature label for Data Views, Advanced Setting, Saved Object management pages Feature:Rollups release_note:skip Skip the PR/issue when compiling release notes v7.14.0 v7.15.0 v8.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[rollup index patterns] rollup fields displayed incorrectly in index pattern management
5 participants
@mattkime @elasticmachine @kibanamachine @Dosant @wylieconlon