[data views] index negation doesn't work with cross cluster search #147926
Labels
bug
Fixes for quality problems that affect the customer experience
Feature:Data Views
Data Views code and UI - index patterns before 8.0
impact:high
Addressing this issue will have a high level of impact on the quality/strength of our product.
Team:DataDiscovery
Discover, search (e.g. data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL.
Comments
|
I've reproduced the problem. I working on looking at the request between Kibana and ES to see where the bug exists. Field caps works as expected, meaning it doesn't exhibit the bug. |
lukeelmers
added
the
Team:DataDiscovery
|
Pinging @elastic/kibana-data-discovery (Team:DataDiscovery) |
This was referenced Dec 22, 2022
|
Update - I was able to see the problem by turning on elasticsearch query logging in kibana. Kibana was swallowing cross cluster negation index patterns - there's code that checks for negation patterns but it fails to handle the cross cluster ones. I'm now working to determine whether I should specifically handle cross cluster patterns or whether the validation code can be removed entirely. |
kertal
added
bug
mattkime
added a commit
that referenced
this issue
Dec 29, 2022
## Summary The data views api examines comma delimited sections to see if there are matching indices before fetching the field list. The existing code checked for index pattern negation - patterns that started with a `-`. However, it didn't check for this in cross cluster case - `this_cluster:-kibana*`. The code now handles the cross cluster case appropriately. Still, its unclear to me whether this logic is necessary, hence why I opened #147970 - I was able to resolve the failed tests most familiar to me but I will need to work with engineers from other teams to investigate other failures. Closes #147926
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Dec 29, 2022
## Summary The data views api examines comma delimited sections to see if there are matching indices before fetching the field list. The existing code checked for index pattern negation - patterns that started with a `-`. However, it didn't check for this in cross cluster case - `this_cluster:-kibana*`. The code now handles the cross cluster case appropriately. Still, its unclear to me whether this logic is necessary, hence why I opened elastic#147970 - I was able to resolve the failed tests most familiar to me but I will need to work with engineers from other teams to investigate other failures. Closes elastic#147926 (cherry picked from commit e3cac21)
kibanamachine
referenced
this issue
Dec 29, 2022
… (#148206) # Backport This will backport the following commits from `main` to `8.6`: - [[data views] allow cross cluster index pattern negation (#147968)](#147968) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Matthew Kime","email":"matt@mattki.me"},"sourceCommit":{"committedDate":"2022-12-29T15:51:20Z","message":"[data views] allow cross cluster index pattern negation (#147968)\n\n## Summary\r\n\r\nThe data views api examines comma delimited sections to see if there are\r\nmatching indices before fetching the field list. The existing code\r\nchecked for index pattern negation - patterns that started with a `-`.\r\nHowever, it didn't check for this in cross cluster case -\r\n`this_cluster:-kibana*`. The code now handles the cross cluster case\r\nappropriately.\r\n\r\nStill, its unclear to me whether this logic is necessary, hence why I\r\nopened #147970 - I was able to\r\nresolve the failed tests most familiar to me but I will need to work\r\nwith engineers from other teams to investigate other failures.\r\n\r\nCloses https://github.com/elastic/kibana/issues/147926","sha":"e3cac218f011b3de0b39700feed1500f6129fd99","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:fix","Feature:Data Views","Team:DataDiscovery","backport:prev-minor","v8.7.0"],"number":147968,"url":"https://github.com/elastic/kibana/pull/147968","mergeCommit":{"message":"[data views] allow cross cluster index pattern negation (#147968)\n\n## Summary\r\n\r\nThe data views api examines comma delimited sections to see if there are\r\nmatching indices before fetching the field list. The existing code\r\nchecked for index pattern negation - patterns that started with a `-`.\r\nHowever, it didn't check for this in cross cluster case -\r\n`this_cluster:-kibana*`. The code now handles the cross cluster case\r\nappropriately.\r\n\r\nStill, its unclear to me whether this logic is necessary, hence why I\r\nopened #147970 - I was able to\r\nresolve the failed tests most familiar to me but I will need to work\r\nwith engineers from other teams to investigate other failures.\r\n\r\nCloses https://github.com/elastic/kibana/issues/147926","sha":"e3cac218f011b3de0b39700feed1500f6129fd99"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/147968","number":147968,"mergeCommit":{"message":"[data views] allow cross cluster index pattern negation (#147968)\n\n## Summary\r\n\r\nThe data views api examines comma delimited sections to see if there are\r\nmatching indices before fetching the field list. The existing code\r\nchecked for index pattern negation - patterns that started with a `-`.\r\nHowever, it didn't check for this in cross cluster case -\r\n`this_cluster:-kibana*`. The code now handles the cross cluster case\r\nappropriately.\r\n\r\nStill, its unclear to me whether this logic is necessary, hence why I\r\nopened #147970 - I was able to\r\nresolve the failed tests most familiar to me but I will need to work\r\nwith engineers from other teams to investigate other failures.\r\n\r\nCloses https://github.com/elastic/kibana/issues/147926","sha":"e3cac218f011b3de0b39700feed1500f6129fd99"}}]}] BACKPORT--> Co-authored-by: Matthew Kime <matt@mattki.me>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
the index pattern
*:remote_test_index_*,*:-remote_test_index_2will pull in fields fromremote_test_index_2The text was updated successfully, but these errors were encountered: