[Reporting] Correct the docvalue_fields params in the search query Download CSV from Dashboard Panel #52833

tsullivan · 2019-12-12T00:04:24Z

Summary

Release note: Fixed an issue with CSV download from a saved search panel in a dashboard, where docvalue_fields would not be in the CSV.

Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

~~[ ] This was checked for cross-browser compatibility, including a check against IE11~~
~~[ ] Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support~~
~~[ ] Documentation was added for features that require explanation or tutorials~~
Unit or functional tests were updated or added to match the most common scenarios
~~[ ] This was checked for keyboard-only and screenreader accessibility~~

tsullivan · 2019-12-12T00:31:34Z

x-pack/test/reporting/api/generate/csv_saved_search.ts

+        await esArchiver.load('reporting/ecommerce');
+        await esArchiver.load('reporting/ecommerce_kibana');
+
+        const params = {


This is a copy of the POST body that I captured from a network inspector after running the use case through a browser.

elasticmachine · 2019-12-12T01:46:53Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: c341b98

History

💔 Build #15010 failed 13092da

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

tsullivan · 2019-12-12T17:25:54Z

...legacy/plugins/reporting/export_types/csv_from_savedobject/server/lib/generate_csv_search.ts

  if (jobParams.post && jobParams.post.state) {
    ({
      post: {
-        state: { query: payloadQuery, sort: payloadSort = [] },
+        state: { query: payloadQuery, sort: payloadSort = [], docvalue_fields: docValueFields },


@elastic/kibana-app can you help us understand why the docvalue_fields info is part of the state that we get from the panel info? It means the id of the saved search doc isn't enough to derive an actual ES query - the state data is additional information.

kertal

Code LGTM, would need a bit more time to dive into the code to answer your questions, tested the CSV generation in Chrome, works as expected.

tsullivan · 2020-01-02T20:41:36Z

@elasticmachine merge upstream

kibanamachine · 2020-01-02T22:03:54Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: d103b80

History

💚 Build #15014 succeeded c341b98
💔 Build #15010 failed 13092da

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…wnload CSV from Dashboard Panel (elastic#52833) * fix the bug and add a test * fix query bug of empty array Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* master: Rename `/api/security/oidc` to `/api/security/oidc/callback`. (elastic#53886) Updating transitive dependencies to use handlebars@4.5.3 (elastic#53899) [Reporting/Tests] consolidate functional test configs (elastic#52671) [Reporting] Correct the docvalue_fields params in the search query Download CSV from Dashboard Panel (elastic#52833) [Test/Newsfeed] Re-enable test and add news item to be filtered (elastic#53905) cleanup server-log action (elastic#53326) [Uptime] Delete uptime eslint rule skip (elastic#50912) [skip-ci] Expression Lifecycle Docs (elastic#51494) [Endpoint] add react router to endpoint app (elastic#53808) [SIEM][Detection Engine] Silence 409 errors on signal creation (elastic#53859) [Maps] get max_result_window and max_inner_result_window from index settings (elastic#53500) [ML] New Platform server shim: update analytics routes to use new platform router (elastic#53521) fixes typo on engine detection page (elastic#53877)

* master: Rename `/api/security/oidc` to `/api/security/oidc/callback`. (elastic#53886) Updating transitive dependencies to use handlebars@4.5.3 (elastic#53899) [Reporting/Tests] consolidate functional test configs (elastic#52671) [Reporting] Correct the docvalue_fields params in the search query Download CSV from Dashboard Panel (elastic#52833) [Test/Newsfeed] Re-enable test and add news item to be filtered (elastic#53905) cleanup server-log action (elastic#53326) [Uptime] Delete uptime eslint rule skip (elastic#50912) [skip-ci] Expression Lifecycle Docs (elastic#51494) [Endpoint] add react router to endpoint app (elastic#53808) [SIEM][Detection Engine] Silence 409 errors on signal creation (elastic#53859) [Maps] get max_result_window and max_inner_result_window from index settings (elastic#53500) [ML] New Platform server shim: update analytics routes to use new platform router (elastic#53521) fixes typo on engine detection page (elastic#53877) [Maps] push mapbox value extraction from VectorStyle and into DynamicStyleProperty (elastic#53806) Fix suggested value for time_zone in range query (elastic#53841) Clean up generic hooks, use react-use instead (elastic#53822)

…ris/kibana into alerting/created_at-and-updated_at * 'alerting/created_at-and-updated_at' of github.com:gmmorris/kibana: Rename `/api/security/oidc` to `/api/security/oidc/callback`. (elastic#53886) Updating transitive dependencies to use handlebars@4.5.3 (elastic#53899) [Reporting/Tests] consolidate functional test configs (elastic#52671) [Reporting] Correct the docvalue_fields params in the search query Download CSV from Dashboard Panel (elastic#52833) [Test/Newsfeed] Re-enable test and add news item to be filtered (elastic#53905) cleanup server-log action (elastic#53326) [Uptime] Delete uptime eslint rule skip (elastic#50912) [skip-ci] Expression Lifecycle Docs (elastic#51494) [Endpoint] add react router to endpoint app (elastic#53808) [SIEM][Detection Engine] Silence 409 errors on signal creation (elastic#53859) [Maps] get max_result_window and max_inner_result_window from index settings (elastic#53500) [ML] New Platform server shim: update analytics routes to use new platform router (elastic#53521) fixes typo on engine detection page (elastic#53877) [Maps] push mapbox value extraction from VectorStyle and into DynamicStyleProperty (elastic#53806) Fix suggested value for time_zone in range query (elastic#53841) Clean up generic hooks, use react-use instead (elastic#53822)

…t-types * alerting/created_at-and-updated_at: updatedAt should equal createdAt on creation Move index patterns: src/legacy/core_plugins/data 👉 src/plugins/data (elastic#53794) moved Task Manager server code under "server" directory (elastic#53777) Rename `/api/security/oidc` to `/api/security/oidc/callback`. (elastic#53886) Updating transitive dependencies to use handlebars@4.5.3 (elastic#53899) [Reporting/Tests] consolidate functional test configs (elastic#52671) [Reporting] Correct the docvalue_fields params in the search query Download CSV from Dashboard Panel (elastic#52833) [Test/Newsfeed] Re-enable test and add news item to be filtered (elastic#53905) cleanup server-log action (elastic#53326) [Uptime] Delete uptime eslint rule skip (elastic#50912) [skip-ci] Expression Lifecycle Docs (elastic#51494) [Endpoint] add react router to endpoint app (elastic#53808) [SIEM][Detection Engine] Silence 409 errors on signal creation (elastic#53859) [Maps] get max_result_window and max_inner_result_window from index settings (elastic#53500) [ML] New Platform server shim: update analytics routes to use new platform router (elastic#53521) fixes typo on engine detection page (elastic#53877) [Maps] push mapbox value extraction from VectorStyle and into DynamicStyleProperty (elastic#53806) Fix suggested value for time_zone in range query (elastic#53841) Clean up generic hooks, use react-use instead (elastic#53822)

…wnload CSV from Dashboard Panel (#52833) (#53918) * fix the bug and add a test * fix query bug of empty array Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

tsullivan · 2020-05-20T22:23:14Z

...legacy/plugins/reporting/export_types/csv_from_savedobject/server/lib/generate_csv_search.ts

+    } else {
+      docValueFields = [indexPatternTimeField];
+    }
+  }


I can no longer remember why this block of code was added, but it seems to be causing a regression where the time field value shows up as a multi-value. The effect is even baked into a unit test fixture: https://github.com/elastic/kibana/blob/a8b1a6b/x-pack/test/reporting_api_integration/fixtures.ts#L179

Noticed this while working on: #56371

fix the bug and add a test

13092da

tsullivan added release_note:fix review v7.6.0 v8.0.0 labels Dec 12, 2019

tsullivan requested review from joelgriffith and ppisljar December 12, 2019 00:04

fix query bug of empty array

c341b98

tsullivan commented Dec 12, 2019

View reviewed changes

tsullivan requested a review from lukasolson December 12, 2019 00:38

tsullivan commented Dec 12, 2019

View reviewed changes

kertal approved these changes Dec 13, 2019

View reviewed changes

Merge branch 'master' into reporting/fix-csv-search

d103b80

tsullivan merged commit 01dd08e into elastic:master Jan 2, 2020

tsullivan deleted the reporting/fix-csv-search branch January 2, 2020 22:30

tsullivan mentioned this pull request Jan 2, 2020

[7.x] [Reporting] Correct the docvalue_fields params in the search query Download CSV from Dashboard Panel (#52833) #53918

Merged

tsullivan commented May 20, 2020

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Reporting] Correct the docvalue_fields params in the search query Download CSV from Dashboard Panel #52833

[Reporting] Correct the docvalue_fields params in the search query Download CSV from Dashboard Panel #52833

tsullivan commented Dec 12, 2019 •

edited

Loading

tsullivan Dec 12, 2019

elasticmachine commented Dec 12, 2019

tsullivan Dec 12, 2019

kertal left a comment

tsullivan commented Jan 2, 2020

kibanamachine commented Jan 2, 2020

tsullivan May 20, 2020

[Reporting] Correct the docvalue_fields params in the search query Download CSV from Dashboard Panel #52833

[Reporting] Correct the docvalue_fields params in the search query Download CSV from Dashboard Panel #52833

Conversation

tsullivan commented Dec 12, 2019 • edited Loading

Summary

Checklist

tsullivan Dec 12, 2019

Choose a reason for hiding this comment

elasticmachine commented Dec 12, 2019

💚 Build Succeeded

History

tsullivan Dec 12, 2019

Choose a reason for hiding this comment

kertal left a comment

Choose a reason for hiding this comment

tsullivan commented Jan 2, 2020

kibanamachine commented Jan 2, 2020

💚 Build Succeeded

History

tsullivan May 20, 2020

Choose a reason for hiding this comment

tsullivan commented Dec 12, 2019 •

edited

Loading