Merge branch 'master' into chore/siem-enzyme-to-json

docs/user/reporting/index.asciidoc

@@ -6,11 +6,11 @@
 
 --
 
-You can generate a report that contains a {kib} dashboard, visualization, 
-saved search, or Canvas workpad. Depending on the object type, you can export the data as 
+You can generate a report that contains a {kib} dashboard, visualization,
+saved search, or Canvas workpad. Depending on the object type, you can export the data as
 a PDF, PNG, or CSV document, which you can keep for yourself, or share with others.
 
-Reporting is available from the *Share* menu 
+Reporting is available from the *Share* menu
 in *Discover*, *Visualize*, *Dashboard*, and *Canvas*.
 
 [role="screenshot"]
@@ -40,9 +40,9 @@ for an example.
 [[manually-generate-reports]]
 == Generate a report manually
 
-. Open the dashboard, visualization, Canvas workpad, or saved search that you want to include in the report. 
+. Open the dashboard, visualization, Canvas workpad, or saved search that you want to include in the report.
 
-. In the {kib} toolbar, click *Share*.  If you are working in Canvas, 
+. In the {kib} toolbar, click *Share*.  If you are working in Canvas,
 click the share icon image:user/reporting/images/canvas-share-button.png["Canvas Share button"].
 
 . Select the option appropriate for your object. You can export:
@@ -55,14 +55,36 @@ click the share icon image:user/reporting/images/canvas-share-button.png["Canvas
 +
 A notification appears when the report is complete.
 
+[float]
+[[reporting-layout-sizing]]
+== Layout and sizing
+The layout and size of the PDF or PNG image depends on the {kib} app
+with which the Reporting plugin is integrated. For Canvas, the
+worksheet dimensions determine the size for Reporting. In other apps,
+the dimensions are taken on the fly by looking at
+the size of the visualization elements or panels on the page.
+
+The size dimensions are part of the reporting job parameters. Therefore, to
+make the report output larger or smaller, you can change the size of the browser.
+This resizes the shareable container before generating the
+report, so the desired dimensions are passed in the job parameters.
+
+In the following {kib} dashboard, the shareable container is highlighted.
+The shareable container is captured when you click the
+*Generate* or *Copy POST URL* button. It might take some trial and error
+before you're satisfied with the layout and dimensions in the resulting
+PNG or PDF image.
+
+[role="screenshot"]
+image::user/reporting/images/shareable-container.png["Shareable Container"]
+
+
+
 [float]
 [[optimize-pdf]]
 == Optimize PDF for print—dashboard only
 
-By default, {kib} creates a PDF  
-using the existing layout and size of the dashboard. To create a 
-printer-friendly PDF with multiple A4 portrait pages and two visualizations 
-per page, turn on *Optimize for printing*.
+To create a printer-friendly PDF with multiple A4 portrait pages and two visualizations per page, turn on *Optimize for printing*.
 
 [role="screenshot"]
 image::user/reporting/images/preserve-layout-switch.png["Share"]
@@ -72,8 +94,8 @@ image::user/reporting/images/preserve-layout-switch.png["Share"]
 [[manage-report-history]]
 == View and manage report history
 
-For a list of your reports, go to *Management > Reporting*.  
-From this view, you can monitor the generation of a report and 
+For a list of your reports, go to *Management > Reporting*.
+From this view, you can monitor the generation of a report and
 download reports that you previously generated.
 
 [float]

docs/user/reporting/reporting-troubleshooting.asciidoc

@@ -7,12 +7,20 @@
 
 Having trouble? Here are solutions to common problems you might encounter while using Reporting.
 
+* <<reporting-troubleshooting-system-dependencies>>
+* <<reporting-troubleshooting-text-incorrect>>
+* <<reporting-troubleshooting-missing-data>>
+* <<reporting-troubleshooting-file-permissions>>
+* <<reporting-troubleshooting-error-messages>>
+* <<reporting-troubleshooting-puppeteer-debug-logs>>
+* <<reporting-troubleshooting-system-requirements>>
+
 [float]
 [[reporting-troubleshooting-system-dependencies]]
 === System dependencies
 Reporting launches a "headless" web browser called Chromium on the Kibana server. It is a custom build made by Elastic of an open source
 project, and it is intended to have minimal dependencies on OS libraries. However, the Kibana server OS might still require additional
-dependencies for Chromium.
+dependencies to run the Chromium executable.
 
 Make sure Kibana server OS has the appropriate packages installed for the distribution.
 
@@ -33,19 +41,30 @@ If you are using Ubuntu/Debian systems, install the following packages:
 * `fonts-liberation`
 * `libfontconfig1`
 
+If the system is missing dependencies, then Reporting will fail in a non-deterministic way. {kib} runs a self-test at server startup, and
+if it encounters errors, logs them in the Console. Unfortunately, the error message does not include
+information about why Chromium failed to run. The most common error message is `Error: connect ECONNREFUSED`, which indicates
+that {kib} could not connect to the Chromium process.
+
+To troubleshoot the problem, start the {kib} server with environment variables that tell Chromium to print verbose logs. See the
+<<reporting-troubleshooting-puppeteer-debug-logs, Puppeteer debug method>> for more information.
+
 [float]
-=== Text is rendered incorrectly in generated reports
+[[reporting-troubleshooting-text-incorrect]]
+=== Text rendered incorrectly in generated reports
 
 If a report label is rendered as an empty rectangle, no system fonts are available. Install at least one font package on the system.
 
 If the report is missing certain Chinese, Japanese or Korean characters, ensure that a system font with those characters is installed.
 
 [float]
+[[reporting-troubleshooting-missing-data]]
 === Missing data in PDF report of data table visualization
 There is currently a known limitation with the Data Table visualization that only the first page of data rows, which are the only data
 visible on the screen, are shown in PDF reports.
 
 [float]
+[[reporting-troubleshooting-file-permissions]]
 === File permissions
 Ensure that the `headless_shell` binary located in your Kibana data directory is owned by the user who is running Kibana, that the
 user has the execute permission, and if applicable, that the filesystem is mounted with the `exec` option.
@@ -63,25 +82,25 @@ Whenever possible, a Reporting error message tries to be as self-explanatory as
 along with the solution.
 
 [float]
-==== "Max attempts reached"
+==== Max attempts reached
 There are two primary causes of this error:
 
-. You're creating a PDF of a visualization or dashboard that spans a large amount of data and Kibana is hitting the `xpack.reporting.queue.timeout`
+* You're creating a PDF of a visualization or dashboard that spans a large amount of data and Kibana is hitting the `xpack.reporting.queue.timeout`
 
-. Kibana is hosted behind a reverse-proxy, and the <<reporting-kibana-server-settings, Kibana server settings>> are not configured correctly
+* Kibana is hosted behind a reverse-proxy, and the <<reporting-kibana-server-settings, Kibana server settings>> are not configured correctly
 
 Create a Markdown visualization and then create a PDF report. If this succeeds, increase the `xpack.reporting.queue.timeout` setting. If the
 PDF report fails with "Max attempts reached," check your <<reporting-kibana-server-settings, Kibana server settings>>.
 
 [float]
 [[reporting-troubleshooting-nss-dependency]]
-==== "You must install nss for Reporting to work"
+==== You must install nss for Reporting to work
 Reporting using the Chromium browser relies on the Network Security Service libraries (NSS). Install the appropriate nss package for your
 distribution.
 
 [float]
 [[reporting-troubleshooting-sandbox-dependency]]
-==== "Unable to use Chromium sandbox"
+==== Unable to use Chromium sandbox
 Chromium uses sandboxing techniques that are built on top of operating system primitives. The Linux sandbox depends on user namespaces,
 which were introduced with the 3.8 Linux kernel. However, many distributions don't have user namespaces enabled by default, or they require
 the CAP_SYS_ADMIN capability.
@@ -90,6 +109,7 @@ Elastic recommends that you research the feasibility of enabling unprivileged us
 is if you are running Kibana in Docker because the container runs in a user namespace with the built-in seccomp/bpf filters.
 
 [float]
+[[reporting-troubleshooting-verbose-logs]]
 === Verbose logs
 {kib} server logs have a lot of useful information for troubleshooting and understanding how things work. If you're having any issues at
 all, the full logs from Reporting will be the first place to look. In `kibana.yml`:
@@ -101,10 +121,12 @@ logging.verbose: true
 
 For more information about logging, see <<logging-verbose,Kibana configuration settings>>.
 
+[float]
+[[reporting-troubleshooting-puppeteer-debug-logs]]
 === Puppeteer debug logs
 The Chromium browser that {kib} launches on the server is driven by a NodeJS library for Chromium called Puppeteer. The Puppeteer library
 has its own command-line method to generate its own debug logs, which can sometimes be helpful, particularly to figure out if a problem is
-caused by Kibana or Chromium. See more at https://github.com/GoogleChrome/puppeteer/blob/v1.19.0/README.md#debugging-tips
+caused by Kibana or Chromium. See more at https://github.com/GoogleChrome/puppeteer/blob/v1.19.0/README.md#debugging-tips[debugging tips].
 
 Using Puppeteer's debug method when launching Kibana would look like:
 ```
@@ -114,3 +136,14 @@ The internal DevTools protocol traffic will be logged via the `debug` module und
 
 
 The Puppeteer logs are very verbose and could possibly contain sensitive information. Handle the generated output with care.
+
+[float]
+[[reporting-troubleshooting-system-requirements]]
+=== System requirements
+In Elastic Cloud, the {kib} instances that most configurations provide by default is for 1GB of RAM for the instance. That is enough for
+{kib} Reporting when the visualization or dashboard is relatively simple, such as a single pie chart or a dashboard with
+a few visualizations. However, certain visualization types incur more load than others. For example, a TSVB panel has a lot of network
+requests to render.
+
+If the {kib} instance doesn't have enough memory to run the report, the report fails with an error such as `Error: Page crashed!`
+In this case, try increasing the memory for the {kib} instance to 2GB.

packages/kbn-analytics/src/report.ts

@@ -78,6 +78,7 @@ export class ReportManager {
   }
   assignReports(newMetrics: Metric | Metric[]) {
     wrapArray(newMetrics).forEach(newMetric => this.assignReport(this.report, newMetric));
+    return { report: this.report };
   }
   static createMetricKey(metric: Metric): string {
     switch (metric.type) {
@@ -101,7 +102,7 @@ export class ReportManager {
       case METRIC_TYPE.USER_AGENT: {
         const { appName, type, userAgent } = metric;
         if (userAgent) {
-          this.report.userAgent = {
+          report.userAgent = {
             [key]: {
               key,
               appName,
@@ -110,23 +111,22 @@ export class ReportManager {
             },
           };
         }
+
         return;
       }
       case METRIC_TYPE.CLICK:
       case METRIC_TYPE.LOADED:
       case METRIC_TYPE.COUNT: {
         const { appName, type, eventName, count } = metric;
-        if (report.uiStatsMetrics) {
-          const existingStats = (report.uiStatsMetrics[key] || {}).stats;
-          this.report.uiStatsMetrics = this.report.uiStatsMetrics || {};
-          this.report.uiStatsMetrics[key] = {
-            key,
-            appName,
-            eventName,
-            type,
-            stats: this.incrementStats(count, existingStats),
-          };
-        }
+        report.uiStatsMetrics = report.uiStatsMetrics || {};
+        const existingStats = (report.uiStatsMetrics[key] || {}).stats;
+        report.uiStatsMetrics[key] = {
+          key,
+          appName,
+          eventName,
+          type,
+          stats: this.incrementStats(count, existingStats),
+        };
         return;
       }
       default:

src/legacy/core_plugins/console/public/kibana.json

@@ -3,5 +3,6 @@
   "version": "kibana",
   "server": true,
   "ui": true,
-  "requiredPlugins": ["home"]
+  "requiredPlugins": ["home"],
+  "optionalPlugins": ["usageCollection"]
 }

src/legacy/core_plugins/console/public/legacy.ts

@@ -22,7 +22,13 @@ import { I18nContext } from 'ui/i18n';
 import chrome from 'ui/chrome';
 import { FeatureCatalogueCategory } from 'ui/registry/feature_catalogue';
 
+import { plugin } from './np_ready';
+import { DevToolsSetup } from '../../../../plugins/dev_tools/public';
+import { HomePublicPluginSetup } from '../../../../plugins/home/public';
+import { UsageCollectionSetup } from '../../../../plugins/usage_collection/public';
+
 export interface XPluginSet {
+  usageCollection: UsageCollectionSetup;
   dev_tools: DevToolsSetup;
   home: HomePublicPluginSetup;
   __LEGACY: {
@@ -32,10 +38,6 @@ export interface XPluginSet {
   };
 }
 
-import { plugin } from './np_ready';
-import { DevToolsSetup } from '../../../../plugins/dev_tools/public';
-import { HomePublicPluginSetup } from '../../../../plugins/home/public';
-
 const pluginInstance = plugin({} as any);
 
 (async () => {

src/legacy/core_plugins/console/public/np_ready/application/containers/editor/legacy/console_editor/editor.test.mock.tsx

@@ -22,6 +22,7 @@ jest.mock('../../../../contexts/editor_context/editor_registry.ts', () => ({
     setInputEditor: () => {},
     getInputEditor: () => ({
       getRequestsInRange: async () => [{ test: 'test' }],
+      getCoreEditor: () => ({ getCurrentPosition: jest.fn() }),
     }),
   },
 }));
@@ -52,3 +53,6 @@ jest.mock('../../../../models/sense_editor', () => {
 jest.mock('../../../../hooks/use_send_current_request_to_es/send_request_to_es', () => ({
   sendRequestToES: jest.fn(),
 }));
+jest.mock('../../../../../lib/autocomplete/get_endpoint_from_position', () => ({
+  getEndpointFromPosition: jest.fn(),
+}));

src/legacy/core_plugins/console/public/np_ready/application/containers/editor/legacy/console_editor/editor.test.tsx

@@ -32,14 +32,18 @@ import {
   ServicesContextProvider,
   EditorContextProvider,
   RequestContextProvider,
+  ContextValue,
 } from '../../../../contexts';
 
+// Mocked functions
 import { sendRequestToES } from '../../../../hooks/use_send_current_request_to_es/send_request_to_es';
+import { getEndpointFromPosition } from '../../../../../lib/autocomplete/get_endpoint_from_position';
+
 import * as consoleMenuActions from '../console_menu_actions';
 import { Editor } from './editor';
 
 describe('Legacy (Ace) Console Editor Component Smoke Test', () => {
-  let mockedAppContextValue: any;
+  let mockedAppContextValue: ContextValue;
   const sandbox = sinon.createSandbox();
 
   const doMount = () =>
@@ -58,22 +62,28 @@ describe('Legacy (Ace) Console Editor Component Smoke Test', () => {
   beforeEach(() => {
     document.queryCommandSupported = sinon.fake(() => true);
     mockedAppContextValue = {
+      elasticsearchUrl: 'test',
       services: {
+        trackUiMetric: { count: () => {}, load: () => {} },
+        settings: {} as any,
+        storage: {} as any,
         history: {
-          getSavedEditorState: () => null,
+          getSavedEditorState: () => ({} as any),
           updateCurrentState: jest.fn(),
-        },
+        } as any,
         notifications: notificationServiceMock.createSetupContract(),
       },
       docLinkVersion: 'NA',
     };
   });
 
   afterEach(() => {
+    jest.clearAllMocks();
     sandbox.restore();
   });
 
   it('calls send current request to ES', async () => {
+    (getEndpointFromPosition as jest.Mock).mockReturnValue({ patterns: [] });
     (sendRequestToES as jest.Mock).mockRejectedValue({});
     const editor = doMount();
     act(() => {

src/legacy/core_plugins/console/public/np_ready/application/containers/editor/legacy/console_menu_actions.ts

@@ -17,8 +17,7 @@
  * under the License.
  */
 
-// @ts-ignore
-import { getEndpointFromPosition } from '../../../../lib/autocomplete/autocomplete';
+import { getEndpointFromPosition } from '../../../../lib/autocomplete/get_endpoint_from_position';
 import { SenseEditor } from '../../../models/sense_editor';
 
 export async function autoIndent(editor: SenseEditor, event: Event) {
@@ -40,7 +39,7 @@ export function getDocumentation(
     }
     const position = requests[0].range.end;
     position.column = position.column - 1;
-    const endpoint = getEndpointFromPosition(editor, position, editor.parser);
+    const endpoint = getEndpointFromPosition(editor.getCoreEditor(), position, editor.parser);
     if (endpoint && endpoint.documentation && endpoint.documentation.indexOf('http') !== -1) {
       return endpoint.documentation
         .replace('/master/', `/${docLinkVersion}/`)

src/legacy/core_plugins/console/public/np_ready/application/contexts/index.ts

@@ -17,7 +17,7 @@
  * under the License.
  */
 
-export { useServicesContext, ServicesContextProvider } from './services_context';
+export { useServicesContext, ServicesContextProvider, ContextValue } from './services_context';
 
 export {
   useRequestActionContext,

src/legacy/core_plugins/console/public/np_ready/application/contexts/services_context.tsx

@@ -20,13 +20,15 @@
 import React, { createContext, useContext } from 'react';
 import { NotificationsSetup } from 'kibana/public';
 import { History, Storage, Settings } from '../../services';
+import { MetricsTracker } from '../../types';
 
-interface ContextValue {
+export interface ContextValue {
   services: {
     history: History;
     storage: Storage;
     settings: Settings;
     notifications: NotificationsSetup;
+    trackUiMetric: MetricsTracker;
   };
   elasticsearchUrl: string;
   docLinkVersion: string;