diff --git a/Jenkinsfile b/Jenkinsfile index ad5bd23b76b5d..8e678c1e440ac 100644 --- a/Jenkinsfile +++ b/Jenkinsfile @@ -41,6 +41,7 @@ kibanaPipeline(timeoutMinutes: 135, checkPrChanges: true) { 'xpack-ciGroup9': kibanaPipeline.xpackCiGroupProcess(9), 'xpack-ciGroup10': kibanaPipeline.xpackCiGroupProcess(10), 'xpack-accessibility': kibanaPipeline.functionalTestProcess('xpack-accessibility', './test/scripts/jenkins_xpack_accessibility.sh'), + 'xpack-pageLoadMetrics': kibanaPipeline.functionalTestProcess('xpack-pageLoadMetrics', './test/scripts/jenkins_xpack_page_load_metrics.sh'), 'xpack-siemCypress': { processNumber -> whenChanged(['x-pack/plugins/siem/', 'x-pack/test/siem_cypress/']) { kibanaPipeline.functionalTestProcess('xpack-siemCypress', './test/scripts/jenkins_siem_cypress.sh')(processNumber) diff --git a/docs/CHANGELOG.asciidoc b/docs/CHANGELOG.asciidoc index 36125da1039c9..966b14c468961 100644 --- a/docs/CHANGELOG.asciidoc +++ b/docs/CHANGELOG.asciidoc @@ -89,7 +89,7 @@ Canvas:: * Improves expression autocomplete {pull}52035[#52035] Dashboard:: * Use Elasticsearch `_async_search` instead of `_search` when it is available (excluding TSVB, Timelion, and Vega) {pull}59224[#59224} -* When queries run more than 10 seconds, show a pop-up to allow users to run the queries beyond the configured timeout or cancel the queries {pull}60706[#60706] +* When queries run more than 10 seconds, show a pop-up to allow users to run the queries beyond the configured Elasticsearch query timeout or cancel the queries {pull}60706[#60706] * Dashboard/add panel flow {pull}59918[#59918] * Moves the "Create New" button in add panel flyout to the top to make it more visible to the user {pull}56428[#56428] Lens and visualizations:: diff --git a/docs/apm/advanced-queries.asciidoc b/docs/apm/advanced-queries.asciidoc index f89f994e59e57..7b771eb662616 100644 --- a/docs/apm/advanced-queries.asciidoc +++ b/docs/apm/advanced-queries.asciidoc @@ -11,7 +11,7 @@ or, to only show transactions that are slower than a specified time threshold. ==== Example APM app queries * Exclude response times slower than 2000 ms: `transaction.duration.us > 2000000` -* Filter by response status code: `context.response.status_code >= 400` +* Filter by response status code: `context.response.status_code ≥ 400` * Filter by single user ID: `context.user.id : 12` When querying in the APM app, you're merely searching and selecting data from fields in Elasticsearch documents. diff --git a/docs/apm/service-maps.asciidoc b/docs/apm/service-maps.asciidoc index 3a6a96fca9d09..db2f85c54c762 100644 --- a/docs/apm/service-maps.asciidoc +++ b/docs/apm/service-maps.asciidoc @@ -62,9 +62,9 @@ Machine learning jobs can be created to calculate anomaly scores on APM transact When these jobs are active, service maps will display a color-coded anomaly indicator based on the detected anomaly score: [horizontal] -image:apm/images/green-service.png[APM green service]:: Max anomaly score **<=25**. Service is healthy. +image:apm/images/green-service.png[APM green service]:: Max anomaly score **≤25**. Service is healthy. image:apm/images/yellow-service.png[APM yellow service]:: Max anomaly score **26-74**. Anomalous activity detected. Service may be degraded. -image:apm/images/red-service.png[APM red service]:: Max anomaly score **>=75**. Anomalous activity detected. Service is unhealthy. +image:apm/images/red-service.png[APM red service]:: Max anomaly score **≥75**. Anomalous activity detected. Service is unhealthy. [role="screenshot"] image::apm/images/apm-service-map-anomaly.png[Example view of anomaly scores on service maps in the APM app] @@ -92,10 +92,10 @@ Type and subtype are based on `span.type`, and `span.subtype`. Service maps are supported for the following Agent versions: [horizontal] -Go Agent:: >= v1.7.0 -Java Agent:: >= v1.13.0 -.NET Agent:: >= v1.3.0 -Node.js Agent:: >= v3.6.0 -Python Agent:: >= v5.5.0 -Ruby Agent:: >= v3.6.0 -Real User Monitoring (RUM) Agent:: >= v4.7.0 +Go Agent:: ≥ v1.7.0 +Java Agent:: ≥ v1.13.0 +.NET Agent:: ≥ v1.3.0 +Node.js Agent:: ≥ v3.6.0 +Python Agent:: ≥ v5.5.0 +Ruby Agent:: ≥ v3.6.0 +Real User Monitoring (RUM) Agent:: ≥ v4.7.0 diff --git a/docs/images/add-data-fv.png b/docs/images/add-data-fv.png new file mode 100755 index 0000000000000..45313d133822c Binary files /dev/null and b/docs/images/add-data-fv.png differ diff --git a/docs/images/add-data-tutorials.png b/docs/images/add-data-tutorials.png new file mode 100644 index 0000000000000..74deedc57b42e Binary files /dev/null and b/docs/images/add-data-tutorials.png differ diff --git a/docs/management/managing-remote-clusters.asciidoc b/docs/management/managing-remote-clusters.asciidoc index 00ec5c7d2ddea..51d9f42a0b83e 100644 --- a/docs/management/managing-remote-clusters.asciidoc +++ b/docs/management/managing-remote-clusters.asciidoc @@ -31,6 +31,10 @@ to reproduce indices in the remote cluster on a local cluster. [role="screenshot"] image::images/add_remote_cluster.png[][UI for adding a remote cluster] +To create an index pattern to search across clusters, +use the same syntax that you’d use in a raw cross-cluster search request in {es}: :. +See <> for examples. + [float] [[manage-remote-clusters]] === Manage remote clusters diff --git a/docs/maps/images/fu_gs_select_source_file_upload.png b/docs/maps/images/fu_gs_select_source_file_upload.png index 6939f6a82b297..4fe1162acb29c 100644 Binary files a/docs/maps/images/fu_gs_select_source_file_upload.png and b/docs/maps/images/fu_gs_select_source_file_upload.png differ diff --git a/docs/setup/connect-to-elasticsearch.asciidoc b/docs/setup/connect-to-elasticsearch.asciidoc index 00acb73bd276f..6137e028db3fd 100644 --- a/docs/setup/connect-to-elasticsearch.asciidoc +++ b/docs/setup/connect-to-elasticsearch.asciidoc @@ -1,44 +1,105 @@ [[connect-to-elasticsearch]] -== Connect Kibana with Elasticsearch +== Adding data -Before you can start using Kibana, you need to tell it which Elasticsearch indices you want to explore. -The first time you access Kibana, you are prompted to define an _index pattern_ that matches the name of -one or more of your indices. That's it. That's all you need to configure to start using Kibana. You can -add index patterns at any time from the <>. +To start working with your data in {kib}, you can: -TIP: By default, Kibana connects to the Elasticsearch instance running on `localhost`. To connect to a -different Elasticsearch instance, modify the Elasticsearch URL in the `kibana.yml` configuration file and -restart Kibana. For information about using Kibana with your production nodes, see <>. +* Upload a CSV, JSON, or log file with the File Data Visualizer. -To configure the Elasticsearch indices you want to access with Kibana: +* Upload geospatial data with the GeoJSON Upload feature. -. Point your browser at port 5601 to access the Kibana UI. For example, `localhost:5601` or -`http://YOURDOMAIN.com:5601`. -+ -image:images/Start-Page.png[Kibana start page] -+ -. Specify an index pattern that matches the name of one or more of your Elasticsearch indices. The pattern -can include an asterisk (*) to matches zero or more characters in an index's name. When filling out your -index pattern, any matched indices will be displayed. -. Click *Next Step* to select the index field that contains the timestamp you want to use to perform time-based -comparisons. Kibana reads the index mapping to list all of the fields that contain a timestamp. If your -index doesn't have time-based data, choose *I don't want to use the Time Filter* option. -+ -. Click *Create index pattern* to add the index pattern. This first pattern is automatically configured as the default. -When you have more than one index pattern, you can designate which one to use as the default by clicking -on the star icon above the index pattern title from *Management > Index Patterns*. +* Index logs, metrics, events, or application data by setting up a Beats module. + +* Connect {kib} with existing {es} indices. + +If you're not ready to use your own data, you can add a <> +to see all that you can do in {kib}. + +[float] +[[upload-data-kibana]] +=== Upload a CSV, JSON, or log file + +To visualize data in a CSV, JSON, or log file, you can +upload it using the File Data Visualizer. On the home page, +click *Import a CSV, NDSON, or log file*, and then drag your file into the +File Data Visualizer. + +You can upload a file up to 100 MB. This value is configurable up to 1 GB in +<>. + +[role="screenshot"] +image::images/add-data-fv.png[File Data Visualizer] + +The File Data Visualizer uses the {ref}/ml-find-file-structure.html[find_file_structure API] to analyze +the uploaded file and to suggest ingest pipelines and mappings for your data. + +NOTE: This feature is not intended for use as part of a +repeated production process, but rather for the initial exploration of your data. + +[float] +[[upload-geoipdata-kibana]] +=== Upload geospatial data + +To visualize geospatial data in a point or shape file, you can upload it using the <> +feature in *Elastic Maps*, and then use that data as a layer in a map. +The data is also available for use in the broader Kibana ecosystem, for example, +in visualizations and Canvas workpads. +With GeoJSON Upload, you can upload a file up to 50 MB. + +[role="screenshot"] +image::images/fu_gs_select_source_file_upload.png[] -All done! Kibana is now connected to your Elasticsearch data. Kibana displays a read-only list of fields -configured for the matching index. [float] -[[explore]] -=== Start Exploring your Data! -You're ready to dive in to your data: +[[add-data-tutorial-kibana]] +=== Index metrics, log, security, and application data -* Search and browse your data interactively from the <> page. -* Chart and map your data from the <> page. -* Create and view custom dashboards from the <> page. +The built-in data tutorials can help you quickly get up and running with +metrics data, log analytics, security events, and application data. +These tutorials walk you through installing and configuring a +Beats data shipper to periodically collect and send data to {es}. +You can then use the pre-built dashboards to explore and analyze the data. -For a step-by-step introduction to these core Kibana concepts, see the <> tutorial. +You access the tutorials from the home page. +If a tutorial doesn’t exist for your data, go to the {beats-ref}/beats-reference.html[Beats overview] +to learn about other data shippers in the Beats family. + +[role="screenshot"] +image::images/add-data-tutorials.png[Add Data tutorials] + + +[float] +[[connect-to-es]] +=== Connect with {es} indices + +To visualize data in existing {es} indices, you must +create an index pattern that matches the names of the indices that you want to explore. +When you add data with the File Data Visualizer, GeoJSON Upload feature, +or built-in tutorial, an index pattern is created for you. + +. Go to *Stack Management*, and then click *Index Patterns*. + +. Click *Create index pattern*. + +. Specify an index pattern that matches the name of one or more of your Elasticsearch indices. ++ +For example, an index pattern can point to your Apache data from yesterday, +`filebeat-apache-4-3-2022`, or any index that matches the pattern, `filebeat-*`. +Using a wildcard is the more popular approach. + + +. Click *Next Step*, and then select the index field that contains the timestamp you want to use to perform time-based +comparisons. ++ +Kibana reads the index mapping and lists all fields that contain a timestamp. If your +index doesn't have time-based data, choose *I don't want to use the Time Filter*. ++ +You must select a time field to use global time filters on your dashboards. + +. Click *Create index pattern*. ++ +{kib} is now configured to access your {es} indices. +You’ll see a list of fields configured for the matching index. +You can designate your index pattern as the default by clicking the star icon on this page. ++ +When searching in *Discover* and creating visualizations, you choose a pattern +from the index pattern menu to specify the {es} indices that contain the data you want to explore. diff --git a/packages/kbn-test/package.json b/packages/kbn-test/package.json index 0ab0048619358..8e2fd1c9182ff 100644 --- a/packages/kbn-test/package.json +++ b/packages/kbn-test/package.json @@ -14,6 +14,7 @@ "@kbn/babel-preset": "1.0.0", "@kbn/dev-utils": "1.0.0", "@types/parse-link-header": "^1.0.0", + "@types/puppeteer": "^3.0.0", "@types/strip-ansi": "^5.2.1", "@types/xml2js": "^0.4.5", "diff": "^4.0.1" @@ -25,6 +26,7 @@ "getopts": "^2.2.4", "glob": "^7.1.2", "parse-link-header": "^1.0.1", + "puppeteer": "^3.3.0", "strip-ansi": "^5.2.0", "rxjs": "^6.5.3", "tar-fs": "^1.16.3", diff --git a/packages/kbn-test/src/index.ts b/packages/kbn-test/src/index.ts index 3a938280a90c5..e6b180d979f17 100644 --- a/packages/kbn-test/src/index.ts +++ b/packages/kbn-test/src/index.ts @@ -51,3 +51,5 @@ export { runFailedTestsReporterCli } from './failed_tests_reporter'; export { makeJunitReportPath } from './junit_report_path'; export { CI_PARALLEL_PROCESS_PREFIX } from './ci_parallel_process_prefix'; + +export * from './page_load_metrics'; diff --git a/packages/kbn-test/src/page_load_metrics/capture_page_load_metrics.ts b/packages/kbn-test/src/page_load_metrics/capture_page_load_metrics.ts new file mode 100644 index 0000000000000..013d49a29a51c --- /dev/null +++ b/packages/kbn-test/src/page_load_metrics/capture_page_load_metrics.ts @@ -0,0 +1,81 @@ +/* + * Licensed to Elasticsearch B.V. under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch B.V. licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import { ToolingLog } from '@kbn/dev-utils'; +import { NavigationOptions, createUrl, navigateToApps } from './navigation'; + +export async function capturePageLoadMetrics(log: ToolingLog, options: NavigationOptions) { + const responsesByPageView = await navigateToApps(log, options); + + const assetSizeMeasurements = new Map(); + + const numberOfPagesVisited = responsesByPageView.size; + + for (const [, frameResponses] of responsesByPageView) { + for (const [, { url, dataLength }] of frameResponses) { + if (url.length === 0) { + throw new Error('navigateToApps(); failed to identify the url of the request'); + } + if (assetSizeMeasurements.has(url)) { + assetSizeMeasurements.set(url, [dataLength].concat(assetSizeMeasurements.get(url) || [])); + } else { + assetSizeMeasurements.set(url, [dataLength]); + } + } + } + + return Array.from(assetSizeMeasurements.entries()) + .map(([url, measurements]) => { + const baseUrl = createUrl('/', options.appConfig.url); + const relativeUrl = url + // remove the baseUrl (expect the trailing slash) to make url relative + .replace(baseUrl.slice(0, -1), '') + // strip the build number from asset urls + .replace(/^\/\d+\//, '/'); + return [relativeUrl, measurements] as const; + }) + .filter(([url, measurements]) => { + if (measurements.length !== numberOfPagesVisited) { + // ignore urls seen only on some pages + return false; + } + + if (url.startsWith('data:')) { + // ignore data urls since they are already counted by other assets + return false; + } + + if (url.startsWith('/api/') || url.startsWith('/internal/')) { + // ignore api requests since they don't have deterministic sizes + return false; + } + + const allMetricsAreEqual = measurements.every((x, i) => + i === 0 ? true : x === measurements[i - 1] + ); + if (!allMetricsAreEqual) { + throw new Error(`measurements for url [${url}] are not equal [${measurements.join(',')}]`); + } + + return true; + }) + .map(([url, measurements]) => { + return { group: 'page load asset size', id: url, value: measurements[0] }; + }); +} diff --git a/packages/kbn-test/src/page_load_metrics/cli.ts b/packages/kbn-test/src/page_load_metrics/cli.ts new file mode 100644 index 0000000000000..95421384c79cb --- /dev/null +++ b/packages/kbn-test/src/page_load_metrics/cli.ts @@ -0,0 +1,90 @@ +/* + * Licensed to Elasticsearch B.V. under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch B.V. licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import Url from 'url'; + +import { run, createFlagError } from '@kbn/dev-utils'; +import { resolve, basename } from 'path'; +import { capturePageLoadMetrics } from './capture_page_load_metrics'; + +const defaultScreenshotsDir = resolve(__dirname, 'screenshots'); + +export function runPageLoadMetricsCli() { + run( + async ({ flags, log }) => { + const kibanaUrl = flags['kibana-url']; + if (!kibanaUrl || typeof kibanaUrl !== 'string') { + throw createFlagError('Expect --kibana-url to be a string'); + } + + const parsedUrl = Url.parse(kibanaUrl); + + const [username, password] = parsedUrl.auth + ? parsedUrl.auth.split(':') + : [flags.username, flags.password]; + + if (typeof username !== 'string' || typeof password !== 'string') { + throw createFlagError( + 'Mising username and/or password, either specify in --kibana-url or pass --username and --password' + ); + } + + const headless = !flags.head; + + const screenshotsDir = flags.screenshotsDir || defaultScreenshotsDir; + + if (typeof screenshotsDir !== 'string' || screenshotsDir === basename(screenshotsDir)) { + throw createFlagError('Expect screenshotsDir to be valid path string'); + } + + const metrics = await capturePageLoadMetrics(log, { + headless, + appConfig: { + url: kibanaUrl, + username, + password, + }, + screenshotsDir, + }); + for (const metric of metrics) { + log.info(`${metric.id}: ${metric.value}`); + } + }, + { + description: `Loads several pages with Puppeteer to capture the size of assets`, + flags: { + string: ['kibana-url', 'username', 'password', 'screenshotsDir'], + boolean: ['head'], + default: { + username: 'elastic', + password: 'changeme', + debug: true, + screenshotsDir: defaultScreenshotsDir, + }, + help: ` + --kibana-url Url for Kibana we should connect to, can include login info + --head Run puppeteer with graphical user interface + --username Set username, defaults to 'elastic' + --password Set password, defaults to 'changeme' + --screenshotsDir Set screenshots directory, defaults to '${defaultScreenshotsDir}' + `, + }, + } + ); +} diff --git a/packages/kbn-test/src/page_load_metrics/event.ts b/packages/kbn-test/src/page_load_metrics/event.ts new file mode 100644 index 0000000000000..481954bbf672e --- /dev/null +++ b/packages/kbn-test/src/page_load_metrics/event.ts @@ -0,0 +1,34 @@ +/* + * Licensed to Elasticsearch B.V. under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch B.V. licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +export interface ResponseReceivedEvent { + frameId: string; + loaderId: string; + requestId: string; + response: Record; + timestamp: number; + type: string; +} + +export interface DataReceivedEvent { + encodedDataLength: number; + dataLength: number; + requestId: string; + timestamp: number; +} diff --git a/packages/kbn-test/src/page_load_metrics/index.ts b/packages/kbn-test/src/page_load_metrics/index.ts new file mode 100644 index 0000000000000..4309d558518a6 --- /dev/null +++ b/packages/kbn-test/src/page_load_metrics/index.ts @@ -0,0 +1,21 @@ +/* + * Licensed to Elasticsearch B.V. under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch B.V. licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +export * from './cli'; +export { capturePageLoadMetrics } from './capture_page_load_metrics'; diff --git a/packages/kbn-test/src/page_load_metrics/navigation.ts b/packages/kbn-test/src/page_load_metrics/navigation.ts new file mode 100644 index 0000000000000..fa1bbfbf9b129 --- /dev/null +++ b/packages/kbn-test/src/page_load_metrics/navigation.ts @@ -0,0 +1,167 @@ +/* + * Licensed to Elasticsearch B.V. under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch B.V. licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import Fs from 'fs'; +import Url from 'url'; +import _ from 'lodash'; +import puppeteer from 'puppeteer'; +import { resolve } from 'path'; +import { ToolingLog } from '@kbn/dev-utils'; +import { ResponseReceivedEvent, DataReceivedEvent } from './event'; + +export interface NavigationOptions { + headless: boolean; + appConfig: { url: string; username: string; password: string }; + screenshotsDir: string; +} + +export type NavigationResults = Map>; + +interface FrameResponse { + url: string; + dataLength: number; +} + +function joinPath(pathA: string, pathB: string) { + return `${pathA.endsWith('/') ? pathA.slice(0, -1) : pathA}/${ + pathB.startsWith('/') ? pathB.slice(1) : pathB + }`; +} + +export function createUrl(path: string, url: string) { + const baseUrl = Url.parse(url); + + return Url.format({ + protocol: baseUrl.protocol, + hostname: baseUrl.hostname, + port: baseUrl.port, + pathname: joinPath(baseUrl.pathname || '', path.includes('#') ? path.split('#')[0] : path), + hash: path.includes('#') ? path.split('#')[1] : undefined, + }); +} + +async function loginToKibana( + log: ToolingLog, + browser: puppeteer.Browser, + options: NavigationOptions +) { + log.debug(`log in to the app..`); + const page = await browser.newPage(); + const loginUrl = createUrl('/login', options.appConfig.url); + await page.goto(loginUrl, { + waitUntil: 'networkidle0', + }); + await page.type('[data-test-subj="loginUsername"]', options.appConfig.username); + await page.type('[data-test-subj="loginPassword"]', options.appConfig.password); + await page.click('[data-test-subj="loginSubmit"]'); + await page.waitForNavigation({ waitUntil: 'networkidle0' }); + await page.close(); +} + +export async function navigateToApps(log: ToolingLog, options: NavigationOptions) { + const browser = await puppeteer.launch({ headless: options.headless, args: ['--no-sandbox'] }); + const devToolsResponses: NavigationResults = new Map(); + const apps = [ + { path: '/app/kibana#/discover', locator: '[data-test-subj="discover-sidebar"]' }, + { path: '/app/kibana#/home', locator: '[data-test-subj="homeApp"]' }, + { path: '/app/canvas', locator: '[data-test-subj="create-workpad-button"]' }, + { path: '/app/maps', locator: '[title="Maps"]' }, + { path: '/app/apm', locator: '[data-test-subj="apmMainContainer"]' }, + ]; + + await loginToKibana(log, browser, options); + + await Promise.all( + apps.map(async (app) => { + const page = await browser.newPage(); + page.setCacheEnabled(false); + page.setDefaultNavigationTimeout(0); + const frameResponses = new Map(); + devToolsResponses.set(app.path, frameResponses); + + const client = await page.target().createCDPSession(); + await client.send('Network.enable'); + + function getRequestData(requestId: string) { + if (!frameResponses.has(requestId)) { + frameResponses.set(requestId, { url: '', dataLength: 0 }); + } + + return frameResponses.get(requestId)!; + } + + client.on('Network.responseReceived', (event: ResponseReceivedEvent) => { + getRequestData(event.requestId).url = event.response.url; + }); + + client.on('Network.dataReceived', (event: DataReceivedEvent) => { + getRequestData(event.requestId).dataLength += event.dataLength; + }); + + const url = createUrl(app.path, options.appConfig.url); + log.debug(`goto ${url}`); + await page.goto(url, { + waitUntil: 'networkidle0', + }); + + let readyAttempt = 0; + let selectorFound = false; + while (!selectorFound) { + readyAttempt += 1; + try { + await page.waitForSelector(app.locator, { timeout: 5000 }); + selectorFound = true; + } catch (error) { + log.error( + `Page '${app.path}' was not loaded properly, unable to find '${ + app.locator + }', url: ${page.url()}` + ); + + if (readyAttempt < 6) { + continue; + } + + const failureDir = resolve(options.screenshotsDir, 'failure'); + const screenshotPath = resolve( + failureDir, + `${app.path.slice(1).split(/#|\//).join('_')}_navigation.png` + ); + Fs.mkdirSync(failureDir, { recursive: true }); + + await page.bringToFront(); + await page.screenshot({ + path: screenshotPath, + type: 'png', + fullPage: true, + }); + log.debug(`Saving screenshot to ${screenshotPath}`); + + throw new Error(`Page load timeout: ${app.path} not loaded after 30 seconds`); + } + } + + await page.close(); + }) + ); + + await browser.close(); + + return devToolsResponses; +} diff --git a/scripts/page_load_metrics.js b/scripts/page_load_metrics.js new file mode 100644 index 0000000000000..37500c26e0b20 --- /dev/null +++ b/scripts/page_load_metrics.js @@ -0,0 +1,21 @@ +/* + * Licensed to Elasticsearch B.V. under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch B.V. licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +require('../src/setup_node_env'); +require('@kbn/test').runPageLoadMetricsCli(); diff --git a/test/functional/page_objects/common_page.ts b/test/functional/page_objects/common_page.ts index 35717f09f1599..e8f2da52834a6 100644 --- a/test/functional/page_objects/common_page.ts +++ b/test/functional/page_objects/common_page.ts @@ -67,17 +67,17 @@ export function CommonPageProvider({ getService, getPageObjects }: FtrProviderCo * @param appUrl Kibana URL */ private async loginIfPrompted(appUrl: string, insertTimestamp: boolean) { + // Disable the welcome screen. This is relevant for environments + // which don't allow to use the yml setting, e.g. cloud production. + // It is done here so it applies to logins but also to a login re-use. + await browser.setLocalStorageItem('home:welcome:show', 'false'); + let currentUrl = await browser.getCurrentUrl(); log.debug(`currentUrl = ${currentUrl}\n appUrl = ${appUrl}`); await testSubjects.find('kibanaChrome', 6 * defaultFindTimeout); // 60 sec waiting const loginPage = currentUrl.includes('/login'); const wantedLoginPage = appUrl.includes('/login') || appUrl.includes('/logout'); - // Disable the welcome screen. This is relevant for environments - // which don't allow to use the yml setting, e.g. cloud production. - // It is done here so it applies to logins but also to a login re-use. - await browser.setLocalStorageItem('home:welcome:show', 'false'); - if (loginPage && !wantedLoginPage) { log.debug('Found login page'); if (config.get('security.disableTestUser')) { diff --git a/test/scripts/jenkins_xpack_page_load_metrics.sh b/test/scripts/jenkins_xpack_page_load_metrics.sh new file mode 100644 index 0000000000000..679f0b8d2ddc5 --- /dev/null +++ b/test/scripts/jenkins_xpack_page_load_metrics.sh @@ -0,0 +1,9 @@ +#!/usr/bin/env bash + +source test/scripts/jenkins_test_setup_xpack.sh + +checks-reporter-with-killswitch "Capture Kibana page load metrics" \ + node scripts/functional_tests \ + --debug --bail \ + --kibana-install-dir "$installDir" \ + --config test/page_load_metrics/config.ts; diff --git a/x-pack/.gitignore b/x-pack/.gitignore index b9c237356cb8f..07be67e2376d0 100644 --- a/x-pack/.gitignore +++ b/x-pack/.gitignore @@ -3,6 +3,7 @@ /target /test/functional/failure_debug /test/functional/screenshots +/test/page_load_metrics/screenshots /test/functional/apps/reporting/reports/session /test/reporting/configs/failure_debug/ /legacy/plugins/reporting/.chromium/ diff --git a/x-pack/plugins/actions/server/builtin_action_types/case/types.ts b/x-pack/plugins/actions/server/builtin_action_types/case/types.ts index 459e9d2b03f92..992b2cb16fb06 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/case/types.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/case/types.ts @@ -21,6 +21,7 @@ import { ExecutorSubActionGetIncidentParamsSchema, ExecutorSubActionHandshakeParamsSchema, } from './schema'; +import { LicenseType } from '../../../../../legacy/common/constants'; export interface AnyParams { [index: string]: string | number | object | undefined | null; @@ -51,6 +52,7 @@ export type Comment = TypeOf; export interface ExternalServiceConfiguration { id: string; name: string; + minimumLicenseRequired: LicenseType; } export interface ExternalServiceCredentials { diff --git a/x-pack/plugins/actions/server/builtin_action_types/case/utils.ts b/x-pack/plugins/actions/server/builtin_action_types/case/utils.ts index 315d13b5aa773..dd8d971b7df44 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/case/utils.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/case/utils.ts @@ -120,9 +120,7 @@ export const createConnector = ({ configurationUtilities, executor = createConnectorExecutor({ api, createExternalService }), }: CreateActionTypeArgs): ActionType => ({ - id: config.id, - name: config.name, - minimumLicenseRequired: 'platinum', + ...config, validate: { config: schema.object(validationSchema.config, { validate: curry(validate.config)(configurationUtilities), diff --git a/x-pack/plugins/actions/server/builtin_action_types/jira/config.ts b/x-pack/plugins/actions/server/builtin_action_types/jira/config.ts index 7e415109f1bd9..54f28e447010a 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/jira/config.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/jira/config.ts @@ -10,4 +10,5 @@ import * as i18n from './translations'; export const config: ExternalServiceConfiguration = { id: '.jira', name: i18n.NAME, + minimumLicenseRequired: 'gold', }; diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/config.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/config.ts index 4ad8108c3b137..70d53ab79f631 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/config.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/config.ts @@ -10,4 +10,5 @@ import * as i18n from './translations'; export const config: ExternalServiceConfiguration = { id: '.servicenow', name: i18n.NAME, + minimumLicenseRequired: 'platinum', }; diff --git a/x-pack/plugins/siem/public/components/timeline/helpers.test.tsx b/x-pack/plugins/siem/public/components/timeline/helpers.test.tsx index 4d8d4097e92fc..4b9127a503ed1 100644 --- a/x-pack/plugins/siem/public/components/timeline/helpers.test.tsx +++ b/x-pack/plugins/siem/public/components/timeline/helpers.test.tsx @@ -18,7 +18,7 @@ const endDate = new Date('2018-03-24T03:33:52.253Z').valueOf(); describe('Build KQL Query', () => { test('Build KQL query with one data provider', () => { - const dataProviders = mockDataProviders.slice(0, 1); + const dataProviders = cloneDeep(mockDataProviders.slice(0, 1)); const kqlQuery = buildGlobalQuery(dataProviders, mockBrowserFields); expect(cleanUpKqlQuery(kqlQuery)).toEqual('name : "Provider 1"'); }); @@ -56,18 +56,40 @@ describe('Build KQL Query', () => { }); test('Build KQL query with two data provider', () => { - const dataProviders = mockDataProviders.slice(0, 2); + const dataProviders = cloneDeep(mockDataProviders.slice(0, 2)); + const kqlQuery = buildGlobalQuery(dataProviders, mockBrowserFields); + expect(cleanUpKqlQuery(kqlQuery)).toEqual('(name : "Provider 1") or (name : "Provider 2")'); + }); + + test('Build KQL query with two data provider and first is disabled', () => { + const dataProviders = cloneDeep(mockDataProviders.slice(0, 2)); + dataProviders[0].enabled = false; + const kqlQuery = buildGlobalQuery(dataProviders, mockBrowserFields); + expect(cleanUpKqlQuery(kqlQuery)).toEqual('name : "Provider 2"'); + }); + + test('Build KQL query with two data provider and second is disabled', () => { + const dataProviders = cloneDeep(mockDataProviders.slice(0, 2)); + dataProviders[1].enabled = false; const kqlQuery = buildGlobalQuery(dataProviders, mockBrowserFields); - expect(cleanUpKqlQuery(kqlQuery)).toEqual('(name : "Provider 1" ) or (name : "Provider 2" )'); + expect(cleanUpKqlQuery(kqlQuery)).toEqual('name : "Provider 1"'); }); test('Build KQL query with one data provider and one and', () => { const dataProviders = cloneDeep(mockDataProviders.slice(0, 1)); - dataProviders[0].and = mockDataProviders.slice(1, 2); + dataProviders[0].and = cloneDeep(mockDataProviders.slice(1, 2)); const kqlQuery = buildGlobalQuery(dataProviders, mockBrowserFields); expect(cleanUpKqlQuery(kqlQuery)).toEqual('name : "Provider 1" and name : "Provider 2"'); }); + test('Build KQL query with one disabled data provider and one and', () => { + const dataProviders = cloneDeep(mockDataProviders.slice(0, 1)); + dataProviders[0].enabled = false; + dataProviders[0].and = cloneDeep(mockDataProviders.slice(1, 2)); + const kqlQuery = buildGlobalQuery(dataProviders, mockBrowserFields); + expect(cleanUpKqlQuery(kqlQuery)).toEqual('name : "Provider 2"'); + }); + test('Build KQL query with one data provider and one and as timestamp (string input)', () => { const dataProviders = cloneDeep(mockDataProviders.slice(0, 1)); dataProviders[0].and = cloneDeep(mockDataProviders.slice(1, 2)); @@ -106,28 +128,50 @@ describe('Build KQL Query', () => { test('Build KQL query with two data provider and multiple and', () => { const dataProviders = cloneDeep(mockDataProviders.slice(0, 2)); - dataProviders[0].and = mockDataProviders.slice(2, 4); - dataProviders[1].and = mockDataProviders.slice(4, 5); + dataProviders[0].and = cloneDeep(mockDataProviders.slice(2, 4)); + dataProviders[1].and = cloneDeep(mockDataProviders.slice(4, 5)); const kqlQuery = buildGlobalQuery(dataProviders, mockBrowserFields); expect(cleanUpKqlQuery(kqlQuery)).toEqual( '(name : "Provider 1" and name : "Provider 3" and name : "Provider 4") or (name : "Provider 2" and name : "Provider 5")' ); }); + test('Build KQL query with two data provider and multiple and and first data provider is disabled', () => { + const dataProviders = cloneDeep(mockDataProviders.slice(0, 2)); + dataProviders[0].enabled = false; + dataProviders[0].and = cloneDeep(mockDataProviders.slice(2, 4)); + dataProviders[1].and = cloneDeep(mockDataProviders.slice(4, 5)); + const kqlQuery = buildGlobalQuery(dataProviders, mockBrowserFields); + expect(cleanUpKqlQuery(kqlQuery)).toEqual( + '(name : "Provider 3" and name : "Provider 4") or (name : "Provider 2" and name : "Provider 5")' + ); + }); + + test('Build KQL query with two data provider and multiple and and first and provider is disabled', () => { + const dataProviders = cloneDeep(mockDataProviders.slice(0, 2)); + dataProviders[0].and = cloneDeep(mockDataProviders.slice(2, 4)); + dataProviders[0].and[0].enabled = false; + dataProviders[1].and = cloneDeep(mockDataProviders.slice(4, 5)); + const kqlQuery = buildGlobalQuery(dataProviders, mockBrowserFields); + expect(cleanUpKqlQuery(kqlQuery)).toEqual( + '(name : "Provider 1" and name : "Provider 4") or (name : "Provider 2" and name : "Provider 5")' + ); + }); + test('Build KQL query with all data provider', () => { const kqlQuery = buildGlobalQuery(mockDataProviders, mockBrowserFields); expect(cleanUpKqlQuery(kqlQuery)).toEqual( - '(name : "Provider 1" ) or (name : "Provider 2" ) or (name : "Provider 3" ) or (name : "Provider 4" ) or (name : "Provider 5" ) or (name : "Provider 6" ) or (name : "Provider 7" ) or (name : "Provider 8" ) or (name : "Provider 9" ) or (name : "Provider 10" )' + '(name : "Provider 1") or (name : "Provider 2") or (name : "Provider 3") or (name : "Provider 4") or (name : "Provider 5") or (name : "Provider 6") or (name : "Provider 7") or (name : "Provider 8") or (name : "Provider 9") or (name : "Provider 10")' ); }); test('Build complex KQL query with and and or', () => { const dataProviders = cloneDeep(mockDataProviders); - dataProviders[0].and = mockDataProviders.slice(2, 4); - dataProviders[1].and = mockDataProviders.slice(4, 5); + dataProviders[0].and = cloneDeep(mockDataProviders.slice(2, 4)); + dataProviders[1].and = cloneDeep(mockDataProviders.slice(4, 5)); const kqlQuery = buildGlobalQuery(dataProviders, mockBrowserFields); expect(cleanUpKqlQuery(kqlQuery)).toEqual( - '(name : "Provider 1" and name : "Provider 3" and name : "Provider 4") or (name : "Provider 2" and name : "Provider 5") or (name : "Provider 3" ) or (name : "Provider 4" ) or (name : "Provider 5" ) or (name : "Provider 6" ) or (name : "Provider 7" ) or (name : "Provider 8" ) or (name : "Provider 9" ) or (name : "Provider 10" )' + '(name : "Provider 1" and name : "Provider 3" and name : "Provider 4") or (name : "Provider 2" and name : "Provider 5") or (name : "Provider 3") or (name : "Provider 4") or (name : "Provider 5") or (name : "Provider 6") or (name : "Provider 7") or (name : "Provider 8") or (name : "Provider 9") or (name : "Provider 10")' ); }); }); @@ -223,7 +267,7 @@ describe('Combined Queries', () => { }); test('Only Data Provider', () => { - const dataProviders = mockDataProviders.slice(0, 1); + const dataProviders = cloneDeep(mockDataProviders.slice(0, 1)); const { filterQuery } = combineQueries({ config, dataProviders, @@ -338,7 +382,7 @@ describe('Combined Queries', () => { }); test('Data Provider & KQL search query', () => { - const dataProviders = mockDataProviders.slice(0, 1); + const dataProviders = cloneDeep(mockDataProviders.slice(0, 1)); const { filterQuery } = combineQueries({ config, dataProviders, @@ -356,7 +400,7 @@ describe('Combined Queries', () => { }); test('Data Provider & KQL filter query', () => { - const dataProviders = mockDataProviders.slice(0, 1); + const dataProviders = cloneDeep(mockDataProviders.slice(0, 1)); const { filterQuery } = combineQueries({ config, dataProviders, @@ -375,8 +419,8 @@ describe('Combined Queries', () => { test('Data Provider & KQL search query multiple', () => { const dataProviders = cloneDeep(mockDataProviders.slice(0, 2)); - dataProviders[0].and = mockDataProviders.slice(2, 4); - dataProviders[1].and = mockDataProviders.slice(4, 5); + dataProviders[0].and = cloneDeep(mockDataProviders.slice(2, 4)); + dataProviders[1].and = cloneDeep(mockDataProviders.slice(4, 5)); const { filterQuery } = combineQueries({ config, dataProviders, @@ -395,8 +439,8 @@ describe('Combined Queries', () => { test('Data Provider & KQL filter query multiple', () => { const dataProviders = cloneDeep(mockDataProviders.slice(0, 2)); - dataProviders[0].and = mockDataProviders.slice(2, 4); - dataProviders[1].and = mockDataProviders.slice(4, 5); + dataProviders[0].and = cloneDeep(mockDataProviders.slice(2, 4)); + dataProviders[1].and = cloneDeep(mockDataProviders.slice(4, 5)); const { filterQuery } = combineQueries({ config, dataProviders, diff --git a/x-pack/plugins/siem/public/components/timeline/helpers.tsx b/x-pack/plugins/siem/public/components/timeline/helpers.tsx index 603f6ae7c3487..b0e8572e08623 100644 --- a/x-pack/plugins/siem/public/components/timeline/helpers.tsx +++ b/x-pack/plugins/siem/public/components/timeline/helpers.tsx @@ -63,35 +63,30 @@ const buildQueryMatch = ( : `${dataProvider.queryMatch.field} ${EXISTS_OPERATOR}` }`.trim(); -const buildQueryForAndProvider = ( - dataAndProviders: DataProvidersAnd[], - browserFields: BrowserFields -) => - dataAndProviders - .reduce((andQuery, andDataProvider) => { - const prepend = (q: string) => `${q !== '' ? `${q} and ` : ''}`; - return andDataProvider.enabled - ? `${prepend(andQuery)} ${buildQueryMatch(andDataProvider, browserFields)}` - : andQuery; - }, '') - .trim(); - export const buildGlobalQuery = (dataProviders: DataProvider[], browserFields: BrowserFields) => dataProviders - .reduce((query, dataProvider: DataProvider, i) => { - const prepend = (q: string) => `${q !== '' ? `${q} or ` : ''}`; - const openParen = i >= 0 && dataProviders.length > 1 ? '(' : ''; - const closeParen = i >= 0 && dataProviders.length > 1 ? ')' : ''; - return dataProvider.enabled - ? `${prepend(query)}${openParen}${buildQueryMatch(dataProvider, browserFields)} - ${ - dataProvider.and.length > 0 - ? ` and ${buildQueryForAndProvider(dataProvider.and, browserFields)}` - : '' - }${closeParen}`.trim() - : query; - }, '') - .trim(); + .reduce((queries: string[], dataProvider: DataProvider) => { + const flatDataProviders = [dataProvider, ...dataProvider.and]; + const activeDataProviders = flatDataProviders.filter( + (flatDataProvider) => flatDataProvider.enabled + ); + + if (!activeDataProviders.length) return queries; + + const activeDataProvidersQueries = activeDataProviders.map((activeDataProvider) => + buildQueryMatch(activeDataProvider, browserFields) + ); + + const activeDataProvidersQueryMatch = activeDataProvidersQueries.join(' and '); + + return [...queries, activeDataProvidersQueryMatch]; + }, []) + .filter((queriesItem) => !isEmpty(queriesItem)) + .reduce((globalQuery: string, queryMatch: string, index: number, queries: string[]) => { + if (queries.length <= 1) return queryMatch; + + return !index ? `(${queryMatch})` : `${globalQuery} or (${queryMatch})`; + }, ''); export const combineQueries = ({ config, diff --git a/x-pack/plugins/siem/public/pages/detection_engine/rules/components/step_rule_actions/index.tsx b/x-pack/plugins/siem/public/pages/detection_engine/rules/components/step_rule_actions/index.tsx index a9f68e2c8a6d1..9f6ea44c5dffb 100644 --- a/x-pack/plugins/siem/public/pages/detection_engine/rules/components/step_rule_actions/index.tsx +++ b/x-pack/plugins/siem/public/pages/detection_engine/rules/components/step_rule_actions/index.tsx @@ -158,7 +158,6 @@ const StepRuleActionsComponent: FC = ({ {myStepData.throttle !== stepActionsDefaultValue.throttle ? ( <> - = ({ messageVariables: actionMessageParams, }} /> - ) : ( = ({ component={GhostFormField} /> )} + diff --git a/x-pack/plugins/siem/server/lib/detection_engine/rules/update_rules_notifications.ts b/x-pack/plugins/siem/server/lib/detection_engine/rules/update_rules_notifications.ts index c5cf85d7ba014..ddcd34b18cae9 100644 --- a/x-pack/plugins/siem/server/lib/detection_engine/rules/update_rules_notifications.ts +++ b/x-pack/plugins/siem/server/lib/detection_engine/rules/update_rules_notifications.ts @@ -45,8 +45,5 @@ export const updateRulesNotifications = async ({ interval: ruleActions.alertThrottle, }); - // TODO: Workaround for https://github.com/elastic/kibana/issues/67290 - await alertsClient.updateApiKey({ id: ruleAlertId }); - return ruleActions; }; diff --git a/x-pack/test/page_load_metrics/config.ts b/x-pack/test/page_load_metrics/config.ts new file mode 100644 index 0000000000000..641099ff8e934 --- /dev/null +++ b/x-pack/test/page_load_metrics/config.ts @@ -0,0 +1,42 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { resolve } from 'path'; + +import { FtrConfigProviderContext } from '@kbn/test/types/ftr'; +import { PuppeteerTestRunner } from './runner'; + +export default async function ({ readConfigFile }: FtrConfigProviderContext) { + const kibanaCommonTestsConfig = await readConfigFile( + require.resolve('../../../test/common/config.js') + ); + const xpackFunctionalTestsConfig = await readConfigFile( + require.resolve('../functional/config.js') + ); + + return { + ...kibanaCommonTestsConfig.getAll(), + + testRunner: PuppeteerTestRunner, + + esArchiver: { + directory: resolve(__dirname, 'es_archives'), + }, + + screenshots: { + directory: resolve(__dirname, 'screenshots'), + }, + + esTestCluster: { + ...xpackFunctionalTestsConfig.get('esTestCluster'), + serverArgs: [...xpackFunctionalTestsConfig.get('esTestCluster.serverArgs')], + }, + + kbnTestServer: { + ...xpackFunctionalTestsConfig.get('kbnTestServer'), + }, + }; +} diff --git a/x-pack/test/page_load_metrics/es_archives/default/data.json.gz b/x-pack/test/page_load_metrics/es_archives/default/data.json.gz new file mode 100644 index 0000000000000..5a5290ddf6447 Binary files /dev/null and b/x-pack/test/page_load_metrics/es_archives/default/data.json.gz differ diff --git a/x-pack/test/page_load_metrics/es_archives/default/mappings.json b/x-pack/test/page_load_metrics/es_archives/default/mappings.json new file mode 100644 index 0000000000000..c36f9576c4df1 --- /dev/null +++ b/x-pack/test/page_load_metrics/es_archives/default/mappings.json @@ -0,0 +1,2402 @@ +{ + "type": "index", + "value": { + "aliases": { + ".kibana": { + } + }, + "index": ".kibana_1", + "mappings": { + "_meta": { + "migrationMappingPropertyHashes": { + "action": "6e96ac5e648f57523879661ea72525b7", + "action_task_params": "a9d49f184ee89641044be0ca2950fa3a", + "alert": "7b44fba6773e37c806ce290ea9b7024e", + "apm-indices": "9bb9b2bf1fa636ed8619cbab5ce6a1dd", + "apm-telemetry": "3525d7c22c42bc80f5e6e9cb3f2b26a2", + "application_usage_totals": "c897e4310c5f24b07caaff3db53ae2c1", + "application_usage_transactional": "965839e75f809fefe04f92dc4d99722a", + "canvas-element": "7390014e1091044523666d97247392fc", + "canvas-workpad": "b0a1706d356228dbdcb4a17e6b9eb231", + "cases": "32aa96a6d3855ddda53010ae2048ac22", + "cases-comments": "c2061fb929f585df57425102fa928b4b", + "cases-configure": "42711cbb311976c0687853f4c1354572", + "cases-user-actions": "32277330ec6b721abe3b846cfd939a71", + "config": "ae24d22d5986d04124cc6568f771066f", + "dashboard": "d00f614b29a80360e1190193fd333bab", + "file-upload-telemetry": "0ed4d3e1983d1217a30982630897092e", + "graph-workspace": "cd7ba1330e6682e9cc00b78850874be1", + "index-pattern": "66eccb05066c5a89924f48a9e9736499", + "infrastructure-ui-source": "ddc0ecb18383f6b26101a2fadb2dab0c", + "inventory-view": "88fc7e12fd1b45b6f0787323ce4f18d2", + "kql-telemetry": "d12a98a6f19a2d273696597547e064ee", + "lens": "21c3ea0763beb1ecb0162529706b88c5", + "lens-ui-telemetry": "509bfa5978586998e05f9e303c07a327", + "map": "23d7aa4a720d4938ccde3983f87bd58d", + "maps-telemetry": "bfd39d88aadadb4be597ea984d433dbe", + "metrics-explorer-view": "428e319af3e822c80a84cf87123ca35c", + "migrationVersion": "4a1746014a75ade3a714e1db5763276f", + "ml-telemetry": "257fd1d4b4fdbb9cb4b8a3b27da201e9", + "namespace": "2f4316de49999235636386fe51dc06c1", + "namespaces": "2f4316de49999235636386fe51dc06c1", + "query": "11aaeb7f5f7fa5bb43f25e18ce26e7d9", + "references": "7997cf5a56cc02bdc9c93361bde732b0", + "sample-data-telemetry": "7d3cfeb915303c9641c59681967ffeb4", + "search": "181661168bbadd1eff5902361e2a0d5c", + "space": "c5ca8acafa0beaa4d08d014a97b6bc6b", + "telemetry": "36a616f7026dfa617d6655df850fe16d", + "timelion-sheet": "9a2a2748877c7a7b582fef201ab1d4cf", + "tsvb-validation-telemetry": "3a37ef6c8700ae6fc97d5c7da00e9215", + "type": "2f4316de49999235636386fe51dc06c1", + "ui-metric": "0d409297dc5ebe1e3a1da691c6ee32e3", + "updated_at": "00da57df13e94e9d98437d13ace4bfe0", + "upgrade-assistant-reindex-operation": "296a89039fc4260292be36b1b005d8f2", + "upgrade-assistant-telemetry": "56702cec857e0a9dacfb696655b4ff7b", + "uptime-dynamic-settings": "fcdb453a30092f022f2642db29523d80", + "url": "b675c3be8d76ecf029294d51dc7ec65d", + "visualization": "52d7a13ad68a150c4525b292d23e12cc" + } + }, + "dynamic": "strict", + "properties": { + "action": { + "properties": { + "actionTypeId": { + "type": "keyword" + }, + "config": { + "enabled": false, + "type": "object" + }, + "name": { + "fields": { + "keyword": { + "type": "keyword" + } + }, + "type": "text" + }, + "secrets": { + "type": "binary" + } + } + }, + "action_task_params": { + "properties": { + "actionId": { + "type": "keyword" + }, + "apiKey": { + "type": "binary" + }, + "params": { + "enabled": false, + "type": "object" + } + } + }, + "alert": { + "properties": { + "actions": { + "properties": { + "actionRef": { + "type": "keyword" + }, + "actionTypeId": { + "type": "keyword" + }, + "group": { + "type": "keyword" + }, + "params": { + "enabled": false, + "type": "object" + } + }, + "type": "nested" + }, + "alertTypeId": { + "type": "keyword" + }, + "apiKey": { + "type": "binary" + }, + "apiKeyOwner": { + "type": "keyword" + }, + "consumer": { + "type": "keyword" + }, + "createdAt": { + "type": "date" + }, + "createdBy": { + "type": "keyword" + }, + "enabled": { + "type": "boolean" + }, + "muteAll": { + "type": "boolean" + }, + "mutedInstanceIds": { + "type": "keyword" + }, + "name": { + "fields": { + "keyword": { + "type": "keyword" + } + }, + "type": "text" + }, + "params": { + "enabled": false, + "type": "object" + }, + "schedule": { + "properties": { + "interval": { + "type": "keyword" + } + } + }, + "scheduledTaskId": { + "type": "keyword" + }, + "tags": { + "type": "keyword" + }, + "throttle": { + "type": "keyword" + }, + "updatedBy": { + "type": "keyword" + } + } + }, + "apm-indices": { + "properties": { + "apm_oss": { + "properties": { + "errorIndices": { + "type": "keyword" + }, + "metricsIndices": { + "type": "keyword" + }, + "onboardingIndices": { + "type": "keyword" + }, + "sourcemapIndices": { + "type": "keyword" + }, + "spanIndices": { + "type": "keyword" + }, + "transactionIndices": { + "type": "keyword" + } + } + } + } + }, + "apm-telemetry": { + "properties": { + "agents": { + "properties": { + "dotnet": { + "properties": { + "agent": { + "properties": { + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "service": { + "properties": { + "framework": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "language": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "runtime": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + } + } + } + } + }, + "go": { + "properties": { + "agent": { + "properties": { + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "service": { + "properties": { + "framework": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "language": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "runtime": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + } + } + } + } + }, + "java": { + "properties": { + "agent": { + "properties": { + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "service": { + "properties": { + "framework": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "language": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "runtime": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + } + } + } + } + }, + "js-base": { + "properties": { + "agent": { + "properties": { + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "service": { + "properties": { + "framework": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "language": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "runtime": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + } + } + } + } + }, + "nodejs": { + "properties": { + "agent": { + "properties": { + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "service": { + "properties": { + "framework": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "language": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "runtime": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + } + } + } + } + }, + "python": { + "properties": { + "agent": { + "properties": { + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "service": { + "properties": { + "framework": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "language": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "runtime": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + } + } + } + } + }, + "ruby": { + "properties": { + "agent": { + "properties": { + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "service": { + "properties": { + "framework": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "language": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "runtime": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + } + } + } + } + }, + "rum-js": { + "properties": { + "agent": { + "properties": { + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "service": { + "properties": { + "framework": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "language": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + }, + "runtime": { + "properties": { + "composite": { + "ignore_above": 1024, + "type": "keyword" + }, + "name": { + "ignore_above": 1024, + "type": "keyword" + }, + "version": { + "ignore_above": 1024, + "type": "keyword" + } + } + } + } + } + } + } + } + }, + "cardinality": { + "properties": { + "transaction": { + "properties": { + "name": { + "properties": { + "all_agents": { + "properties": { + "1d": { + "type": "long" + } + } + }, + "rum": { + "properties": { + "1d": { + "type": "long" + } + } + } + } + } + } + }, + "user_agent": { + "properties": { + "original": { + "properties": { + "all_agents": { + "properties": { + "1d": { + "type": "long" + } + } + }, + "rum": { + "properties": { + "1d": { + "type": "long" + } + } + } + } + } + } + } + } + }, + "counts": { + "properties": { + "agent_configuration": { + "properties": { + "all": { + "type": "long" + } + } + }, + "error": { + "properties": { + "1d": { + "type": "long" + }, + "all": { + "type": "long" + } + } + }, + "max_error_groups_per_service": { + "properties": { + "1d": { + "type": "long" + } + } + }, + "max_transaction_groups_per_service": { + "properties": { + "1d": { + "type": "long" + } + } + }, + "metric": { + "properties": { + "1d": { + "type": "long" + }, + "all": { + "type": "long" + } + } + }, + "onboarding": { + "properties": { + "1d": { + "type": "long" + }, + "all": { + "type": "long" + } + } + }, + "services": { + "properties": { + "1d": { + "type": "long" + } + } + }, + "sourcemap": { + "properties": { + "1d": { + "type": "long" + }, + "all": { + "type": "long" + } + } + }, + "span": { + "properties": { + "1d": { + "type": "long" + }, + "all": { + "type": "long" + } + } + }, + "traces": { + "properties": { + "1d": { + "type": "long" + } + } + }, + "transaction": { + "properties": { + "1d": { + "type": "long" + }, + "all": { + "type": "long" + } + } + } + } + }, + "has_any_services": { + "type": "boolean" + }, + "indices": { + "properties": { + "all": { + "properties": { + "total": { + "properties": { + "docs": { + "properties": { + "count": { + "type": "long" + } + } + }, + "store": { + "properties": { + "size_in_bytes": { + "type": "long" + } + } + } + } + } + } + }, + "shards": { + "properties": { + "total": { + "type": "long" + } + } + } + } + }, + "integrations": { + "properties": { + "ml": { + "properties": { + "all_jobs_count": { + "type": "long" + } + } + } + } + }, + "retainment": { + "properties": { + "error": { + "properties": { + "ms": { + "type": "long" + } + } + }, + "metric": { + "properties": { + "ms": { + "type": "long" + } + } + }, + "onboarding": { + "properties": { + "ms": { + "type": "long" + } + } + }, + "span": { + "properties": { + "ms": { + "type": "long" + } + } + }, + "transaction": { + "properties": { + "ms": { + "type": "long" + } + } + } + } + }, + "services_per_agent": { + "properties": { + "dotnet": { + "null_value": 0, + "type": "long" + }, + "go": { + "null_value": 0, + "type": "long" + }, + "java": { + "null_value": 0, + "type": "long" + }, + "js-base": { + "null_value": 0, + "type": "long" + }, + "nodejs": { + "null_value": 0, + "type": "long" + }, + "python": { + "null_value": 0, + "type": "long" + }, + "ruby": { + "null_value": 0, + "type": "long" + }, + "rum-js": { + "null_value": 0, + "type": "long" + } + } + }, + "tasks": { + "properties": { + "agent_configuration": { + "properties": { + "took": { + "properties": { + "ms": { + "type": "long" + } + } + } + } + }, + "agents": { + "properties": { + "took": { + "properties": { + "ms": { + "type": "long" + } + } + } + } + }, + "cardinality": { + "properties": { + "took": { + "properties": { + "ms": { + "type": "long" + } + } + } + } + }, + "groupings": { + "properties": { + "took": { + "properties": { + "ms": { + "type": "long" + } + } + } + } + }, + "indices_stats": { + "properties": { + "took": { + "properties": { + "ms": { + "type": "long" + } + } + } + } + }, + "integrations": { + "properties": { + "took": { + "properties": { + "ms": { + "type": "long" + } + } + } + } + }, + "processor_events": { + "properties": { + "took": { + "properties": { + "ms": { + "type": "long" + } + } + } + } + }, + "services": { + "properties": { + "took": { + "properties": { + "ms": { + "type": "long" + } + } + } + } + }, + "versions": { + "properties": { + "took": { + "properties": { + "ms": { + "type": "long" + } + } + } + } + } + } + }, + "version": { + "properties": { + "apm_server": { + "properties": { + "major": { + "type": "long" + }, + "minor": { + "type": "long" + }, + "patch": { + "type": "long" + } + } + } + } + } + } + }, + "application_usage_totals": { + "properties": { + "appId": { + "type": "keyword" + }, + "minutesOnScreen": { + "type": "float" + }, + "numberOfClicks": { + "type": "long" + } + } + }, + "application_usage_transactional": { + "properties": { + "appId": { + "type": "keyword" + }, + "minutesOnScreen": { + "type": "float" + }, + "numberOfClicks": { + "type": "long" + }, + "timestamp": { + "type": "date" + } + } + }, + "canvas-element": { + "dynamic": "false", + "properties": { + "@created": { + "type": "date" + }, + "@timestamp": { + "type": "date" + }, + "content": { + "type": "text" + }, + "help": { + "type": "text" + }, + "image": { + "type": "text" + }, + "name": { + "fields": { + "keyword": { + "type": "keyword" + } + }, + "type": "text" + } + } + }, + "canvas-workpad": { + "dynamic": "false", + "properties": { + "@created": { + "type": "date" + }, + "@timestamp": { + "type": "date" + }, + "name": { + "fields": { + "keyword": { + "type": "keyword" + } + }, + "type": "text" + } + } + }, + "cases": { + "properties": { + "closed_at": { + "type": "date" + }, + "closed_by": { + "properties": { + "email": { + "type": "keyword" + }, + "full_name": { + "type": "keyword" + }, + "username": { + "type": "keyword" + } + } + }, + "connector_id": { + "type": "keyword" + }, + "created_at": { + "type": "date" + }, + "created_by": { + "properties": { + "email": { + "type": "keyword" + }, + "full_name": { + "type": "keyword" + }, + "username": { + "type": "keyword" + } + } + }, + "description": { + "type": "text" + }, + "external_service": { + "properties": { + "connector_id": { + "type": "keyword" + }, + "connector_name": { + "type": "keyword" + }, + "external_id": { + "type": "keyword" + }, + "external_title": { + "type": "text" + }, + "external_url": { + "type": "text" + }, + "pushed_at": { + "type": "date" + }, + "pushed_by": { + "properties": { + "email": { + "type": "keyword" + }, + "full_name": { + "type": "keyword" + }, + "username": { + "type": "keyword" + } + } + } + } + }, + "status": { + "type": "keyword" + }, + "tags": { + "type": "keyword" + }, + "title": { + "type": "keyword" + }, + "updated_at": { + "type": "date" + }, + "updated_by": { + "properties": { + "email": { + "type": "keyword" + }, + "full_name": { + "type": "keyword" + }, + "username": { + "type": "keyword" + } + } + } + } + }, + "cases-comments": { + "properties": { + "comment": { + "type": "text" + }, + "created_at": { + "type": "date" + }, + "created_by": { + "properties": { + "email": { + "type": "keyword" + }, + "full_name": { + "type": "keyword" + }, + "username": { + "type": "keyword" + } + } + }, + "pushed_at": { + "type": "date" + }, + "pushed_by": { + "properties": { + "email": { + "type": "keyword" + }, + "full_name": { + "type": "keyword" + }, + "username": { + "type": "keyword" + } + } + }, + "updated_at": { + "type": "date" + }, + "updated_by": { + "properties": { + "email": { + "type": "keyword" + }, + "full_name": { + "type": "keyword" + }, + "username": { + "type": "keyword" + } + } + } + } + }, + "cases-configure": { + "properties": { + "closure_type": { + "type": "keyword" + }, + "connector_id": { + "type": "keyword" + }, + "connector_name": { + "type": "keyword" + }, + "created_at": { + "type": "date" + }, + "created_by": { + "properties": { + "email": { + "type": "keyword" + }, + "full_name": { + "type": "keyword" + }, + "username": { + "type": "keyword" + } + } + }, + "updated_at": { + "type": "date" + }, + "updated_by": { + "properties": { + "email": { + "type": "keyword" + }, + "full_name": { + "type": "keyword" + }, + "username": { + "type": "keyword" + } + } + } + } + }, + "cases-user-actions": { + "properties": { + "action": { + "type": "keyword" + }, + "action_at": { + "type": "date" + }, + "action_by": { + "properties": { + "email": { + "type": "keyword" + }, + "full_name": { + "type": "keyword" + }, + "username": { + "type": "keyword" + } + } + }, + "action_field": { + "type": "keyword" + }, + "new_value": { + "type": "text" + }, + "old_value": { + "type": "text" + } + } + }, + "config": { + "dynamic": "true", + "properties": { + "buildNum": { + "type": "keyword" + }, + "defaultIndex": { + "fields": { + "keyword": { + "ignore_above": 256, + "type": "keyword" + } + }, + "type": "text" + } + } + }, + "dashboard": { + "properties": { + "description": { + "type": "text" + }, + "hits": { + "type": "integer" + }, + "kibanaSavedObjectMeta": { + "properties": { + "searchSourceJSON": { + "type": "text" + } + } + }, + "optionsJSON": { + "type": "text" + }, + "panelsJSON": { + "type": "text" + }, + "refreshInterval": { + "properties": { + "display": { + "type": "keyword" + }, + "pause": { + "type": "boolean" + }, + "section": { + "type": "integer" + }, + "value": { + "type": "integer" + } + } + }, + "timeFrom": { + "type": "keyword" + }, + "timeRestore": { + "type": "boolean" + }, + "timeTo": { + "type": "keyword" + }, + "title": { + "type": "text" + }, + "version": { + "type": "integer" + } + } + }, + "file-upload-telemetry": { + "properties": { + "filesUploadedTotalCount": { + "type": "long" + } + } + }, + "graph-workspace": { + "properties": { + "description": { + "type": "text" + }, + "kibanaSavedObjectMeta": { + "properties": { + "searchSourceJSON": { + "type": "text" + } + } + }, + "numLinks": { + "type": "integer" + }, + "numVertices": { + "type": "integer" + }, + "title": { + "type": "text" + }, + "version": { + "type": "integer" + }, + "wsState": { + "type": "text" + } + } + }, + "index-pattern": { + "properties": { + "fieldFormatMap": { + "type": "text" + }, + "fields": { + "type": "text" + }, + "intervalName": { + "type": "keyword" + }, + "notExpandable": { + "type": "boolean" + }, + "sourceFilters": { + "type": "text" + }, + "timeFieldName": { + "type": "keyword" + }, + "title": { + "type": "text" + }, + "type": { + "type": "keyword" + }, + "typeMeta": { + "type": "keyword" + } + } + }, + "infrastructure-ui-source": { + "properties": { + "description": { + "type": "text" + }, + "fields": { + "properties": { + "container": { + "type": "keyword" + }, + "host": { + "type": "keyword" + }, + "pod": { + "type": "keyword" + }, + "tiebreaker": { + "type": "keyword" + }, + "timestamp": { + "type": "keyword" + } + } + }, + "logAlias": { + "type": "keyword" + }, + "logColumns": { + "properties": { + "fieldColumn": { + "properties": { + "field": { + "type": "keyword" + }, + "id": { + "type": "keyword" + } + } + }, + "messageColumn": { + "properties": { + "id": { + "type": "keyword" + } + } + }, + "timestampColumn": { + "properties": { + "id": { + "type": "keyword" + } + } + } + }, + "type": "nested" + }, + "metricAlias": { + "type": "keyword" + }, + "name": { + "type": "text" + } + } + }, + "inventory-view": { + "properties": { + "accountId": { + "type": "keyword" + }, + "autoBounds": { + "type": "boolean" + }, + "autoReload": { + "type": "boolean" + }, + "boundsOverride": { + "properties": { + "max": { + "type": "integer" + }, + "min": { + "type": "integer" + } + } + }, + "customMetrics": { + "properties": { + "aggregation": { + "type": "keyword" + }, + "field": { + "type": "keyword" + }, + "id": { + "type": "keyword" + }, + "label": { + "type": "keyword" + }, + "type": { + "type": "keyword" + } + }, + "type": "nested" + }, + "customOptions": { + "properties": { + "field": { + "type": "keyword" + }, + "text": { + "type": "keyword" + } + }, + "type": "nested" + }, + "filterQuery": { + "properties": { + "expression": { + "type": "keyword" + }, + "kind": { + "type": "keyword" + } + } + }, + "groupBy": { + "properties": { + "field": { + "type": "keyword" + }, + "label": { + "type": "keyword" + } + }, + "type": "nested" + }, + "legend": { + "properties": { + "palette": { + "type": "keyword" + }, + "reverseColors": { + "type": "boolean" + }, + "steps": { + "type": "long" + } + } + }, + "metric": { + "properties": { + "aggregation": { + "type": "keyword" + }, + "field": { + "type": "keyword" + }, + "id": { + "type": "keyword" + }, + "label": { + "type": "keyword" + }, + "type": { + "type": "keyword" + } + } + }, + "name": { + "type": "keyword" + }, + "nodeType": { + "type": "keyword" + }, + "region": { + "type": "keyword" + }, + "sort": { + "properties": { + "by": { + "type": "keyword" + }, + "direction": { + "type": "keyword" + } + } + }, + "time": { + "type": "long" + }, + "view": { + "type": "keyword" + } + } + }, + "kql-telemetry": { + "properties": { + "optInCount": { + "type": "long" + }, + "optOutCount": { + "type": "long" + } + } + }, + "lens": { + "properties": { + "expression": { + "index": false, + "type": "keyword" + }, + "state": { + "type": "flattened" + }, + "title": { + "type": "text" + }, + "visualizationType": { + "type": "keyword" + } + } + }, + "lens-ui-telemetry": { + "properties": { + "count": { + "type": "integer" + }, + "date": { + "type": "date" + }, + "name": { + "type": "keyword" + }, + "type": { + "type": "keyword" + } + } + }, + "map": { + "properties": { + "bounds": { + "type": "geo_shape" + }, + "description": { + "type": "text" + }, + "layerListJSON": { + "type": "text" + }, + "mapStateJSON": { + "type": "text" + }, + "title": { + "type": "text" + }, + "uiStateJSON": { + "type": "text" + }, + "version": { + "type": "integer" + } + } + }, + "maps-telemetry": { + "properties": { + "attributesPerMap": { + "properties": { + "dataSourcesCount": { + "properties": { + "avg": { + "type": "long" + }, + "max": { + "type": "long" + }, + "min": { + "type": "long" + } + } + }, + "emsVectorLayersCount": { + "dynamic": "true", + "type": "object" + }, + "layerTypesCount": { + "dynamic": "true", + "type": "object" + }, + "layersCount": { + "properties": { + "avg": { + "type": "long" + }, + "max": { + "type": "long" + }, + "min": { + "type": "long" + } + } + } + } + }, + "indexPatternsWithGeoFieldCount": { + "type": "long" + }, + "indexPatternsWithGeoPointFieldCount": { + "type": "long" + }, + "indexPatternsWithGeoShapeFieldCount": { + "type": "long" + }, + "mapsTotalCount": { + "type": "long" + }, + "settings": { + "properties": { + "showMapVisualizationTypes": { + "type": "boolean" + } + } + }, + "timeCaptured": { + "type": "date" + } + } + }, + "metrics-explorer-view": { + "properties": { + "chartOptions": { + "properties": { + "stack": { + "type": "boolean" + }, + "type": { + "type": "keyword" + }, + "yAxisMode": { + "type": "keyword" + } + } + }, + "currentTimerange": { + "properties": { + "from": { + "type": "keyword" + }, + "interval": { + "type": "keyword" + }, + "to": { + "type": "keyword" + } + } + }, + "name": { + "type": "keyword" + }, + "options": { + "properties": { + "aggregation": { + "type": "keyword" + }, + "filterQuery": { + "type": "keyword" + }, + "forceInterval": { + "type": "boolean" + }, + "groupBy": { + "type": "keyword" + }, + "limit": { + "type": "integer" + }, + "metrics": { + "properties": { + "aggregation": { + "type": "keyword" + }, + "color": { + "type": "keyword" + }, + "field": { + "type": "keyword" + }, + "label": { + "type": "keyword" + } + }, + "type": "nested" + } + } + } + } + }, + "migrationVersion": { + "dynamic": "true", + "properties": { + "index-pattern": { + "fields": { + "keyword": { + "ignore_above": 256, + "type": "keyword" + } + }, + "type": "text" + }, + "space": { + "fields": { + "keyword": { + "ignore_above": 256, + "type": "keyword" + } + }, + "type": "text" + } + } + }, + "ml-telemetry": { + "properties": { + "file_data_visualizer": { + "properties": { + "index_creation_count": { + "type": "long" + } + } + } + } + }, + "namespace": { + "type": "keyword" + }, + "namespaces": { + "type": "keyword" + }, + "query": { + "properties": { + "description": { + "type": "text" + }, + "filters": { + "enabled": false, + "type": "object" + }, + "query": { + "properties": { + "language": { + "type": "keyword" + }, + "query": { + "index": false, + "type": "keyword" + } + } + }, + "timefilter": { + "enabled": false, + "type": "object" + }, + "title": { + "type": "text" + } + } + }, + "references": { + "properties": { + "id": { + "type": "keyword" + }, + "name": { + "type": "keyword" + }, + "type": { + "type": "keyword" + } + }, + "type": "nested" + }, + "sample-data-telemetry": { + "properties": { + "installCount": { + "type": "long" + }, + "unInstallCount": { + "type": "long" + } + } + }, + "search": { + "properties": { + "columns": { + "type": "keyword" + }, + "description": { + "type": "text" + }, + "hits": { + "type": "integer" + }, + "kibanaSavedObjectMeta": { + "properties": { + "searchSourceJSON": { + "type": "text" + } + } + }, + "sort": { + "type": "keyword" + }, + "title": { + "type": "text" + }, + "version": { + "type": "integer" + } + } + }, + "space": { + "properties": { + "_reserved": { + "type": "boolean" + }, + "color": { + "type": "keyword" + }, + "description": { + "type": "text" + }, + "disabledFeatures": { + "type": "keyword" + }, + "imageUrl": { + "index": false, + "type": "text" + }, + "initials": { + "type": "keyword" + }, + "name": { + "fields": { + "keyword": { + "ignore_above": 2048, + "type": "keyword" + } + }, + "type": "text" + } + } + }, + "telemetry": { + "properties": { + "allowChangingOptInStatus": { + "type": "boolean" + }, + "enabled": { + "type": "boolean" + }, + "lastReported": { + "type": "date" + }, + "lastVersionChecked": { + "type": "keyword" + }, + "reportFailureCount": { + "type": "integer" + }, + "reportFailureVersion": { + "type": "keyword" + }, + "sendUsageFrom": { + "type": "keyword" + }, + "userHasSeenNotice": { + "type": "boolean" + } + } + }, + "timelion-sheet": { + "properties": { + "description": { + "type": "text" + }, + "hits": { + "type": "integer" + }, + "kibanaSavedObjectMeta": { + "properties": { + "searchSourceJSON": { + "type": "text" + } + } + }, + "timelion_chart_height": { + "type": "integer" + }, + "timelion_columns": { + "type": "integer" + }, + "timelion_interval": { + "type": "keyword" + }, + "timelion_other_interval": { + "type": "keyword" + }, + "timelion_rows": { + "type": "integer" + }, + "timelion_sheet": { + "type": "text" + }, + "title": { + "type": "text" + }, + "version": { + "type": "integer" + } + } + }, + "tsvb-validation-telemetry": { + "properties": { + "failedRequests": { + "type": "long" + } + } + }, + "type": { + "type": "keyword" + }, + "ui-metric": { + "properties": { + "count": { + "type": "integer" + } + } + }, + "updated_at": { + "type": "date" + }, + "upgrade-assistant-reindex-operation": { + "properties": { + "errorMessage": { + "type": "keyword" + }, + "indexName": { + "type": "keyword" + }, + "lastCompletedStep": { + "type": "integer" + }, + "locked": { + "type": "date" + }, + "newIndexName": { + "type": "keyword" + }, + "reindexOptions": { + "properties": { + "openAndClose": { + "type": "boolean" + }, + "queueSettings": { + "properties": { + "queuedAt": { + "type": "long" + }, + "startedAt": { + "type": "long" + } + } + } + } + }, + "reindexTaskId": { + "type": "keyword" + }, + "reindexTaskPercComplete": { + "type": "float" + }, + "runningReindexCount": { + "type": "integer" + }, + "status": { + "type": "integer" + } + } + }, + "upgrade-assistant-telemetry": { + "properties": { + "features": { + "properties": { + "deprecation_logging": { + "properties": { + "enabled": { + "null_value": true, + "type": "boolean" + } + } + } + } + }, + "ui_open": { + "properties": { + "cluster": { + "null_value": 0, + "type": "long" + }, + "indices": { + "null_value": 0, + "type": "long" + }, + "overview": { + "null_value": 0, + "type": "long" + } + } + }, + "ui_reindex": { + "properties": { + "close": { + "null_value": 0, + "type": "long" + }, + "open": { + "null_value": 0, + "type": "long" + }, + "start": { + "null_value": 0, + "type": "long" + }, + "stop": { + "null_value": 0, + "type": "long" + } + } + } + } + }, + "uptime-dynamic-settings": { + "properties": { + "certAgeThreshold": { + "type": "long" + }, + "certExpirationThreshold": { + "type": "long" + }, + "heartbeatIndices": { + "type": "keyword" + } + } + }, + "url": { + "properties": { + "accessCount": { + "type": "long" + }, + "accessDate": { + "type": "date" + }, + "createDate": { + "type": "date" + }, + "url": { + "fields": { + "keyword": { + "type": "keyword" + } + }, + "type": "text" + } + } + }, + "visualization": { + "properties": { + "description": { + "type": "text" + }, + "kibanaSavedObjectMeta": { + "properties": { + "searchSourceJSON": { + "type": "text" + } + } + }, + "savedSearchRefName": { + "type": "keyword" + }, + "title": { + "type": "text" + }, + "uiStateJSON": { + "type": "text" + }, + "version": { + "type": "integer" + }, + "visState": { + "type": "text" + } + } + } + } + }, + "settings": { + "index": { + "auto_expand_replicas": "0-1", + "number_of_replicas": "0", + "number_of_shards": "1" + } + } + } +} + +{ + "type": "index", + "value": { + "aliases": { + }, + "index": "test", + "mappings": { + "properties": { + "foo": { + "fields": { + "keyword": { + "ignore_above": 256, + "type": "keyword" + } + }, + "type": "text" + } + } + }, + "settings": { + "index": { + "number_of_replicas": "1", + "number_of_shards": "1" + } + } + } +} \ No newline at end of file diff --git a/x-pack/test/page_load_metrics/runner.ts b/x-pack/test/page_load_metrics/runner.ts new file mode 100644 index 0000000000000..05f293730f843 --- /dev/null +++ b/x-pack/test/page_load_metrics/runner.ts @@ -0,0 +1,33 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { CiStatsReporter } from '@kbn/dev-utils'; +import { capturePageLoadMetrics } from '@kbn/test'; +// @ts-ignore not TS yet +import getUrl from '../../../src/test_utils/get_url'; + +import { FtrProviderContext } from './../functional/ftr_provider_context'; + +export async function PuppeteerTestRunner({ getService }: FtrProviderContext) { + const log = getService('log'); + const config = getService('config'); + const esArchiver = getService('esArchiver'); + + await esArchiver.load('default'); + const metrics = await capturePageLoadMetrics(log, { + headless: true, + appConfig: { + url: getUrl.baseUrl(config.get('servers.kibana')), + username: config.get('servers.kibana.username'), + password: config.get('servers.kibana.password'), + }, + screenshotsDir: config.get('screenshots.directory'), + }); + const reporter = CiStatsReporter.fromEnv(log); + + log.debug('Report page load asset size'); + await reporter.metrics(metrics); +} diff --git a/yarn.lock b/yarn.lock index 324941e623ced..50bcce2a807cb 100644 --- a/yarn.lock +++ b/yarn.lock @@ -4574,6 +4574,13 @@ dependencies: "@types/node" "*" +"@types/puppeteer@^3.0.0": + version "3.0.0" + resolved "https://registry.yarnpkg.com/@types/puppeteer/-/puppeteer-3.0.0.tgz#24cdcc131e319477608d893f0017e08befd70423" + integrity sha512-59+fkfHHXHzX5rgoXIMnZyzum7ZLx/Wc3fhsOduFThpTpKbzzdBHMZsrkKGLunimB4Ds/tI5lXTRLALK8Mmnhg== + dependencies: + "@types/node" "*" + "@types/q@^1.5.1": version "1.5.2" resolved "https://registry.yarnpkg.com/@types/q/-/q-1.5.2.tgz#690a1475b84f2a884fd07cd797c00f5f31356ea8" @@ -7637,6 +7644,15 @@ bl@^3.0.0: dependencies: readable-stream "^3.0.1" +bl@^4.0.1: + version "4.0.2" + resolved "https://registry.yarnpkg.com/bl/-/bl-4.0.2.tgz#52b71e9088515d0606d9dd9cc7aa48dc1f98e73a" + integrity sha512-j4OH8f6Qg2bGuWfRiltT2HYGx0e1QcBTrK9KAHNMwMZdQnDZFk0ZSYIpADjYCB3U12nicC5tVJwSIhwOWjb4RQ== + dependencies: + buffer "^5.5.0" + inherits "^2.0.4" + readable-stream "^3.4.0" + blob@0.0.5: version "0.0.5" resolved "https://registry.yarnpkg.com/blob/-/blob-0.0.5.tgz#d680eeef25f8cd91ad533f5b01eed48e64caf683" @@ -8158,6 +8174,14 @@ buffer@^5.1.0, buffer@^5.2.0: base64-js "^1.0.2" ieee754 "^1.1.4" +buffer@^5.2.1, buffer@^5.5.0: + version "5.6.0" + resolved "https://registry.yarnpkg.com/buffer/-/buffer-5.6.0.tgz#a31749dc7d81d84db08abf937b6b8c4033f62786" + integrity sha512-/gDYp/UtU0eA1ys8bOs9J6a+E/KWIY+DZ+Q2WESNUA0jFRsJOc0SNUO6xJ5SGA1xueg3NL65W6s+NY5l9cunuw== + dependencies: + base64-js "^1.0.2" + ieee754 "^1.1.4" + builtin-modules@^1.0.0: version "1.1.1" resolved "https://registry.yarnpkg.com/builtin-modules/-/builtin-modules-1.1.1.tgz#270f076c5a72c02f5b65a47df94c5fe3a278892f" @@ -16649,7 +16673,7 @@ inflight@^1.0.4: once "^1.3.0" wrappy "1" -inherits@2, inherits@2.0.4, inherits@~2.0.3, inherits@~2.0.4: +inherits@2, inherits@2.0.4, inherits@^2.0.4, inherits@~2.0.3, inherits@~2.0.4: version "2.0.4" resolved "https://registry.yarnpkg.com/inherits/-/inherits-2.0.4.tgz#0fa2c64f932917c3433a0ded55363aae37416b7c" integrity sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ== @@ -20773,6 +20797,11 @@ mixin-object@^2.0.1: for-in "^0.1.3" is-extendable "^0.1.1" +mkdirp-classic@^0.5.2: + version "0.5.3" + resolved "https://registry.yarnpkg.com/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz#fa10c9115cc6d8865be221ba47ee9bed78601113" + integrity sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A== + mkdirp@0.5.1: version "0.5.1" resolved "https://registry.yarnpkg.com/mkdirp/-/mkdirp-0.5.1.tgz#30057438eac6cf7f8c4767f38648d6697d75c903" @@ -23895,6 +23924,22 @@ puppeteer@^2.0.0: rimraf "^2.6.1" ws "^6.1.0" +puppeteer@^3.3.0: + version "3.3.0" + resolved "https://registry.yarnpkg.com/puppeteer/-/puppeteer-3.3.0.tgz#95839af9fdc0aa4de7e5ee073a4c0adeb9e2d3d7" + integrity sha512-23zNqRltZ1PPoK28uRefWJ/zKb5Jhnzbbwbpcna2o5+QMn17F0khq5s1bdH3vPlyj+J36pubccR8wiNA/VE0Vw== + dependencies: + debug "^4.1.0" + extract-zip "^2.0.0" + https-proxy-agent "^4.0.0" + mime "^2.0.3" + progress "^2.0.1" + proxy-from-env "^1.0.0" + rimraf "^3.0.2" + tar-fs "^2.0.0" + unbzip2-stream "^1.3.3" + ws "^7.2.3" + q@^1.1.2: version "1.5.1" resolved "https://registry.yarnpkg.com/q/-/q-1.5.1.tgz#7e32f75b41381291d04611f1bf14109ac00651d7" @@ -26251,6 +26296,13 @@ rimraf@^2.5.4, rimraf@^2.7.1: dependencies: glob "^7.1.3" +rimraf@^3.0.2: + version "3.0.2" + resolved "https://registry.yarnpkg.com/rimraf/-/rimraf-3.0.2.tgz#f1a5402ba6220ad52cc1282bac1ae3aa49fd061a" + integrity sha512-JZkJMZkAGFFPP2YqXZXPbMlMBgsxzE8ILs4lMIX/2o0L9UBw9O/Y3o6wFw/i9YLapcUJWwqbi3kdxIPdC62TIA== + dependencies: + glob "^7.1.3" + rimraf@~2.0.2: version "2.0.3" resolved "https://registry.yarnpkg.com/rimraf/-/rimraf-2.0.3.tgz#f50a2965e7144e9afd998982f15df706730f56a9" @@ -28598,6 +28650,16 @@ tar-fs@^1.16.3: pump "^1.0.0" tar-stream "^1.1.2" +tar-fs@^2.0.0: + version "2.1.0" + resolved "https://registry.yarnpkg.com/tar-fs/-/tar-fs-2.1.0.tgz#d1cdd121ab465ee0eb9ccde2d35049d3f3daf0d5" + integrity sha512-9uW5iDvrIMCVpvasdFHW0wJPez0K4JnMZtsuIeDI7HyMGJNxmDZDOCQROr7lXyS+iL/QMpj07qcjGYTSdRFXUg== + dependencies: + chownr "^1.1.1" + mkdirp-classic "^0.5.2" + pump "^3.0.0" + tar-stream "^2.0.0" + tar-stream@^1.1.2, tar-stream@^1.5.2: version "1.5.5" resolved "https://registry.yarnpkg.com/tar-stream/-/tar-stream-1.5.5.tgz#5cad84779f45c83b1f2508d96b09d88c7218af55" @@ -28608,6 +28670,17 @@ tar-stream@^1.1.2, tar-stream@^1.5.2: readable-stream "^2.0.0" xtend "^4.0.0" +tar-stream@^2.0.0: + version "2.1.2" + resolved "https://registry.yarnpkg.com/tar-stream/-/tar-stream-2.1.2.tgz#6d5ef1a7e5783a95ff70b69b97455a5968dc1325" + integrity sha512-UaF6FoJ32WqALZGOIAApXx+OdxhekNMChu6axLJR85zMMjXKWFGjbIRe+J6P4UnRGg9rAwWvbTT0oI7hD/Un7Q== + dependencies: + bl "^4.0.1" + end-of-stream "^1.4.1" + fs-constants "^1.0.0" + inherits "^2.0.3" + readable-stream "^3.1.1" + tar-stream@^2.1.0: version "2.1.0" resolved "https://registry.yarnpkg.com/tar-stream/-/tar-stream-2.1.0.tgz#d1aaa3661f05b38b5acc9b7020efdca5179a2cc3" @@ -28897,7 +28970,7 @@ through2@~2.0.3: readable-stream "~2.3.6" xtend "~4.0.1" -through@2, "through@>=2.2.7 <3", through@^2.3.4, through@^2.3.6, through@~2.3.4, through@~2.3.6, through@~2.3.8: +through@2, "through@>=2.2.7 <3", through@^2.3.4, through@^2.3.6, through@^2.3.8, through@~2.3.4, through@~2.3.6, through@~2.3.8: version "2.3.8" resolved "https://registry.yarnpkg.com/through/-/through-2.3.8.tgz#0dd4c9ffaabc357960b1b724115d7e0e86a2e1f5" integrity sha1-DdTJ/6q8NXlgsbckEV1+Doai4fU= @@ -30091,6 +30164,14 @@ unbzip2-stream@^1.0.9: buffer "^3.0.1" through "^2.3.6" +unbzip2-stream@^1.3.3: + version "1.4.2" + resolved "https://registry.yarnpkg.com/unbzip2-stream/-/unbzip2-stream-1.4.2.tgz#84eb9e783b186d8fb397515fbb656f312f1a7dbf" + integrity sha512-pZMVAofMrrHX6Ik39hCk470kulCbmZ2SWfQLPmTWqfJV/oUm0gn1CblvHdUu4+54Je6Jq34x8kY6XjTy6dMkOg== + dependencies: + buffer "^5.2.1" + through "^2.3.8" + unc-path-regex@^0.1.2: version "0.1.2" resolved "https://registry.yarnpkg.com/unc-path-regex/-/unc-path-regex-0.1.2.tgz#e73dd3d7b0d7c5ed86fbac6b0ae7d8c6a69d50fa" @@ -32065,6 +32146,11 @@ ws@^7.0.0: dependencies: async-limiter "^1.0.0" +ws@^7.2.3: + version "7.3.0" + resolved "https://registry.yarnpkg.com/ws/-/ws-7.3.0.tgz#4b2f7f219b3d3737bc1a2fbf145d825b94d38ffd" + integrity sha512-iFtXzngZVXPGgpTlP1rBqsUK82p9tKqsWRPg5L56egiljujJT3vGAYnHANvFxBieXrTFavhzhxW52jnaWV+w2w== + ws@~3.3.1: version "3.3.3" resolved "https://registry.yarnpkg.com/ws/-/ws-3.3.3.tgz#f1cf84fe2d5e901ebce94efaece785f187a228f2"