Don't use filter to get default config & output #69088

jfsiii · 2020-06-12T22:22:22Z

Summary

The KQL parsing takes up a lot of CPU time. Avoiding it by finding with search instead of filter. Also adds another refresh: false as suggested by @kobelb

Current

PR

Current

PR

I'll admit the gains aren't obvious to me looking at this

but the response times are better elastic_kibana_pr_69088.zip

@bkobel

the KQL parsing takes up a lot of CPU time. Avoid it with a search string Also add a `refresh: false` as suggested by @bkobel

jfsiii · 2020-06-14T23:52:51Z

@elasticmachine merge upstream

elasticmachine · 2020-06-15T13:28:49Z

Pinging @elastic/ingest-management (Team:Ingest Management)

jfsiii · 2020-06-15T13:29:15Z

CI is failing one test. I'm looking into it

  1) apis
       Fleet Endpoints
         fleet_agent_flow
           should work:
     Error: expected 200 "OK", got 404 "Not Found"
      at Test._assertStatus (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:268:12)
      at Test._assertFunction (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:283:11)
      at Test.assert (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:173:18)
      at assert (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:131:12)
      at /dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:128:5
      at Test.Request.callback (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/index.js:718:3)
      at parser (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/index.js:906:18)
      at IncomingMessage.res.on (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/parsers/json.js:19:7)
      at endReadableNT (_stream_readable.js:1145:12)
      at process._tickCallback (internal/process/next_tick.js:63:19)

               └- ✖ fail: "apis Fleet Endpoints fleet_agent_flow should work"
               │
             └-> "after all" hook
             └-> "after all" hook
           └-> "after all" hook
         └-> "after all" hook
     │

nchaulet · 2020-06-15T13:44:54Z

x-pack/plugins/ingest_manager/server/services/agents/enroll.ts

+    {
+      access_api_key_id: accessAPIKey.id,
+    },
+    { refresh: false }


You probably need to update the checkin code to make the tests green, currently we are making a search using the access_api_key_id during the checkin so if the agent is not refreshed the search will not work.

We can change that in the checkin to get the agent by id (the id is in the request params) and check if the access_api_key_id match.

Thanks, @nchaulet that makes sense. I'll look into it. I'm also curious/concerned why these pass locally for me but fail in CI

My guess is your macbook is faster than the CI server and the refresh is done before the checkin call.

Does this this fix the failing test? It does keep the PR more focused

kibanamachine · 2020-06-15T17:06:04Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: bb489d0

Build metrics

✅ unchanged

History

💔 Build #53953 failed d9c0339
💔 Build #53877 failed f558eb8
💔 Build #53762 failed 50bf157

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

jfsiii · 2020-06-15T19:02:47Z

@EricDavisX #69088 (comment) is an example of an error case we can define in the OpenAPI spec file and/or the consistent error fixes

jfsiii · 2020-06-15T20:48:03Z

@ph @ruflin should we backport this to 7.8?

ruflin · 2020-06-16T13:24:16Z

Even thought the performance improvement is massive I would keep it out of 7.8 as it is not directly a bug.

@bkobel

* Don't use filter to get default config & output the KQL parsing takes up a lot of CPU time. Avoid it with a search string Also add a `refresh: false` as suggested by @bkobel * Does restoring await fix CI? * Remove 'refresh: false' from agent enroll Does this this fix the failing test? It does keep the PR more focused Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

@bkobel

* Don't use filter to get default config & output the KQL parsing takes up a lot of CPU time. Avoid it with a search string Also add a `refresh: false` as suggested by @bkobel * Does restoring await fix CI? * Remove 'refresh: false' from agent enroll Does this this fix the failing test? It does keep the PR more focused Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

chore(NA): merge changes on renovate config file chore(NA): move everything under alias lodash4 chore(NA): missing lodash to lodash4 alias chore(NA): migrate packages to lodash4 chore(NA): update renovate cfg file chore(NA): skip failing test chore(NA): first remove of alias lodash4 chore(NA): remove lodash4 alias chore(NA): remove skip from test [ES UI] Error handling (#68809) [Console] Scrub the lead and trailing brackets from ipv6 host names (#68991) * scrub the lead and trailing brackets from ipv6 host names * Update comment * refactor: scrub -> sanitize Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> fix spec color highlighting not working on vega vis (#68995) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [SECURITY] rename server plugin to siem (#69044) * rename server plugin to siem to avoid privilege issue * review from alerting * missing change with rename * fix tests * missing api integration test * fix api integration spaces Type safe browser.executeAsync (#69018) * make browserAsync type safe * adopt tests * prefer unknown over any * simplify signature fixes 'Configures a new connector' flaky test (#69011) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Embed dashboard by value example & some embeddable clean up (#67783) Added example for using dashboard container by value 1.1 Refactored embeddable explorer e2e test to use new example, removed not needed kbn_tp_embeddable_explorer plugin. For embeddable explorer examples went away from using getFactoryById() to improve type checks There is new component a replacement for EmbeddableFactoryRenderer with slightly more flexible api: EmbeddableRenderer. 3.1 We can improve it going forward to support more use case [Endpoint] Adding alerts route (#68183) * Adding alerts route * Adding related alerts generator changes, tests, and script updates * Fixing missed parameter * Aligning the AlertEvent and ResolverEvent definition * Fixing type errors * Fixing import error Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [Monitoring] Ensure we use existing Elasticsearch config (#68389) * Ensure we use existing Elasticsearch config * Use separate type for this to ensure custom properties work * PR suggestions * PR feedback * PR feedback * Fix type issues * PR feedback [APM] Changes to duration formatting (#69039) * [APM] Changes to duration formatting * Change the threshold for showing microseconds to 1 millisecond instead of 10. This means you now get "900 µs/1.3 ms/20.0 ms" instead of "900 µs/1300 µs/20 ms." * Change milliseconds formatted with `asDuration` to be `asDecimal` instead of `asInteger`. That means you get "0.0 ms/2.5 ms/3.0 ms" instead of "0 ms/2 ms/3 ms." * Tables were all using their own module-scoped functions that called `asDuration` to format things as decimal milliseconds. Extract this to an `asMillisecondDuration` function exported from the duration helpers and use it in all the tables. * Change `getResponseTimeseries` in chart selectors to use `asDuration` to make all chart timeseries units consistent. * Don't export `convertTo` from the duration helpers as it's now not used anywhere. Always use a more specific exported function for more consistency. * Change ">=" to "≥" in the ML flyout text. * Update e2e snapshot Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [SECURITY SOLUTION] Task/endpoint details to fleet (#68710) [SECURITYSOLUTION][INGEST] Fixes endpoint link, adds link from endpoint details to fleet Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Set wrap to the errorLink so it doesn't go outside of box (#67797) * set wrap to the errorLink so it doesnt go outside of box * updating * set wrap to the errorLink so it doesnt go outside of box * updating * Show full message on hover Co-authored-by: Nathan L Smith <smith@nlsmith.com> [Metrics UI] Add preview feature for metric threshold alerts (#67684) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [skip-ci] Rename @elastic/pulse to @elastic/kibana-telemetry (#69181) try to fix integration test by making template more specific (#69138) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [Alerting] Update Index Threshold to use columns in EuiExpression (#69051) Add enroll agent action to config action menu (#68840) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Don't use filter to get default config & output (#69088) * Don't use filter to get default config & output the KQL parsing takes up a lot of CPU time. Avoid it with a search string Also add a `refresh: false` as suggested by @bkobel * Does restoring await fix CI? * Remove 'refresh: false' from agent enroll Does this this fix the failing test? It does keep the PR more focused Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [SIEM][Detections] In progress alert state (#68569) [Endpoint] Remove dependency on ingest for the index patterns (#69058) * Remove dependency on ingest for the index patterns * Fixing the tests * Fixing test * Use variable instead of class Updates the Release Notes content in CONTRIBUTING (#69032) * Updates the Release Notes content in CONTRIBUTING * Release Notes guidelines * Fixes bulleted list indentation Fix plugin lifecycle log to only include server plugins (#68686) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [Ingest Manager] Add ability to copy an agent config (#68737) * Add copy agent config endpoint, schema types, and request hook * Add copy config to context menu and modal * Change data sources to be copied via bulk instead of singles in parallel * Add copy endpoint to openapi spec file * Add api integration tests Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [ML] Allow editing of model_snapshot_retention_days (#69003) * [ML] Allow editing of model_snapshot_retention_days * removing commented code * flooring number input * adding daily_model_snapshot_retention_after_days * updating default values Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Remove obsolete custom types from super date picker handlers. (#69038) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Using correct index patterns (#69208) [Endpoint] [ES Archiver] Allowing create option to be passed through the cli for es archiver (#69191) * Allowing create option to be passed through the cli * Using kebab casing [Security] siem to securitySolution SO migration (#68776) chore(NA): include hidden files when creating package with plugin helpers (#68247) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [Reporting/Server] register plugin routes synchronously (#68976) * register routes synchronously * back out some refactoring * comment fix * fix tests * register route handler context provider * Add function level comments in core methods * fix tests * revert editor help * route context is the ReportingStart contract * Fix reporting job route tests * Fix generation tests Co-authored-by: Joel Griffith <joel.griffith@elastic.co> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [release-notes] add script to generate release notes from PRs (#68816) Co-authored-by: spalger <spalger@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Sort filter fields by items that start with typed characters. (#68585) [kbn/optimizer] share all plugin bundles (#68986) Co-authored-by: spalger <spalger@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [CI] GitHub check details link to test reports and errorlevel (#69157) [Alerting] Adds API for executing actions immediately (#68897) This PR adds an API to the AlertsClient which allows users to execute actions immediately, rather than via TaskManager. This also moves the TaskManager based execution to a new api on ActionsClient along side the immediate execution. stabilize flaky embeddables/adding_children take 2 (#68873) * wait for loading indicator to be deleted * improve with retry * Update test/examples/embeddables/adding_children.ts Co-authored-by: Spencer <email@spalger.com> Co-authored-by: Spencer <email@spalger.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Update vega version (#68639) * update vega version * a11y skip test * a11y skip test attempt 2 * adding back all a11y tests * fix failed karma test * remove extra 0BSD license * coalesce yarn.lock versions a little * update kbn/pm dist * fix CI * fix Vega View for ML Co-authored-by: Michail Yasonik <michail.yasonik@elastic.co> Co-authored-by: spalger <spalger@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [APM] Fix confusing request/minute viz (#69143) [Ingest pipelines] Fix create pipeline button copy (#69118) [ILM] Update doc link (#68923) [ML] Model snapshot management (#68182) * [ML] Model snapshot management * more updates * adding calendar range * updating layout * multiple calendars * moving calendar creator * fixing chart issues * fixing chart issues * improving calendar rendering * adding capabilities checks * code clean up * fixing end time argument type * fix translations * code clean up * comments based on review * changes based on review * fixing include * adding useMemo to theme function [Maps] Security layer wizards (#68290) * [Maps] security layer wizard * index pattern select * use IndexPattern * destination layer * source layer * cache security index patterns * line layer * use top hits * tslint * unit test * fix loading speed and i18n * handle case where ui setting siem:defaultIndex does not exist * tslint * review feedback * extract lazy load bundle from map embeddables * expose createSecurityLayers via plugin start * tslint * export MapsPluginStart * fix path move * review feedback * fix import Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [Ingest Node Pipelines] Pipeline Processors Editor (#66021) * initial plugin setup * add smoke test * fix license check * refactor plugin setup * Server-side create/update ingest pipelines (#62744) * List pipelines (#62785) * First iteration of ingest table * Add action placeholders * Refactor list and address PR feedback Refactored the list into smaller pieces and assemble in main.tsx Also addressed feedback on copy, removed unused notifications dep * WiP on flyout Showing name in title * Add reload button * Finish first version of flyout * Slight update to copy * `delete` -> `edit` * Address PR feedback Copy and a11y updates * Add on failure JSON to flyout if it is available * Add details json block file and remove ununsed import Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> * [Ingest pipelines] Create pipeline UI (#63017) * First vertical slice of pipeline editor component * Made a space for common parameters * [Ingest pipelines] Edit pipeline page (#63522) * First iteration of CRUD functionality working * WiP on moving the pipeline editor to pipeline processor editor * Finish refactor to work with passing state out * Refactor and fix tests * [Ingest pipelines] Polish details panel and empty list (#63926) * Address some early feedback and use FormDataProvider FormDataProvider gives a more declarative way to listen to form state updates. Also refactored the state reader mechanism. * [Ingest pipelines] Delete pipeline (#63635) * [Ingest Node Pipelines] Clone Pipeline (#64049) * First iteration of clone functionality Wired up for both the list table and the details flyout in the list section. * satisfy eslint * Turn on sorting for the list table * Clean up const declarations * Address PR feedback Sentence-casify and update some other copy. * Mark edit and delete as primary actions in list table * Handle URI encoded chars in pipeline name when cloning * Update to using the more flexible controlled component pattern To make this component and mappings editor more consistent, both will expose a more traditional controlled component interface to consumers. In the current implementation this requires that consumers use `useCallback` for `onUpdate` handlers to avoid an infinite rendering cycle and to avoid staleness bugs in their `onUpdate` handlers they need to think about what might make the handler stale. This approach comes with the benefits and flexibility of controlled components at the cost of slightly more complex consumption of the components. In future, we can explore adding the uncontrolled component interface too so that consumers have the option to more simply render and pull data out only when needed * Handle sub-form validity The pipelines processor editor not emits overall validity to consumers * Fix Jest test * Refactor some names prepareDataOut -> serialize prepareDataIn -> deserialize EditorProcessor -> ProcessorInternal * Mark as private * Major WiP Started working on the drag-and-drop-tree and updated some typings stuff * [Ingest node pipelines] Privileges (#63850) * Create privileges check for ingest pipelines app Also moved the public side logic for checking and rendering privilege related messages to es_ui_shared/public following the new __packages_do_not_import__ convention. * Add , * Fix import paths * Address PR feedback Fix i18n strings (remove reference to snapshot and restore) and fix copy referencing snapshot and restore - all copy-pasta errors. Also remove unused field from missing privileges object. * Fix issue from resolving merge conflicts * Add missing app privilege * Use non-deprecated privilige name * First iteration of drag and drop tree on feature parity * First steps toward add on failure handler Updated reducer logic to create the next copy value using immer. * First iteration of nested tree with add on failure working * Refactor and some UI layout updates - Remove the "id" field on processors for now - Implement the nested remove and update functionality again * Remove immer (not call stack safe) Refac to remove immer and reimplemented the immutable set and get functions. * [Ingest Node Pipelines] More lenient treatment of on-failure value (#64411) * Move file to components folder * [Ingest pipelines] Simulate pipeline (#64223) * Updated tree rendering - turn off dropzones for children - fixed up some padding and margins - fixed integration with pipeline_form.tsx The current implementation still has a lot of jank stemming from the UX with DnD. Unfortunately the nesting has opened a number of issues here. * [Ingest Node Pipelines] Show flyout after editing or creating a pipeline (#64409) * Show flyout after editing or creating a pipeline * JSX comment * Show not found flyout Copied from CCR * update not found flyout and fix behavior when viewing details from table * Reset pipeline name in URI when closing flyout * Remove encodeURI Already using encodingURIComponent for unsafe string. Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com> * Clarification of terms - addProcessor -> addTopLevelProcessor (same in editor modes) - Expanded comment on ProcessorSelector * Implement move between lists functionality Added tests to the reducer for moving in and out of a recusrsive structure. * fix TS * Prevent nesting a parent inside of its own child * Add comment * [Ingest pipelines] Cleanup (#64794) * address review feedback * remove unused import * Big refactor to tree rendering structure DnD tree now converts the nested tree to a flat one and only consists of _1_ droppable area with a flat array of draggables that can be combined. Using the existing logic in the reducer combined with translating the flat structure changes to a format the nested reducer can understand looks like a really promising avenue. There still seems to be a bug with a longer list where items do not interact properly. * Remove unused component * A number of NB changes - Fixed a subtle serialisation bug with on_failure in options - Fixed a performance bug by using useRef in pipeline form - Memoized the drag and drop tree - Removed use of "isValid" from form lib... I think this should be removed entirely. * fix bad conflict resolution * Implemented a slightly better destination resolution algo. Also added tests. * Fix subtle staleness bug, whitelist keys for setValue * NB styling fix!! Due to a parent's setting of overflow: hidden the drag and drop tree had a dead zone that would equal the page overflow limit, unsetting on the component itself (overflow: auto) relaxes this limitation. * Fix stale delete bug too * Update naming of editor modes and update comments * Use field types combo box * Add delete confirmation modal * Refactor delete modal component file name * Better visual integration with existing form * Update layout and styling of form - added some padding around the new processors editor field - Updated use of flex box in the pipeline form fields * Move pipeline processor copy into pipeline processor component The test button is also now inside of the pipeline processor editor. Eventually all of this functionality should be moved into the pipeline processor editor. * First step of refactor to moving between trees * First iteration of x-tree drag and drop * Remove unused import * Fix jest test types * Fix up minor i18n issues and fix up layout of on failure * Remove unnecessary prop * Update spacing above add processor button to make it more center * Fix destination resolution algo * Update dragging resolver unit tests and add a lot more comments * Use one sorting algo (removed use of euiReorder for now) * Add placeholder tests and update comments * Quite a big refactor - Remove DraggableLocation entirely, only use ProcessorSelector this simplifies mapping back to reducer instructions quite a lot - Add tests for special case dragging behaviour * Fix off by one bug in tests and implementation 🤦🏼‍♂ - Also move processor reducer to it's own folder * Update behaviour for dragging up across trees and add tests * Fix combine instruction * Fix test and i18n issues * Remove background color * Fix selector after selector refactor * A major performance - Do not re-render the entire tree when editing settings * Fix component smoke test * Fix reading value from processor state using processor selector * [Ingest pipelines] Custom processor form (#66022) * Re add background color and refactor name of processor item * Fix file naming and refactor 🚜 dnd tree rendering - deserialze and serialize were backwards, fixed - the dnd tree was rendering a flat tree which needed to be mapped back to the nested tree for updates. This is still the case but we do use recursive rendering. This enables tree nodes to better hold local state like whether they are collapsed or not. * Fix getting of initial on failure processors value * Update padding styles for containers * A lot of styling updates to get closer to look of mockup * A WiP version of th click-tree an alternative to dnd As a response to really long pipelines we may pivot away from dnd. * Remove dnd tree * clean up reamining dnd tree references * Clean up and refactor of tree component To simplify the logic of the tree component processor id's were added. Also had to update the jest spec to support this * Added the ability to duplicate a processor * Fix types in test * Added duplicate functionality to ui * Memoize tree components * address es lint issues * remove unused import * Fix editing of custom json * Address form performance issues * Add all known missing processors * Add ability to cancel move * Fix staleness in test and view request flyouts * fix type issue * Remove unused translations and skip funcitonal test for now * add todo comment * Fix type issues * remove poc styles * disable other move buttons if we have a selected processor * Refactor drop zone pin to button and add some styling * Refactor processor editor item * Update styling and use icon for cancel move action too * fix nasty integration bug * some minor optimizations * prevent parent from being placed in own on failure handler * Re-add cancel button * Re-introduce failure processors toggle * Fix typo * Add Handler types for processor editor item * Fix staleness bug for type, refactor classname and fix duplicate bug not immutably copying values * Experimenting with padels (revert this to undo if no further changes have been made) * Add description and unique ids * Share links via component-wide context rather than props * Virtualized list and back to outline dropzones * Refactor id getter to a service and make it an incrementing number * Temporary fix for double rendering issue. This means the pipeline processors component is not controllable but fixes the double rendering even when processors have not changed. This is a problem coming from the ingest pipelines plugin form system outside of this component * add todo comment * remove euicode element * properly handle duplicate flow * attempt to fix i18n * split private_tree into it's own component and add comments * refactor 🚜. rename Tree to ProcessorsTree and move things around * do not delete the top level arrays for processors and onfailures * fix typescript error * Move duplicate, addOnFailure and delete actions into ctx menu * remove unused import * add support for pressing esc key to cancel move * Add outside click listener * always prompt before deleting a processor * refactor remove distinction between adding top level and on fail * add processor button to tree * Hide the add on failure context menu item for processors with failure handlers * Reinstated x-tree moving and highlight and disable for buttons on move and on edit * removing ids step 1: remove idGenerator * remove ids step 2: added inline text input Also refactored a lot of the tree actions away. Now using context and the processors editor tree item component for dispatching actions. The tree item has access to the dispatch and to the selector which makes it a well positioned component. Also saves on some props drilling. * Slight improvement to styling of text input (border) * Re-implement missing failure toggle test * address type todo * Address many type issues and fix yarn.lock * re-enable create pipeline functional test * prevent multiple flyouts from opening * change flyout title when editing an on-failure processor * absolutely position the failure handlers label when we render a dropzone on the label, then we move the label up without affecting overall component height * fix description behaviour not removing tag if empty * some minor clean up * add onflyoutopen cb to tests * refactor processors editor item to multiple files also refactored i18n into it's own file. would be good to come up with a pttaern for doing this more broadly. * fix add on-failure handler in context menu after refactor * tag -> new description field Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Alison Goryachev <alison.goryachev@elastic.co> [ML] DF Analytics Results: do not hide query bar and table on query syntax error (#69196) * do not hide query bar and table if query syntax error * check for query error in analytics exploration [Ingest Manager] Use search param vs KQL for events count (#69209) * Replace filter with search in events count Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [Ingest Manager] Use search param vs KQL to find default agent (#69211) * Remove another instance of filter for is_default Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [SIEM][Detection Engine] Fixes a stale view/query within the signals table when re-opening/closing signals Fixes a one liner bug where we were not using waitFor and could have stale views of data from the timeline. This removes that by adding a `waitFor` when setting the signals to be closed, open, in-progress, etc... This also adds a new `waitFor` for the end to end tests and fixes other tests to use that rather than doing a wait for 5 seconds. This should keep the end to end backend tests fast. Before this you could sometimes try to re-open 3 signals like below and it would not change the signals to being open but rather re-query the stale view and show the same signals as being closed when they are not closed: ![crop_bug](https://user-images.githubusercontent.com/1151048/84713572-8b5e7f00-af28-11ea-80b6-2ad67f16b7df.gif) Now with this PR, those signals will show as being re-opened and the closed signals will update correctly within the same UI on the same tab. - [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios [Observability] Update landing page copy and content (#69247) * removing try it section * removing try it section feat: instrument navigation changes using RUM agent (#67633) * feat: instrument navigation changes using RUM agent * chore: rebase and change application contract * chore: fix type tests * docs: update public.md doc * chore: remove internal application export [Uptime] Monitor availability reporting (#67790) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [DOCS] Adds classification to data frame analytics overview (#69068) Bump react-redux (#69182) [kbn/es] only make one attempt in tests to avoid timeout (#69197) Co-authored-by: spalger <spalger@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Resolve security cloud test failures (#68935) Co-authored-by: Joe Portner <5295965+jportner@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> SIEM] Moves validation up to the common section Moves validation up to the common section so it can be used by others in common for API boundary validation. [QA] Code coverage: fix flaky tests (#69272) * skip test * [page_objects/common_page] update closing toast use navigateToUrl instead of window location (#69167) [kbn/optimizer] fix windows compatibility (#69304) Co-authored-by: spalger <spalger@users.noreply.github.com> [Search service] Refactor the way server-side search strategies are registered (#68452) * [search] Refactor the way search strategies are registered/retrieved on the server * Fix types and tests and update docs * Fix failing test * Fix build of example plugin * Fix functional test * Make server strategies sync Co-authored-by: Liza K <liza.katz@elastic.co> [ci/getCheckoutInfo] retry fetching upstream changes to calculate mergeBase (#69320) Co-authored-by: spalger <spalger@users.noreply.github.com> [QA] [Code Coverage] Doc update (#69204) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [CI] Fix packer cache git branch reference value (#69207) [kbn/pm] only count cached project (#69113) Co-authored-by: spalger <spalger@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [Reporting] Prepare export type definitions for Task Manager (#65213) [APM] Fix service maps not loading when there are no APM ML jobs (#69240) * Closes #69238 by handling 404 thrown error from the query for APM ML jobs. * Improved coded readability * moved anomaly job fetch to new function getApmAnomalyDetectionJobs for improved readability and only handle a 404 status code response else throw [BundleRefPlugin] resolve imports to files too (#69241) Co-authored-by: spalger <spalger@users.noreply.github.com> [DOCS] Fixes license management links (#69347) [Component template] Details flyout (#68732) [SIEM] Adds example unit test to convert KQL using a nested query Adds example unit test to convert KQL using a nested query - [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios Explore underlying data (#68496) * feat: 🎸 stub discover_enhanced plugin * feat: 🎸 improve view in discover action * feat: 🎸 add URL generator to "View in Discover" action * feat: 🎸 implement navigation and getHref in view raw logs actio * fix: 🐛 disable action in "edit" mode * refactor: 💡 renamce context menu view in discover action * feat: 🎸 rename action to "explore data" * fix: 🐛 correctly generate action path * feat: 🎸 add internationalization to "explore action" * fix: 🐛 correctly parse generated Discover URL path * test: 💍 setup basic functional tests * refactor: 💡 modularize url generation logic * feat: 🎸 export CommonlyUsed type * test: 💍 add test subjects to panel custom time range modal * test: 💍 add index patterna and time range functional tests * refactor: 💡 rename action file * refactor: 💡 use URL generator from Discover plugin's contract * test: 💍 add "Explore raw data" action unit tests * fix: 🐛 import share plugin to check if it is enabled * Update x-pack/plugins/discover_enhanced/public/actions/view_in_discover/explore_data_context_menu_action.ts Co-authored-by: Matthias Wilhelm <ankertal@gmail.com> * chore: 🤖 add discover_enhanced to KibanaApp codeowners * test: 💍 improve "Explore underlying data" functional tests * test: 💍 improve <a> link assertion Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Matthias Wilhelm <ankertal@gmail.com> [APM] Only add decimals for numbers below 10 (#69334) Replaces the Custom Color Picker on TSVB with the EuiColorPicker (#68888) * Replace the Custom Color Picker on TSVB with the EuiColorPicker * Remove the custom picker sass * Remove private modules of eui and the custom color swatches * Clear the color * changes in test implementation Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [SIEM][Timeline] Persist timeline to localStorage (#67156) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Xavier Mouligneau <189600+XavierM@users.noreply.github.com> feat: 🎸 don't show drilldown action in "edit" mode (#69371) [ML] Functional tests - add more recognize and setup module API tests (#69251) This PR adds a couple recognize module and setup module API integration tests. [ML] Functional tests - Reduce DFA job model memory (#69295) With the improvements in the model memory estimation for DFA jobs, the required memory limit could be reduced, which allows execution in a low-memory environment (like the 1GB ML node in a cloud trial). Update dependency @elastic/charts to v19.5.2 (#69126) Update @elastic/charts to 19.5.2 Co-authored-by: nickofthyme <nick.ryan.partridge@gmail.com> [SIEM][Timeline] Minor timeline improvement (#69386) [DOCS] Updates titles in Maps docs (#68703) * [DOCS] Updates titles in Maps docs * [DOCS] Changes Elastic Maps to Maps to match the UI * [DOCS] Fixes titles Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [Ingest Manager][Endpoint] Add Endpoint Create Policy flow with Ingest (#68955) * Ingest: add data-test-subj prop support to Ingest components * Ingest: Add Context Provider to bridge History from Kibana to Hash router * Ingest: Added support for route state in Create Datasource page * Endpoint: Add Create button to Polices List header * Endpoint: Added support for passing of state from endpoint to ingest on policy * Endpoint: additional functional test cases [APM] Replace ML index queries with searching via mlAnomalySearch API (#69099) * Closes #69092 by replacing direct queries on ml indices with seaching via the `mlAnomalySearch` client API + job_id filters. Also removes `getMlIndex` since it is no longer relevant. * Use the mlCapabilities API to ensure the required license is active for ml queries Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Drilldown docs 2 (#69375) * docs: ✏️ add drilldown user docs link to REAMDE * feat: 🎸 wire drilldown docs link into <HelloBar> component * feat: 🎸 set new page attributes on drilldown docs link * feat: 🎸 add external prop to link to show icon * fix: 🐛 remove rel= attribute * fix: 🐛 add TypeScript type for drilldown docLink [Endpoint] add policy data to Host list UI (#69202) [Ingest Pipelines] Add doc links to processor types (#69279) * added doc links to processor types * Bring doc links in line with mappings editor Also refactor the processors type map * remove helpText prop from Custom field * fix i18n * rename doc button and refactor type map const name Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Removing flaky axe rule (#69418) [Ingest Manager] Replace `datasources` with `inputs` when generating agent config (#69226) * Adjust agent config generation, change `datasources` to `inputs` * Add dataset.type * Remove dead code * Revert "Add dataset.type" This reverts commit fbcf50cbe2b6a93536386c82d754a6a7c7f2b8e9. * Update endpoint policy test assertion [ML] Add Anomaly Swimlane Embeddable to the dashboard from the Anomaly Explorer page (#68784) * [ML] WIP attach swimlane embeddable to dashboard from the explorer page * [ML] fix deps * [ML] getDefaultPanelTitle * [ML] fix TS issue * [ML] DashboardService * [ML] unit tests * [ML] redirect to the dashboard * [ML] swimlane_panel * [ML] Anomaly Timeline panel * [ML] swimlane container * [ML] fix ts * [ML] Add multiple swimlanes * [ML] fix SwimlaneType usage * [ML] disable edit button on update * [ML] fix i18n translation key * [ML] use ViewMode enum * [ML] use navigateToUrl * [ML] TODO for edit dashboard * [ML] check kibana dashboard capabilities * [ML] mlApiServicesProvider * [ML] mlResultsServiceProvider * [ML] fix alignment * [ML] labels and tooltips * [ML] fix ts issue for proxyHttpStart * [ML] canEditDashboards check * [ML] fix TS * [ML] update add_to_dashboard_control.tsx * [ML] add form label, disable control on empty swimlanes selection * [ML] resolve PR review comments * [ML] e2e test * [ML] increase panel padding * [ML] position in row * [ML] update e2e * [ML] add data-test-subj for search box * [ML] PR remarks [docker] add spaces settings (#69019) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [Maps] layer wizard select re-design (#69313) * [Maps] layer wizard select re-design * review feedback * tslint * add unit test * use smaller gutters * review feedback skip flaky suite (#52854) [APM] License feature tracking for service maps (#69455) Use the license feature API to register the service maps feature and track its usage when the API endpoint is accessed. Fixes #64850. fixes screenshots upload (#69392) Remove unused deps (#69243) * remove unused deps * move types in devDeps [Observability] Creates "Add data" links in all Observability app headers (#69016) * Add data options to all Observability app headers * Updated failing snapshot * [Uptime] Update snapshot Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Beats Management: Pre-migration cleanup (#69155) * adapt KibanaDatabaseAdapter to use core ES service * remove unused `exposeStaticDir` method * create empty KP plugin * remove unused and broken tests * delete unused wallaby config file * delete unused utils * delete unused d.ts and move formsy definition to new plugin * fix findNonExistentItems * remove dead code and useless exports from common package * delete non migratable test suite + remove unused test adapter * remove yet another unused adapter * restore awaits in KibanaDatabaseAdapter Add section about marble testing to `TESTING.md` (#68749) * Add section about marble testing * improve `callServerAPI` example * review comments * add comment on abort observable anti-pattern savedObjects: add `score` to repository.find results (#68894) * add `score` to repository.find results * update generated doc * fix FTR result set * remove score from exports * fix FTR for find API * fix label * fix tsdoc Add functional test for Kibana embedded in iframe (#68544) * convert kbn test config into TS * add test for Kibana embedded in iframe * run embedded tests in functional suite * ignore tls errors in functional tests by default * switch test to https * remove env vars mutation * allow to pass ssl config to Kibana * pass ssl config to axios * adopt KbnClient interfaces * adopt KibanaServer * use KbnRequester in security service * set sameSiteCookies:None in test * acceptInsecureCerts in chrome * remove leftovers * fix type error * remove unnecessary field * address comments * refactor plugin * refactor test * make acceptInsecureCerts configurable * run firefox tests on ci * up TS version * fix firefox.sh script * fix path Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [APM] Add error rate chart to Errors overview and detail views (#67327) * creating error rate chart * adding error line chart * creating error rate chart * using date_histogram * reapplying prettier style * changing to theme color * dont sync tooltips * adding avg on error charts * addressing pr comments * adding possibility to disable legend toggle * removing x-axis ticks from histogram * return no percent when transaction count doesn return hits * addressing PR comments * addressing PR comments * returning null when there is no transaction count Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [APM] Change the unsaved changes text color (#69493) Fixes dark mode issue where the text would render dark on a dark background. [refactor] remove drilldown plugin, move components to uiActionsEnhanced (#69403) * refactor: remove drilldown plugin by moving components to uiActionsEnhanced * fix * fix mistake in test * fix i18n HttpService: duplicate some APIs from `setup` to `start` and clean mocks. (#69021) * add apis to http start contract and clean mocks * add apis to http start contract and clean mocks * add `getStartContract` method * fix new calls * remove isTlsEnabled * deprecates HttpServiceSetup.auth [SIEM] Fixes REST formatter bugs from io-ts migration Fixes io-ts formatter bugs for REST and validation by: * First trying to get the correct key from the io-ts context. If no keys are found, then it will fall back on trying to get the first name from the context. * If the key is a value and an object then this will do a `JSON.stringify()` on the value object * This fixes a few places where `formatError` was not being used within the code base resulting in `[object Object]` within the validations to show up. - [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios [RUM Dashboard] Initial Version (#68778) Co-authored-by: Casper Hübertz <casper@formgeist.com> Fix home page loading if telemetry plugin disabled (#69394) * Fix home page loading * Fix jest test, update telemetry mocks [Maps] Remove bounds param (#69396) [IngestManager] fix endpoint setup in api integration tests (#69503) [APM] Service map download in debug mode (#69350) * [APM] Service map download in debug mode Add a download button when debug mode is enabled that downloads JSON of the map. Add an upload button to the Storybook. Embed documentation on input and output state (#69443) * Embed documentation on input and output state * json -> js * Add section on how id is used by custom time range badge action to determine isCompatible Revert "[RUM Dashboard] Initial Version (#68778)" This reverts commit 72111702e95214594d5d56bc856e1d5c78e6d802. [Ingest Manager] Fleet require encrypted saved object encryption key … (#69408) [rfc][skip-ci][reporting] Rendering API RFC (#64372) * WIP working on reporting rfc round 2 * First draft is complete [Index template] Refactor index template wizard (#69037) Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com> Amend export used for default (#69158) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [SIEM][Exceptions] - Updates exception structure and corresponding UI types (#69120) This PR is meant to update the `ExceptionListItemSchema.entries` structure to align with the most recent conversations regarding the need for a more explicit depiction of `nested` fields. To summarize: - Adds schema validation for requests and responses within `lists/public/exceptions/api.ts`. It was super helpful in catching existing bugs. Anyone that uses the api will run through this validation. If the client tries to send up a malformed request, the request will not be made and an error returned. If the request is successful, but somehow the response is malformed, an error is returned. There may be some UX things to figure out about how to best communicate these errors to the user, or if surfacing the raw error is fine. - Updates `entries` structure in lists plugin api - Updates hooks and tests within `lists/public` that make reference to new structure - Updates and adds unit tests for updated schemas - Removes unused temporary types in `security_solution/public/common/components/exceptions/` to now reference updated schema - Updates UI tests - Updates `lists/server/scripts` [Ingest Pipelines] Add test coverage for ingest pipelines editor component (#69283) * first iteration of CIT tests * address pr feedback - use dot notation where we can - use string literals instead of + concatentation [Ingest Pipelines] Encode URI component pipeline names (#69489) * Properly encode URI component pipeline names * safely URI decode to handle % case [Console] Added license headers to worker files (#69387) * added license headers to worker files * add missing @notice * update notice.txt * fix notice * remove start-end license block markers * Added license pre-amble don't include group fields with no child fields in index pattern (#69457) Update endpoint event and alert types (#69292) * start redoing types * finish redoing types * fix bad test * rework tests * fix more types * fix test * Fix endpoints test and render error * add deletePolicyStream to alerts api tests Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Paul Tavares <paul.tavares@elastic.co> [Security Solution] Add cypress tests for global search bar (#68535) [RUM Dashboard] Initial version resubmit (#69531) Co-authored-by: Casper Hübertz <casper@formgeist.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [Ingest Manager] Use long polling for agent checkin (#68922) Fixes home page welcome link (#69539) [Maps] Migrate maps client router to react (#65079) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [CCR] Fix follower indices table not updating after pausing (#69228) Document authentication settings. (#69284) Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co> Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com> [SIEM][Detection Engine] Fixes 7.8 and 7.9 upgrade issue within rules where you can get the error "params invalid: [lists]: definition for this key is missing" * https://github.com/elastic/kibana/issues/69463 * See here for manual backport to 7.8: https://github.com/elastic/kibana/pull/69434 This fixes a bug where if you import rules and set your overwrite to `true` multiple times in a row within 7.7 you can end up with a lists array. When upgrading to 7.8, we change the name of `lists` to `exceptions_lists` and suddenly when you enable/disable a rule you can get the following error below: ![image](https://user-images.githubusercontent.com/1151048/84945824-fa60e280-b0a4-11ea-8e05-bffdec2e4765.png) The fix is to allow the lists array still if it is present within saved objects to avoid seeing this error screen and being tolerant. We also fix the area of code that is causing the data bug so it cannot happen again with `exceptions_list` which is what the name of lists was renamed to causing this problem. Note that this has unit tests and I also manually tested this by intentionally injecting a `lists` and `exceptions_lists` and using the UI to verify there wasn't another validation spot that needed to be relaxed to allow for the data. - [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios [DOCS] Adds kibana-pull attribute for release docs (#69554) skip tests using hostDetailsPolicyResponseActionBadge Merge/restyle nodes table (#69098) Adds panel views and drilldowns to Resolver [ftr] add support for docker servers (#68173) Co-authored-by: spalger <spalger@users.noreply.github.com> Remove endpoint alert fields from signal mapping (#68934) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [DOCS] Add related link to the ingest management docs (#69467) * [DOCS] Add related link to the ingest management docs * Add link to ingest manager topic in Kibana * Remove link to ingest manager topic in kibana Resolving conflicts (#69597) [ML] Data Grid Histograms (#68359) Adds support for histogram charts to data grid columns. - Adds a toggle button to the data grid's header to enabled/disable column charts. - When enabled, the charts get rendered as part of the data grid header. - Histogram charts will get rendered for fields based on date, number, string and boolean. [APM] Add support for dark mode (#69362) * Use theme colors for stacktrace * [APM] Use theme for all components * Switch HoC withTheme to useTheme and convert classes to hooks * Change hardcoded white to euiColorEmptyShade for ServiceMap * Snapshots and Legends fix * Switch to context and add test helper * Fix tests and update snaps * New snaps + new anomaly detection * Remove shallow from testHelpers * Remove commented tests * Fix prettier * Pass correct theme to cytoscape * Fix ServiceMap * fixes some rendering issues in service maps * removed the old anomaly detection logic from service map popover contents, since it's been moved to a new component. * Fix eslint, tsc lint issues and unit tests * Remove types for styled-components default theme * Update x-pack/plugins/apm/public/components/shared/KueryBar/Typeahead/Suggestions.js Co-authored-by: Casper Hübertz <casper@formgeist.com> * fix OuterTheme * Ise function declaration instead of expression Co-authored-by: Balthazar Gronon <git@balthazar.dev> Co-authored-by: Balthazar Gronon <bgronon@gmail.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Oliver Gupte <olivergupte@gmail.com> Co-authored-by: Casper Hübertz <casper@formgeist.com> [Endpoint] add policy empty state (#69449) Fixing resolver alert generation (#69587) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [Security_Solution][Endpoint] Resolver leverage ancestry array for queries (#69264) * Adding alerts route * Adding related alerts generator changes, tests, and script updates * Fixing missed parameter * Aligning the AlertEvent and ResolverEvent definition * Fixing type errors * Fixing import error * Adding ancestry functionality in generator * Creating some tests for ancestry field * Making progress on the ancestry * Fixing the ancestry verification * Fixing existing tests * Removing unused code and fixing test * Adding more comments * Fixing endgame queries Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> skip failing suite (#69595) [Endpoint] Fix flaky endpoints list unit test (#69591) * Fix flaky endpoints list unit test * un-skip test Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> disable pageLoadMetrics job, it's gotten really flaky [SECURITY] Introduce kibana nav (#68862) * Change the bootstrap of the app * rename SiemPageName to SecurityPageName * modify alerts routes * modify cases routes * modify hosts routes * modify network routes * modify overview routes * modify timelines routes * wip change management route * change route for common * some fixing from the first commit * modify route for management * update url format hook to use history * bug when you click on external alerts from host or network * improvement from josh feedback * redirect siem to security solution * a little clean up * Fix types * fix breadcrumbs * fix unit test * Update index.tsx * Fix cypress * bug remove timeline when you are in case configure * Fix functionel test for management * Fix redirect siem + ml * fixes some cypress tests * adds 'URL compatibility' test * bring ml back to alerts * review I * Fix memory leak in timelines page * fix storage bug for timeline search bar * fix endpoint merge + functional test * avoid timeline flyout toggle * Fix link to ml score * Fix breadcrumb * Fix management url * fix unit test * fixes typecheck issue * fixes remaining url cypress tests * fixes timeline scenario * fix link to details rule from timeline * review remove absolute path for consistency * Fixing resolver alert generation (#69587) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> * [Security_Solution][Endpoint] Resolver leverage ancestry array for queries (#69264) * Adding alerts route * Adding related alerts generator changes, tests, and script updates * Fixing missed parameter * Aligning the AlertEvent and ResolverEvent definition * Fixing type errors * Fixing import error * Adding ancestry functionality in generator * Creating some tests for ancestry field * Making progress on the ancestry * Fixing the ancestry verification * Fixing existing tests * Removing unused code and fixing test * Adding more comments * Fixing endgame queries Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> * fix cypress test * skip failing suite (#69595) * [Endpoint] Fix flaky endpoints list unit test (#69591) * Fix flaky endpoints list unit test * un-skip test Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> * remove flaky test Co-authored-by: patrykkopycinski <contact@patrykkopycinski.com> Co-authored-by: Gloria Hornero <snootchie.boochies@gmail.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Jonathan Buttner <56361221+jonathan-buttner@users.noreply.github.com> Co-authored-by: spalger <spalger@users.noreply.github.com> Co-authored-by: Paul Tavares <56442535+paul-tavares@users.noreply.github.com> Bump jest related packages (#58095) [IM] Move template wizard steps to shared directory + update legacy details panel (#69559) Migrate dashboard mode (#69305) Closes: #67469 Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [Observability]Adding specific Return APIs for each plugin (#69257) * Adding specific apis for each plugin * adding metric hosts stat * addressing PR comment * addressing PR comments * changing series to key/value * exporting interfaces * adding label to stat * refactoring types Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [ML] fix ml api services (#69681) [Discover] Unskip context navigation functional test (#68771) [APM] Use asPercent to format breakdown chart (#69384) Co-authored-by: Søren Louv-Jansen <sorenlouv@gmail.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Updates swim lane UI text (#69454) Fix Advanced Settings Panel number editing in Graph (#69672) Remove App communication from URL (#67064) Removed all inter-app communication via url in favour of a new service in the embeddable start contract called the state transfer service. [Security_Solution] Split up indices (#69589) * Fixing resolver alert generation * Splitting indices up * Removing tests that could randomly fail because of the generation code * Adding support for multiple indices * Updating archives with the new index names * Removing alerts data stream * Switching to process instead of fake Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> completes top-level navigation tests (#69694) [CI] Record Github commit statuses outside of PRs (#69432) Document workaround for using HashRouter (#69140) * add section on hashrouter * review comments Disabled multiple select for preconfigured connectors to avoid requesting bulk delete on them (#69459) [chore] TS 3.9: convert ts-ignore to ts-expect-error (#69541) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [Monitoring] Love for APM (#69052) * Fix broken colors for APM * Use default derivative * Fix UI issues with APM * Add new charts * Fix tests * Use EUI color palette * Remove old translations * PR feedback * Fix tests * Fix up overview page Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [ML] Transform: Table enhancements (#69307) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [ML] Transform: Enable force delete if one of the transforms failed (#69472) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> fix link to analytics results from management (#69550) adds kibana navigation tests (#69733) Add plugin API for customizing the logging configuration (#68704) [Maps] Remove extra layer of telemetry nesting under "attributes" (#66137) * Return attributes when telemetry created instead of whole saved object. Update integration test * Change 'maps-telemetry' to 'maps' * No need to create a saved object anymore. This is leftover from task manager telemetry mgmt * Add test confirming attrs undefined. Change tests to check for 'maps' iso 'maps-telemetry' * Add two more tests confirming expected telemetry shape * Review feedback. Use TELEMETRY_TYPE constant and set to APP_ID [Index Management] Fix API Integration Test and use of `timestamp_field` (#69666) * fix types and functional api integration test * access timestamp field name in object * temporarily skip the API integration test and fix ts issue [SECURITY] Add endpoint alerts url (#69707) * Add back endpoint alerts url * hack to move on * fix type * fix test remove scroll in drag & drop context (#69710) [Metrics UI] Add inventory alert preview (#68909) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [Obs] Update Observability landing page text (#69727) Support deep links inside of `RelayState` for SAML IdP initiated login. (#69401) [ML] Fixes anomaly chart and validation for one week bucket span (#69671) * [ML] Fixes anomaly chart and validation for one week bucket span * [ML] Fix interval Jest tests Migrate legacy import/export endpoints (#69474) * migrate legacy export routes to `legacy_export` plugin * adapt unit tests * remove already dead (already moved) libs Fixes #69344: Don't allow empty string for server.basePath config (#69377) * Fixes #69344: Don't allow empty string for server.basePath config * Remove unused basepath group Redirect to Logged Out UI on SAML Logout Response. Prefer Login Selector UI to Logged Out UI whenever possible. (#69676) Properly redirect legacy URLs (#68284) expose DocLinks API from start only (#68745) * exose DocLinks API from start only * update docs * fix type errors * update docs Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> remove top-nav. it is not rendered in KP (#69488) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [APM] Storybook theme fixes (#69730) * [APM] Storybook theme fixes The changes adding theme support in #69362 broke some of the Storybook stories. Add decorators to wrap some of the stories in the theme context. This should be done in a global decorator, but our current storybook setup doesn't support this. It also would be nice to be able to switch between light/dark mode, but that's something we can add in the future. * Remove unused import * Adds missing decorator to cytoscape examples + adds a new real-world example Co-authored-by: Oliver Gupte <olivergupte@gmail.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> adding Stats interface with type (#69784) [data.search.aggs]: Add AggConfig.toSerializedFieldFormat (#69114) [Ingest Manager] Do not await in start. Return a Promise (#69505) 1. Do not `await` in the public `start` lifecycle method. Fixes https://github.com/elastic/kibana/issues/66125 PR based on https://github.com/elastic/kibana/issues/66125#issuecomment-640790837 & https://github.com/elastic/kibana/issues/66125#issuecomment-642218799 2. Change `success` to be Promise [pre-req] Convert Palettes and Components (#69065) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [Lens] Stabilize filter popover (#69519) * stabilize filter popovwer * remove text exclusion Add lists plugin to optimized security_solution TS config (#69705) As security_solution continues to integrate with lists, the absents of these types will lead to lots of implicit anys and false positives. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [data.search.aggs] Move agg-specific field formats to search service (#69586) [Security][Network] Exclude glob-only (*) Index Pattern from map layers (#69736) * Exclude glob-only (*) index pattern from map layers This pattern is a special case that our map should ignore, as including it causes all indexes to be queried. * Ignore CCS glob pattern in our embedded map Users may have this pattern for cross-cluster search, and it should similarly be excluded when matching Security indexes. [IM] Move common step containers to shared (#69713) [Maps] New mappings: maps-telemetry -> maps (#69816) Don't set a min-length on encryption key for reportin (#69827) [ML] DF Analytics Creation: add progress indicator (#69583) * add progress indicator to creation wizard page * only show progress bar if job is started immediately * add title and switch to timeout * fix progress check * clean up interval on unmount * fix types * clear interval if stats undefined. show progress if job created [Uptime] Fix charts dark theme (#69748) [Ingest Manager] Support registration of server side callbacks for Create Datasource API (#69428) * Ingest: Expose `registerExternalCallback()` method out of Ingest server `start` lifecycle * Ingest: Add support for External Callbacks on REST `createDatasourceHandler()` * Ingest: expose DatasourceServices to Plugin start interface * Endpoint: Added Endpoint Ingest handler for Create Datasources - Also moved the temporary logic from the middleware to the handler (still temporary) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> [IngestManager] Expose agent authentication using access key (#69650) * [IngestManager] Expose agent authentication using access key * Add unit tests to authenticateAgentWithAccessToken service [APM] Pulls legacy ML code from service maps and integrations (#69779) * Pulls out existing ML integration from the service maps * - removes ML job creation flyout in integrations menu on the service details UI - removes ML searches and transforms in the transaction charts API - removes unused shared functions and types related to the legacy ML integration * removes unused translations for APM anomaly detection * Adds tags to TODOs for easy searching later Convert Positionable, RenderToDom and RenderWithFn to functional/hooks/no recompose. (#68202) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Don't use filter to get default config & output

50bf157

the KQL parsing takes up a lot of CPU time. Avoid it with a search string Also add a `refresh: false` as suggested by @bkobel

elasticmachine and others added 2 commits June 14, 2020 17:52

Merge branch 'master' into remove-filter-from-enroll-route

f558eb8

Does restoring await fix CI?

d9c0339

jfsiii added Team:Fleet Team label for Observability Data Collection Fleet team v7.9.0 v8.0.0 labels Jun 15, 2020

jfsiii self-assigned this Jun 15, 2020

jfsiii marked this pull request as ready for review June 15, 2020 13:28

jfsiii requested a review from a team June 15, 2020 13:28

nchaulet reviewed Jun 15, 2020

View reviewed changes

Remove 'refresh: false' from agent enroll

bb489d0

Does this this fix the failing test? It does keep the PR more focused

jfsiii added the release_note:skip Skip the PR/issue when compiling release notes label Jun 15, 2020

nchaulet approved these changes Jun 15, 2020

View reviewed changes

jfsiii merged commit 0ce2970 into elastic:master Jun 15, 2020

This was referenced Jun 15, 2020

[Ingest Manager] Use search param vs KQL for events count #69209

Merged

[Ingest Manager] Use search param vs KQL to find default agent #69211

Merged

jfsiii mentioned this pull request Jun 16, 2020

[7.x] Don't use filter to get default config & output (#69088) #69325

Merged

jfsiii mentioned this pull request Jun 29, 2020

[Ingest Manager] How to reduce the overhead of Kibana in the work our routes does #67989

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Don't use filter to get default config & output #69088

Don't use filter to get default config & output #69088

jfsiii commented Jun 12, 2020 •

edited

Loading

jfsiii commented Jun 14, 2020

elasticmachine commented Jun 15, 2020

jfsiii commented Jun 15, 2020

nchaulet Jun 15, 2020

jfsiii Jun 15, 2020 •

edited

Loading

nchaulet Jun 15, 2020

kibanamachine commented Jun 15, 2020

jfsiii commented Jun 15, 2020

jfsiii commented Jun 15, 2020

ruflin commented Jun 16, 2020

Don't use filter to get default config & output #69088

Don't use filter to get default config & output #69088

Conversation

jfsiii commented Jun 12, 2020 • edited Loading

Summary

Current

PR

Current

PR

jfsiii commented Jun 14, 2020

elasticmachine commented Jun 15, 2020

jfsiii commented Jun 15, 2020

nchaulet Jun 15, 2020

Choose a reason for hiding this comment

jfsiii Jun 15, 2020 • edited Loading

Choose a reason for hiding this comment

nchaulet Jun 15, 2020

Choose a reason for hiding this comment

kibanamachine commented Jun 15, 2020

💚 Build Succeeded

Build metrics

History

jfsiii commented Jun 15, 2020

jfsiii commented Jun 15, 2020

ruflin commented Jun 16, 2020

jfsiii commented Jun 12, 2020 •

edited

Loading

jfsiii Jun 15, 2020 •

edited

Loading