logstash-* index pattern is created but is not the default

[LeeDr]

Kibana version: 5.0 (master)

OS version: Windows 10 And on Jenkins

Original install method (e.g. download page, yum, from source, etc.): dev environment

Description of the problem including expected versus actual behavior: The automated functional tests have failed at least 2 times on this test
'creating and deleting default index_should allow setting advanced settings'

When it fails, it takes a screenshot which shows a logstash-* index pattern but it does not have the star to mark it as the default index. This causes the "Configure an index pattern" screen to appear when the test isn't expecting it and fails the test.

Steps to reproduce:

1. The before method of the test deletes the .kibana index and waits for it to be created again esClient.deleteAndUpdateConfigDoc();
2. it creates the index pattern settingsPage.createIndexPattern();
3. it navigates to settings settingsPage.navigateTo();
4. then the test attempts to click the advanced settings with settingsPage.clickKibanaSettings() https://github.com/elastic/kibana/blob/master/test/functional/apps/management/_kibana_settings.js#L29 but the link isn't there because logstash-* isn't set to be the default index pattern.

Expected: since this test uses the UI to create the logstash-* index pattern just like a user would, we expect it is automatically set to be the default index.

I'll try to troubleshoot the issue more and add more details if I get any.

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

✓ chrome on any platform - kibana - settings app - index result popularity - change popularity - can be saved (1.964s)
08:53:30.076: tryForTime failure: Index pattern not removed
08:53:30.616: tryForTime failure: [index_not_found_exception] no such index, with { resource.type=index_or_alias resource.id=.kibana index_uuid=_na_ index=.kibana }
08:53:31.120: tryForTime failure: [index_not_found_exception] no such index, with { resource.type=index_or_alias resource.id=.kibana index_uuid=_na_ index=.kibana }
08:53:31.625: tryForTime failure: [index_not_found_exception] no such index, with { resource.type=index_or_alias resource.id=.kibana index_uuid=_na_ index=.kibana }
08:53:32.129: tryForTime failure: [index_not_found_exception] no such index, with { resource.type=index_or_alias resource.id=.kibana index_uuid=_na_ index=.kibana }
08:53:32.634: tryForTime failure: [index_not_found_exception] no such index, with { resource.type=index_or_alias resource.id=.kibana index_uuid=_na_ index=.kibana }
08:53:33.138: tryForTime failure: [index_not_found_exception] no such index, with { resource.type=index_or_alias resource.id=.kibana index_uuid=_na_ index=.kibana }
08:53:33.642: tryForTime failure: [index_not_found_exception] no such index, with { resource.type=index_or_alias resource.id=.kibana index_uuid=_na_ index=.kibana }
08:53:34.147: tryForTime failure: [index_not_found_exception] no such index, with { resource.type=index_or_alias resource.id=.kibana index_uuid=_na_ index=.kibana }
08:53:34.651: tryForTime failure: [index_not_found_exception] no such index, with { resource.type=index_or_alias resource.id=.kibana index_uuid=_na_ index=.kibana }
08:53:35.155: tryForTime failure: [index_not_found_exception] no such index, with { resource.type=index_or_alias resource.id=.kibana index_uuid=_na_ index=.kibana }
08:53:35.660: tryForTime failure: [index_not_found_exception] no such index, with { resource.type=index_or_alias resource.id=.kibana index_uuid=_na_ index=.kibana }
08:53:36.164: tryForTime failure: [index_not_found_exception] no such index, with { resource.type=index_or_alias resource.id=.kibana index_uuid=_na_ index=.kibana }
08:53:36.669: tryForTime failure: [index_not_found_exception] no such index, with { resource.type=index_or_alias resource.id=.kibana index_uuid=_na_ index=.kibana }
08:53:37.174: tryForTime failure: [index_not_found_exception] no such index, with { resource.type=index_or_alias resource.id=.kibana index_uuid=_na_ index=.kibana }
08:53:37.678: tryForTime failure: [index_not_found_exception] no such index, with { resource.type=index_or_alias resource.id=.kibana index_uuid=_na_ index=.kibana }
08:53:38.182: tryForTime failure: Cannot read property '_id' of undefined
08:53:38.686: tryForTime failure: Cannot read property '_id' of undefined
08:53:39.191: config._id =5.0.0-alpha4
08:53:39.191: navigating to settings url: http://localhost:5620/app/kibana#/management
08:53:39.191: navigate to: http://localhost:5620/app/kibana#/management
08:53:39.249: ... sleep(700) start
08:53:39.950: ... sleep(700) end
08:53:39.951: returned from get, calling refresh
08:53:40.363: check testStatusPage
08:53:40.363: self.checkForKibanaApp()
08:53:41.784: current application: kibana
08:53:41.784: kibanaLoaded = true
08:53:41.793: ... sleep(501) start
08:53:42.293: ... sleep(501) end
08:53:42.299: in doNavigation url = http://localhost:5620/app/kibana#/management/data/index/?_g=()
08:53:42.935: currentUrl http://localhost:5620/app/kibana#/management/kibana/indices/logstash-*?_g=()&_a=(tab:indexedFields)
08:53:42.935: Index pattern created: http://localhost:5620/app/kibana#/management/kibana/indices/logstash-*?_g=()&_a=(tab:indexedFields)
08:53:42.937: navigating to settings url: http://localhost:5620/app/kibana#/management
08:53:42.937: navigate to: http://localhost:5620/app/kibana#/management
08:53:43.534: ... sleep(700) start
08:53:44.235: ... sleep(700) end
08:53:44.235: returned from get, calling refresh
08:53:44.582: check testStatusPage
08:53:44.582: self.checkForKibanaApp()
08:53:44.896: current application: kibana
08:53:44.896: kibanaLoaded = true
08:53:44.905: ... sleep(501) start
08:53:45.406: ... sleep(501) end
08:53:45.412: in doNavigation url = http://localhost:5620/app/kibana#/management/data/index/?_g=()
08:53:55.438: Taking screenshot "C:\git\kibana\test\screenshots\failure\failure_1466171635438_creating and deleting default index_should allow setting advanced settings.png"
× chrome on any platform - kibana - settings app - creating and deleting default index - should allow setting advanced settings (10.372s)
NoSuchElement: An element could not be located on the page using the given search parameters.
  at <node_modules\intern\node_modules\leadfoot\lib\findDisplayed.js:37:21>
  at <node_modules\intern\node_modules\dojo\Promise.ts:393:15>
  at run  <node_modules\intern\node_modules\dojo\Promise.ts:237:7>
  at <node_modules\intern\node_modules\dojo\nextTick.ts:44:3>
  at nextTickCallbackWith0Args  <node.js:415:9>
  at process._tickCallback  <node.js:344:13>
  at Command.findDisplayed  <node_modules\intern\node_modules\leadfoot\Command.js:23:10>
  at Command.prototype.(anonymous function) [as findDisplayedByCssSelector]  <node_modules\intern\node_modules\leadfoot\lib\strategies.js:28:16>
  at SettingsPage.clickPath  <test\support\pages\settings_page.js:18:26>
  at SettingsPage.clickKibanaSettings  <test\support\pages\settings_page.js:22:19>
  at Test.<anonymous>  <test\functional\apps\management\_kibana_settings.js:25:25>
  at tryCatcher  <node_modules\bluebird\js\main\util.js:26:23>
  at Promise.attempt.Promise.try  <node_modules\bluebird\js\main\method.js:31:24>
  at Test.test  <test\support\bdd_wrapper.js:15:14>
  at <node_modules\intern\lib\Test.js:181:24>
  at <node_modules\intern\browser_modules\dojo\Promise.ts:393:15>
  at runCallbacks  <node_modules\intern\browser_modules\dojo\Promise.ts:11:11>
  at <node_modules\intern\browser_modules\dojo\Promise.ts:317:4>
  at run  <node_modules\intern\browser_modules\dojo\Promise.ts:237:7>
  at <node_modules\intern\browser_modules\dojo\nextTick.ts:44:3>
  at nextTickCallbackWith0Args  <node.js:415:9>
  at process._tickCallback  <node.js:344:13>

[LeeDr]

I just ran the settings functional tests 14 times in a row and didn't hit this. But I know it CAN happen. Lowering from P3 to P4.

[LeeDr]

Just caused this build to fail http://build-eu-00.elastic.co/job/kibana_core_pr/4255
And here's the screenshot; http://build-eu-00.elastic.co/job/kibana_core_pr/4255/s3/download/failure_1466180739219_creating%20and%20deleting%20default%20index_should%20allow%20setting%20advanced%20settings.png

[LeeDr]

Not sure what I did with P labels back on June 17th..

[cwurm]

This is especially relevant when loading Beats dashboards into Kibana. They also contain index patterns, saved searches, visualisations and dashboards and so one ends up with having all data necessary in Kibana, but still having to set a default pattern when first logging in.

One can just select one of the existing patterns imported by Beats in the top left corner, but that seems like a not very obvious thing many first-time users will likely miss.

[LeeDr]

3 Kibana master jenkins jobs failed on this exact problem. It's not obvious from the selenium test output but the first clue is that it keeps trying to navigate to the discover tab but says the current url management/data because Kibana goes there when there's no default index.
The second clue is the screenshot taken when it fails shows a logstash-* index pattern but it's not set as the default. There could be 2 reasons for this. Either the esClient method used to update the config doc with the default index didn't work and we're missing error, or it did work but Kibana overwrote that update that the automation did behind it's back (see #7055).

tryForTime failure: App failed to load: discover in 10000ms appUrl = http://localhost:5620/app/kibana#/discover currentUrl = http://localhost:5620/app/kibana#/management/data/index/?_g=()

I'm going to try to work around this issue in the selenium tests by checking for logstash-* index pattern which is NOT default and setting it as the default, rather than have all the tests fail.