Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow kibana_settings collector to return nothing #22091

Merged

Conversation

tsullivan
Copy link
Member

@tsullivan tsullivan commented Aug 16, 2018

This fixes a bug that was causing a NPE with a Painless script inside a watch in Elasticsearch. It was found along with elastic/elasticsearch#32923

Important

This PR restores the kibana_settings data to the original behavior in 6.3.x, which was to NOT index multiple documents when an admin email is not set in the kibana settings. It doesn't fix a problem that users would see, but it does resolve a breaking, unexpected change in behavior for the data getting indexed.

@tsullivan
Copy link
Member Author

@chrisronline does this impact anything with #21100 ?

Copy link
Member

@pickypg pickypg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM except nitpick on comment

};
// return nothing when there was no result
let settingsDoc;
if (kibanaSettingsData != null) { // test null or undefined
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't catch undefined, but reading through the code it doesn't have to do that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++ I'd just update the comment to not say or undefined.

Copy link
Member Author

@tsullivan tsullivan Aug 16, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using double-equals to check for null or undefined is a habit I've acquired from working in Canvas:

> var foo
> foo == null ? "hi" : "bye"
"hi"

But you are right, the code doesn't have to do that. The comment is accurate though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh that's interesting and subtle. Not a fan :), but I suppose it works.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pushed b476642 to remove any unclarity

@tsullivan tsullivan removed the blocker label Aug 16, 2018
Copy link
Contributor

@ycombinator ycombinator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Tested with the metricbeat-indexing path as well and things still look good!

@tsullivan
Copy link
Member Author

I added a note in the description about why this PR is not a blocker fix.

@tsullivan
Copy link
Member Author

tsullivan commented Aug 16, 2018

@pickypg what do you think about just closing this PR? It doesn't fix a blocker.

A big concern Shaunak and I found is that this fix will not work in the long term: keeping track of state (shouldUseNull) in the collector is bad: it makes the API for settings (going out in 6.5) basically unusable.

@elasticmachine
Copy link
Contributor

💔 Build Failed

@pickypg
Copy link
Member

pickypg commented Aug 16, 2018

Personally I think changing it (by closing this PR) will cause a breaking change and also leave us with an unexpected / broken state in the future since we'd be intentionally leaving in a bug.

@pickypg
Copy link
Member

pickypg commented Aug 16, 2018

jenkins test this (Kibana failed to start in one test)

@chrisronline
Copy link
Contributor

A big concern Shaunak and I found is that this fix will not work in the long term: keeping track of state (shouldUseNull) in the collector is bad: it makes the API for settings (going out in 6.5) basically unusable.

I don't understand this concern? Can you explain this more please?

@chrisronline
Copy link
Contributor

chrisronline commented Aug 17, 2018

@chrisronline does this impact anything with #21100 ?

Yes, potentially. I'll address this in a follow up PR

Here: #22107

@elasticmachine
Copy link
Contributor

💔 Build Failed

@tsullivan
Copy link
Member Author

A big concern Shaunak and I found is that this fix will not work in the long term: keeping track of state (shouldUseNull) in the collector is bad: it makes the API for settings (going out in 6.5) basically unusable.

I don't understand this concern? Can you explain this more please?

Sorry, I was mistaken about that concern. There is an issue with the kibana_settings API: it should always return the same shape of data, whether default_admin_email is a valid address or a null. That problem does not need to be solved in the settings collector though, it can be solved in the API handler.

In my mistaken understanding, the change in this PR would be a temporary stop-gap. But @pickypg is right that this change is going to stop a breaking change from happening. There's always been this optimization to limit how many documents are indexed, via the statefulness of shouldUseNull. This PR should be merged, and the fix will stick.

I'm putting blocker label back on this PR.

@tsullivan
Copy link
Member Author

fail: "homepage app Kibana takes you home clicking on console on homepage should take you to console app"
23:47:22    │ proc  [ftr]      │        retry.try timeout: Error: retry.try timeout: [POST http://localhost:9515/session/1a166123967f0b2c3dd14376893656aa/element / {"using":"css selector","value":"[data-test-subj~=\"homeSynopsisLinkconsole\"]"}] no such element: Unable to locate element: {"method":"css selector","selector":"[data-test-subj~="homeSynopsisLinkconsole"]"}

jenkins test this

@jasontedor
Copy link
Member

Sorry, I was mistaken about that concern. There is an issue with the kibana_settings API: it should always return the same shape of data, whether default_admin_email is a valid address or a null.

+1

@chrisronline
Copy link
Contributor

There's always been this optimization to limit how many documents are indexed, via the statefulness of shouldUseNull

Is it fair to say this hasn't been the actual case though? Currently in master, we always return some structure so theoretically something is always indexed for each internal collector round trip? Or did you filter it out in the bulk uploader?

@tsullivan
Copy link
Member Author

Or did you filter it out in the bulk uploader?

It is intended to be filtered out in the bulk uploader, via this truthy check: https://github.com/elastic/kibana/blob/master/x-pack/plugins/monitoring/server/kibana_monitoring/bulk_uploader.js#L126

That truthy check was working in 6.3.x because the settings data result wasn't automatically decorated with the "kibana" object that comes from getKibanaInfoForStats.

Now, in master, the settings data is decorated with that kibana property to simplify the combining of different types into documents for bulk upload.

@elasticmachine
Copy link
Contributor

💔 Build Failed

@tsullivan
Copy link
Member Author

jenkins test this

Copy link
Contributor

@chrisronline chrisronline left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@elasticmachine
Copy link
Contributor

💔 Build Failed

@tsullivan
Copy link
Member Author

I'm consistently getting this test failure that has been filed as a flaky test: #21664

@tsullivan tsullivan merged commit 108d59c into elastic:master Aug 17, 2018
tsullivan added a commit to tsullivan/kibana that referenced this pull request Aug 17, 2018
* Fix kibana_settings collector to return nothing when no settings data is found

* make code more clear
tsullivan added a commit to tsullivan/kibana that referenced this pull request Aug 17, 2018
* Fix kibana_settings collector to return nothing when no settings data is found

* make code more clear
tsullivan added a commit that referenced this pull request Aug 17, 2018
* Fix kibana_settings collector to return nothing when no settings data is found

* make code more clear
tsullivan added a commit that referenced this pull request Aug 17, 2018
* Fix kibana_settings collector to return nothing when no settings data is found

* make code more clear
@tsullivan tsullivan deleted the fix/kibana-settings-uploader-for-null branch September 4, 2018 17:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants