Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server doesn't respond until elasticsearch is live #7278

Closed
jbudz opened this issue May 24, 2016 · 16 comments
Closed

Server doesn't respond until elasticsearch is live #7278

jbudz opened this issue May 24, 2016 · 16 comments
Assignees
Labels
blocker bug Fixes for quality problems that affect the customer experience v5.0.0-alpha4 v5.0.0

Comments

@jbudz
Copy link
Member

jbudz commented May 24, 2016

When kibana is running without elasticsearch, requests to /app/kibana will never reply
image

@jbudz jbudz added bug Fixes for quality problems that affect the customer experience v5.0.0-alpha3 labels May 24, 2016
@ycombinator ycombinator self-assigned this May 25, 2016
@LeeDr
Copy link

LeeDr commented May 25, 2016

There's also the x-pack issue 2200 where Kibana (and plugins?) only check for license when they start up. So if Kibana (with x-pack installed) starts before Elasticsearch it gives a license error.

@ycombinator
Copy link
Contributor

@LeeDr This issue seems to be unrelated to x-pack issue 2200. I just downloaded the kibana alpha3 tgz, extracted it and ran ./bin/kibana. I don't have any version of ES running at all. And I didn't install the x-pack plugin in Kibana either.

@jbudz
Copy link
Member Author

jbudz commented May 25, 2016

I'm guessing this is waiting for .kibana/config, which is injected into index.html now

@jbudz
Copy link
Member Author

jbudz commented May 25, 2016

@ycombinator
Copy link
Contributor

ycombinator commented May 25, 2016

@jbudz Thanks for pointing me to that. It's not actually that line, but the one after that: https://github.com/elastic/kibana/blob/master/src/ui/index.js#L75. If I comment it out (for testing), the status page loads up. Looking into a fix now...

ycombinator added a commit that referenced this issue May 25, 2016
If elasticsearch plugin is not ready, do not block
@epixa epixa reopened this May 28, 2016
@epixa
Copy link
Contributor

epixa commented May 28, 2016

This is still an issue on the latest master. If there's a version mismatch between Kibana and ES, then requests to Kibana just timeout instead of the user being dropped into the status page.

@jbudz
Copy link
Member Author

jbudz commented May 31, 2016

Also causing #7318

@ycombinator
Copy link
Contributor

Looking into this now (commenting to make it clear because I was already the assignee from the previous fix).

@ycombinator
Copy link
Contributor

@epixa I'm unable to reproduce this off the latest master (1cf2979). I'm running ES 2.3.2 instead of the compatible ES 5.0.0. The Kibana server logs show:

server    log   [09:00:40.410] [error][status][plugin:elasticsearch] Status changed from yellow to red - This version of Kibana requires Elasticsearch ^5.0.0 on all nodes. I found the following incompatible nodes in your cluster: Elasticsearch v2.3.2 @ 127.0.0.1:9200 (127.0.0.1)

And Kibana loads up in the browser on the status page with the same error message:

screen shot 2016-05-31 at 9 02 02 am

@epixa
Copy link
Contributor

epixa commented May 31, 2016

Change the version number in your local package.json to 5.0.0 (instead of 5.0.0-snapshot). Then run it against es master as you normally would.

@ycombinator
Copy link
Contributor

Yup, if I modify the package.json as you suggested, I am able to reproduce the issue. Thanks! I can run with this now...

@ycombinator
Copy link
Contributor

ycombinator commented Jun 1, 2016

Change the version number in your local package.json to 5.0.0 (instead of 5.0.0-snapshot). Then run it against es master as you normally would.

@epixa This seems to happen only in the case of going from a snapshot release to GA release. It does not happen when going from an alpha release to a GA release (I just verified this by tweaking the package.json.version in 5.0.0-alpha3). This is because of this line of code: https://github.com/elastic/kibana/blob/master/src/plugins/elasticsearch/lib/is_upgradeable.js#L6

At best, that line of code looks outdated (it doesn't account for "alpha"). At worst, it seems to be preventing a legitimate upgrade path (from snapshot or beta to GA). I'm inclined to remove this line but I don't know enough about the rationale for this line and I can't infer much from the commit that introduced it: 1d7749f. Thoughts on how to proceed here? /cc @simianhacker as he was the original committer for this line.

@epixa
Copy link
Contributor

epixa commented Jun 1, 2016

This is interesting...

On one hand, I do want to figure out what this is_upgradeable thing is all about since it's possible we don't need it. On the other hand, returning false from isUpgradeable() is clearly suppose to be a valid circumstance, so it should never cause Kibana to timeout on all requests.

@ycombinator
Copy link
Contributor

ycombinator commented Jun 1, 2016

Alright, I can make it so that the user is dropped into the status page with the Kibana plugin being red because user settings could not be found for the installed version of Kibana. Totally agree that this would be better than just timing out. I will update PR #7333 to do this.

Separately we still need to decide if we want Kibana to create a new doc in .kibana/config when the user goes from a snapshot version to a GA version. That is the root cause of why user settings can't be found for 5.0.0 when upgrading from 5.0.0-snapshot (see @jbudz's explanation here for specifics: #7318 (comment)).

@epixa
Copy link
Contributor

epixa commented Jun 2, 2016

Go for status page error.

@ycombinator
Copy link
Contributor

While fixing this issue, we discovered an edge case that we weren't handling in the .kibana/config doc creation/upgrade path. I've created an issue for it: #7353

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocker bug Fixes for quality problems that affect the customer experience v5.0.0-alpha4 v5.0.0
Projects
None yet
Development

No branches or pull requests

4 participants