Visualize landing page doesn't handle more than 100 visualizations

[trevan]

Version: Build 8467, Commit SHA 6cb7fec

If you have more than 100 visualizations, the landing page makes it seem like you only have a 100. The pagination control shows 100 and you can only paginate through those 100. If you filter, it will find the additional ones.

To show it, I created over 100 visualizations all with a title of '3' and one with "Not Found".

Checking the network requests sent, the landing page is explicitly limiting the request to 100.

[thomasneirynck]

thx @trevan, this looks like a dupe. closing this in favor of #8044.

[trevan]

@thomasneirynck, that bug is for the management page. This one is for the landing page. The management page doesn't have pagination while the landing page does have pagination. It just is limited to 100 items.

[thomasneirynck]

ok, my bad, reopening, thx!

[ppisljar]

@cjcenizal do you want to take a look at this one or should i ?

[stacey-gammon]

I think we should do something similar to #10827

It should show the real number of hits, and once we get to the final page, show a warning that only the first X are retrieved.

[trevan]

@stacey-gammon, is there any reason not to have it actual paginate?

[stacey-gammon]

That would definitely be the better implementation @trevan, just for sake of time, the above would be an easy first step. If you want to throw together a PR to implement paging for real, that would be awesome. 🥇 😄 Otherwise I don't think it's on any ones short term roadmap.

[trevan]

@stacey-gammon, ok, I'll see if I can give it a shot this week.

[trevan]

@stacey-gammon, it took me a bit longer to look at it but there looks to be a problem with the sorting. With pagination happening in ES, we can no longer use the in-memory sorting from the SortableProperties object. So all sortable columns must be sortable through ES. And in the visualization table, we sort on both title and type. Title is a normal field in ES but type is hidden inside of the visState string and I don't know if there is a way to sort on it.

Any ideas how to sort on type through an ES query?

[trevan]

Oh, it is worse than that. I can't even sort on the title column because it is a "text" field. I get the following error:

Fielddata is disabled on text fields by default. Set fielddata=true on [title] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.

The mapping would either have to be updated with a text version of the title field or ES pagination isn't possible.

[stacey-gammon]

Ah yea, @trevan, we have an issue for that here: #10348 and until it's addressed, I don't see a way around the limitation.

Perhaps we better just stick with the simpler, temporary fix, of making the limitation clear during the pagination, and to list the full number of result hits, not the artificial cap of 100.

[trevan]

@stacey-gammon, the temporary fix has issues with sorting as well. With the discover table, it sorts it in ES and then returns the top 500. But with the listing pages, it only sorts the first 100 items. So you would have to display the message and indicate that sorting is broken as well.

[trevan]

@stacey-gammon, I wonder if another temporary fix would be to increase the 100 limit to something like 10,000. Yes that just pushes the issue down the road but I think the browser should be able to handle 10,000 items and I don't see many people reaching that limit. 100 is pretty easy to reach if you have 10s of users that are creating their own visualizations and dashboards.

[stacey-gammon]

Great point @trevan. 100 does seem pretty limited and arbitrary. 10,000 seems like a big jump though. Maybe 1,000? It all seems arbitrary but I worry about a big jump because 10k may not be a big deal for one page, but I'm not sure where the loaders are being used, and I'm not sure how the saved objects management is loading all the different types of items. Will it hold 10k, or is it going to hold 10k for each dashboard, discover, saved search, visualization? I'm also not sure whether those objects are cached in memory.

I just don't want to jump into a large number without being certain that existing users with limited machines won't be affected.

[trevan]

@stacey-gammon, oh I wasn't thinking about having the saved objects page load 10k. Just the visualization and dashboard landing page. I'm not looking at changing the saved objects page as that #8044. We currently have 330 visualizations so I wanted to go a bit bigger than 1k since we might actually hit that limit.

How about I create a setting that specifies the amount of items loaded in the listing page? Then it can be set to 1k but people can increase or decrease it if needed.

[trevan]

@stacey-gammon, I created a PR that increases the limit and makes it configurable while also adding a warning.

[iamhata]

I still have the same issue under version 5.6.1. How to adjust the limit number?

[thomasneirynck]

hi @iamhata, try savedObjects:listingLimit in the Management > Advanced Settings page.

Thanks,

[trevan]

@iamhata, your issue is #8044

[iamhata]

@ thomasneirynck I saw that default value already is 1000 before posted my issue.

[nemoyin]

what about kibana 5.3 , I can't find savedObjects:listingLimit setting in the Management -> Advanced settings ,