Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: set render limits on Styler to automatically trim dataframes #41635

Merged
merged 15 commits into from
Jun 4, 2021

Conversation

attack68
Copy link
Contributor

@attack68 attack68 commented May 23, 2021

Adds an option: styler.max.elements with a default value of 2**18 = 262144.

>>> pd.options.styler.max.elements = 2**18
>>> df = pd.DataFrame(np.random.rand(10000,100))
>>> %timeit df.style.render()
4.11 s ± 30.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Even though this is a 'large' dataframe it is reduced to the max elements and still renders in reasonable time (4s plus about 5s for a browser to render the date).

If the number of data elements exceeds the max then the right side, or bottom, or both of the DataFrame will be trimmed (starting with the largest axis and recursively reducing), leaving '...' filler indicators.

All reduction takes place before index or data looping so it avoids any crashes.

Screen Shot 2021-05-23 at 20 44 15

@simonjayhawkins simonjayhawkins added Bug Enhancement Styler conditional formatting using DataFrame.style labels May 24, 2021
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not but on yet another option as we already have display.max_columns but i suppose ok

doc/source/user_guide/options.rst Outdated Show resolved Hide resolved
@jreback jreback added this to the 1.3 milestone Jun 3, 2021
@@ -487,6 +487,8 @@ styler.sparse.index True "Sparsify" MultiIndex displ
elements in outer levels within groups).
styler.sparse.columns True "Sparsify" MultiIndex display for columns
in Styler output.
styler.max.elements 262144 Maximum number of datapoints that Styler will render
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

styler.display.max_elements ? i think something like that is more instructive no?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or
styler.render.max_elements (as all things are display based)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed, even maybe styler.max_elements, but changed to yours

@jreback jreback merged commit 874d9a7 into pandas-dev:master Jun 4, 2021
@jreback
Copy link
Contributor

jreback commented Jun 4, 2021

thanks @attack68

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Enhancement Styler conditional formatting using DataFrame.style
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ENH: add dataframe render trimming in Styler to avoid browser crash
3 participants