Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize downsample1d when data is shared between layers #6075

Merged
merged 5 commits into from
Jan 30, 2024

Conversation

philippjfr
Copy link
Member

@philippjfr philippjfr commented Jan 18, 2024

Another optimization for downsample1d when multiple layers share the same data. Since the majority of time is spent slicing the data we can apply an optimization when all elements in an (Nd)Overlay that is being downsampled share the same underlying data and are indexed along the same key dimension(s).

In this case we simply get the first element slice it with the current x_range and then pass it down to each separate self._process call.

Here's the test case I was using:

import holoviews as hv
import numpy as np
import pandas as pd

from holoviews.operation.downsample import downsample1d

hv.extension('bokeh')

N = 1_000_000
index = pd._testing.makeDateIndex(k=N, freq='h', name='date')
df = pd.DataFrame({chr(65+i): np.random.randn(N).cumsum() for i in range(25)}, index=index).reset_index()

overlay = hv.NdOverlay({col: hv.Curve(df, 'date', col) for col in df.columns[1:]})

d = downsample1d(overlay).opts(responsive=True, height=300)
d

bokeh_plot - 2024-01-18T155254 315

i = 2000
size_stream = d.streams[0]

%%timeit -r 10 -n 1
global i
i += 1
size_stream.event(width=i)

Results before:

1.41 s ± 60.8 ms per loop (mean ± std. dev. of 10 runs, 1 loop each)

Results after:

319 ms ± 40.8 ms per loop (mean ± std. dev. of 10 runs, 1 loop each)

Addresses part of #6058

Ping @droumis and @hoxbro

@philippjfr philippjfr added the type: enhancement Minor feature or improvement to an existing feature label Jan 18, 2024
@codecov-commenter
Copy link

codecov-commenter commented Jan 18, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (c800e52) 88.65% compared to head (0297598) 88.54%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6075      +/-   ##
==========================================
- Coverage   88.65%   88.54%   -0.12%     
==========================================
  Files         314      314              
  Lines       65942    65972      +30     
==========================================
- Hits        58462    58414      -48     
- Misses       7480     7558      +78     
Flag Coverage Δ
ui-tests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@hoxbro
Copy link
Member

hoxbro commented Jan 30, 2024

@philippjfr, this should be ready for review; I can't request you review it as you are the original author of the issue.

Copy link
Member Author

@philippjfr philippjfr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't review it, but your changes and the tests look good!

def test_downsample1d_shared_data():
runs = [0]

class mocksample(downsample1d):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably would have used unittest.mock.patch on the Interface but this is fine.

@hoxbro hoxbro merged commit c7b98de into main Jan 30, 2024
12 checks passed
@hoxbro hoxbro deleted the downsample_shared_data branch January 30, 2024 21:19
Copy link

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 23, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
type: enhancement Minor feature or improvement to an existing feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants