Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Time series and rprof data gathering is slow #36

Open
amorison opened this issue Mar 23, 2018 · 0 comments
Open

Time series and rprof data gathering is slow #36

amorison opened this issue Mar 23, 2018 · 0 comments
Assignees
Milestone

Comments

@amorison
Copy link
Member

From @RobertoAgrusta

When working with a large number of profiles and time series, gathering these data takes time and memory (even when using the HDF5 output format).

Improving the parsing time and memory usage when requesting the entire data via the sdat.tseries and sdat.rprof accessors will be difficult since we already use the fast pandas and h5py parsers.

However, a possible workaround is to implement partial parsing of these data when using the steps[n].rprof and steps[n].timeinfo accessors.

@amorison amorison self-assigned this Mar 23, 2018
@amorison amorison added this to the v1.0 milestone Apr 11, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant