Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Constraint selection can be unreasonably slow for time coords #3609

Closed
pp-mo opened this issue Dec 19, 2019 · 3 comments
Closed

Constraint selection can be unreasonably slow for time coords #3609

pp-mo opened this issue Dec 19, 2019 · 3 comments
Labels

Comments

@pp-mo
Copy link
Member

pp-mo commented Dec 19, 2019

Discovered that doing a date selection for a given year + month based on categorised coordinates, with an operation like month_data = cube.extract(Constraint(year=y, month=m)).

It seems much quicker to do

indices = (cube.coord('year').points == y) & (cube.coord('month').points == m)
month_data = cube[indices]

In short, this is because operation is not vectorised (unlike the above index-calculating solution).
This is worse for time coordinates, as a unit.num2date() is applied individually to each time cell (points + bounds), before calling the discriminator function.

This could definitely be fixed, within the existing API, at least for the non-user-function coordinate selection styles ("equals-point-value" and "in-list-of-values").
A useful addition would be an extension to the extract API to allow user functions to also be called with vector info.
E.G. either a recognised property visible on the function, or a keyword like vectorise_functions=True ?

TODO:
add a simple testcase demonstrating the performance hit

@github-actions
Copy link
Contributor

In order to maintain a backlog of relevant issues, we automatically label them as stale after 500 days of inactivity.

If this issue is still important to you, then please comment on this issue and the stale label will be removed.

Otherwise this issue will be automatically closed in 28 days time.

@github-actions github-actions bot added the Stale A stale issue/pull-request label Aug 11, 2021
@github-actions
Copy link
Contributor

github-actions bot commented Sep 9, 2021

This stale issue has been automatically closed due to a lack of community activity.

If you still care about this issue, then please either:

  • Re-open this issue, if you have sufficient permissions, or
  • Add a comment pinging @SciTools/iris-devs who will re-open on your behalf.

@rcomer
Copy link
Member

rcomer commented Sep 29, 2022

This was done at #4969 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants