Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TODO: convert from ndarray.take to internal takes, remove platform int <-> int64 conversions? #3033

Closed
stephenwlin opened this issue Mar 13, 2013 · 2 comments
Labels
Enhancement Internals Related to non-user accessible pandas implementation
Milestone

Comments

@stephenwlin
Copy link
Contributor

just throwing this out there, spliting off from discussion at #2892... right now index arrays are constantly being converted back and forth between platform int and int64 on 32-bit platforms, since ndarray.take requires the former but the cython routines use the latter

this can probably be removed if all ndarray.take usages can be converted to use cython routines, simplifying things considerably and probably improving performance (due to avoiding the conversions, and also since ndarray.take is generic and isn't specialized to handle 1-d and 2-d arrays or particular types efficiently)

the only caveats are that take_nd doesn't allow normal negative indices and uses promotion/fill behavior by default, the latter can be turned off with allow_fill=False but that might get pretty redundant if it's used everywhere; probably best to write a helper which does it. as for the former, there might not be any more cases of places where normal negative indicies are required anymore after #3027...not really sure

@wesm
Copy link
Member

wesm commented Mar 15, 2013

I'm strongly +1 on getting rid of all the platform int conversions. Fused types (fuse at minimum int32_t, int64_t) can alleviate some of the hurting. Was the bane of my existence for a long period of time

@jreback
Copy link
Contributor

jreback commented Sep 21, 2013

@stephenwlin any interest still in this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement Internals Related to non-user accessible pandas implementation
Projects
None yet
Development

No branches or pull requests

3 participants