Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[vt] fix for crash when creating VtArray from large python buffer #2064

Merged

Conversation

ablev-nv
Copy link
Contributor

Description of Change(s)

The third parameter of std::accumulate will determine the return type for std::accumulate. If the type is not explicit, then the compiler will infer it. More info here:

https://en.cppreference.com/w/cpp/algorithm/accumulate

The Vt_ArrayFromBuffer helper uses std::accumulate to determine the number of items to be created in the VtArray. Because the third parameter given here is just 1, the compiler assumes this is an int and uses that for the return type from std::accumulate. This is fine for most sane uses of VtArray that are initialized from a Python buffer with less than 2 billion-ish items. But if your Python buffer has more than maxint items, then the int return type overflows, and the process usually crashes.

This change makes the third parameter explicitly Py_ssize_t, which is the type of the other inputs to std::accumulate and was likely the intended output type all along.

Fixes Issue(s)

Crash or incorrect behavior when creating a VtArray from a python buffer with more than maxint elements.

  • I have submitted a signed Contributor License Agreement

…from python buffers with more items than maxint
@gitamohr
Copy link
Contributor

Nice find and fix, thanks Alan!

@tallytalwar
Copy link
Contributor

Filed as internal issue #USD-7719

@pixar-oss pixar-oss merged commit 1520d4a into PixarAnimationStudios:dev Dec 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants