[vt] fix for crash when creating VtArray from large python buffer #2064
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of Change(s)
The third parameter of
std::accumulate
will determine the return type forstd::accumulate
. If the type is not explicit, then the compiler will infer it. More info here:https://en.cppreference.com/w/cpp/algorithm/accumulate
The
Vt_ArrayFromBuffer
helper usesstd::accumulate
to determine the number of items to be created in theVtArray
. Because the third parameter given here is just1
, the compiler assumes this is anint
and uses that for the return type fromstd::accumulate
. This is fine for most sane uses ofVtArray
that are initialized from a Python buffer with less than 2 billion-ish items. But if your Python buffer has more than maxint items, then theint
return type overflows, and the process usually crashes.This change makes the third parameter explicitly
Py_ssize_t
, which is the type of the other inputs tostd::accumulate
and was likely the intended output type all along.Fixes Issue(s)
Crash or incorrect behavior when creating a
VtArray
from a python buffer with more than maxint elements.