You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I was testing some things with a ~3GB list and noticed that my profile was showing negative memory allocated. I was also sometimes getting overflowError at:
However this seems to be double, where 2048MB makes me think it's int overflowing, not double. Regardless, I can do this at __init__.py:479 & __init__.py:501:
And that "fixes" the problem. And I think you could even do size = 4294967294 + size to increase the max size before you end up wrapping around again. But in the very least I'm thinking size should really be a double, because that's much more reasonable to clamp in terms of how often a size would actually be that large.
I can take a shot doing that but have never touched C before so any help/guidance would be appreciated.
The text was updated successfully, but these errors were encountered:
Hi @MacroPower do you happen to have a minimal example that can reproduce this issue? Also, what OS/arch are you running on (e.g. x86/arm/32-bit/64-bit?). It does look like there is an undesirable overflow here. Here are a few places where I could conceivably see it happening:
When intercepting a Python Calloc call, we multiply size_t(nelem) * size_t(elsize), which could overflow. This looks to be present in Python's built-in tracemalloc as well, so we could try to repro with it to help narrow down the issue.
I was testing some things with a ~3GB list and noticed that my profile was showing negative memory allocated. I was also sometimes getting overflowError at:
mprofile/mprofile/__init__.py
Line 531 in be59539
I took a look at tcmalloc and it explains the problem and just clamps:
mprofile/third_party/google/tcmalloc/sampler.cc
Lines 97 to 100 in be59539
However this seems to be double, where 2048MB makes me think it's int overflowing, not double. Regardless, I can do this at __init__.py:479 & __init__.py:501:
And that "fixes" the problem. And I think you could even do
size = 4294967294 + size
to increase the max size before you end up wrapping around again. But in the very least I'm thinking size should really be a double, because that's much more reasonable to clamp in terms of how often a size would actually be that large.I can take a shot doing that but have never touched C before so any help/guidance would be appreciated.
The text was updated successfully, but these errors were encountered: