Skip to content

Commit

Permalink
pythongh-117755: Fix mimalloc for huge allocation on s390x (python#11…
Browse files Browse the repository at this point in the history
…7809)

Fix mimalloc allocator for huge memory allocation (around
8,589,934,592 GiB) on s390x.

Abort allocation early in mimalloc if the number of slices doesn't
fit into uint32_t, to prevent a integer overflow (cast 64-bit
size_t to uint32_t).
  • Loading branch information
vstinner authored Apr 16, 2024
1 parent e05d202 commit 3fe03cc
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Fix mimalloc allocator for huge memory allocation (around 8,589,934,592 GiB) on
s390x. Patch by Victor Stinner.
6 changes: 6 additions & 0 deletions Objects/mimalloc/segment.c
Original file line number Diff line number Diff line change
Expand Up @@ -814,6 +814,9 @@ static mi_segment_t* mi_segment_os_alloc( size_t required, size_t page_alignment
const size_t extra = align_offset - info_size;
// recalculate due to potential guard pages
*psegment_slices = mi_segment_calculate_slices(required + extra, ppre_size, pinfo_slices);

// mi_page_t.slice_count type is uint32_t
if (*psegment_slices > (size_t)UINT32_MAX) return NULL;
}

const size_t segment_size = (*psegment_slices) * MI_SEGMENT_SLICE_SIZE;
Expand Down Expand Up @@ -865,6 +868,9 @@ static mi_segment_t* mi_segment_alloc(size_t required, size_t page_alignment, mi
size_t pre_size;
size_t segment_slices = mi_segment_calculate_slices(required, &pre_size, &info_slices);

// mi_page_t.slice_count type is uint32_t
if (segment_slices > (size_t)UINT32_MAX) return NULL;

// Commit eagerly only if not the first N lazy segments (to reduce impact of many threads that allocate just a little)
const bool eager_delay = (// !_mi_os_has_overcommit() && // never delay on overcommit systems
_mi_current_thread_count() > 1 && // do not delay for the first N threads
Expand Down

0 comments on commit 3fe03cc

Please sign in to comment.