Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat prevent ate memory from being swapped out #3

Conversation

Tomilla
Copy link
Contributor

@Tomilla Tomilla commented Jul 6, 2024

Hi Shawn:

New Feature:

  • Memory read/write periodically, prevent memory from being swapped out
for i := range m.buffer {
	// XOR with 0 keeps the value unchanged but refresh the last modified time of memory
	m.buffer[i] ^= 0
}

start a goroutine to refresh(XOR with 0) the last modified time of every bytes of allocated memory. (a.k.a ActiveMemoryManager)

Performance Improved:

  • Split user-request large memory to multiple small-size chunks(128MB by default)

Because available memory is not equal to contiguously allocatable memory due to memory fragmentation, So directly call make([]byte, m.size) with xxxGB may fail.

	// split request memory to multiple small-size chunks
	const unitChunk = chunkSizeMemoryWokerEachAllocate
	nChunks := m.size / unitChunk
	remain := m.size % unitChunk
	bufSizes := []uint64{}
	for i := uint64(0); i < nChunks; i++ {
		bufSizes = append(bufSizes, unitChunk)
	}
	if remain > 0 {
		bufSizes = append(bufSizes, remain)
	}

	fmt.Printf("Eating %-12sStart\n", "memory...")
	// Because free memory not equal to contiguous free memory, `make([]byte, m.size)` may fail
	// So we change the direct allocation to "divide and conquer", each time we only allocate small chunk of memory.
	m.buffers = make([][]byte, len(bufSizes))
	for i := range m.buffers {
		select {
		case <-ctx.Done():
			m.buffers = nil
			return errors.New("cancel allocation")
		default:
			//
		}
		curSize := bufSizes[i]
		curFreeSize = unimem.FreeMemory()
		if curSize > curFreeSize {
			m.buffers = nil
			return fmt.Errorf("free memory not enough: %d > %d", curSize, curFreeSize)
		}

		buffer := make([]byte, curSize)
		for i := range buffer {
			buffer[i] = byte(i % 256)
		}
		m.buffers[i] = buffer
	}

	fmt.Printf("Ate %d bytes memory\n", m.size)

Thanks for your review.

@shawn-bluce
Copy link
Owner

兄弟,真的太强了 行动力无敌 👍

@shawn-bluce shawn-bluce merged commit 7a65a2d into shawn-bluce:master Jul 8, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants