Improve multicore SDR

[dignifiedquire]

Based on #1477 this updates the parent selection logic in multicore sdr.

It does not implement the bitmask update, as that is not thread safe, and only the thread that fills the buffer is allowed to set this value. This I missed in my earlier review.

[qy3u]

It seems that

rust-fil-proofs/storage-proofs-porep/src/stacked/vanilla/create_label/multi.rs

Lines 173 to 176 in 9cc6444

	// Don't overrun the buffer
	while cur_node > (parents_cache.get_consumer() + lookahead - 1) {
	thread::sleep(Duration::from_micros(10));
	}

makes that update the bitmask before increase consumer in the hash loop thread safe.
Otherwise, the current fill& getimplemention should not be thread safe too? 🤔

[dignifiedquire]

The original code was

*base_parent_missing = 0x20; // Mark base parent 5 as missing

instead of

base_parent_missing.set(5);

which only sets the bit, without clearing. Which resulted in the bug you discovered.

[dignifiedquire]

The last change speeds up replication from 12min to 11min on my machine.

[porcuquine]

@dignifiedquire and I went over this, and I believe it now clears the bitmask correctly (and in correct place). This is consistent with the observation that original code (no clearing) was slowest, but first attempted fix was much faster but failed tests. Given that we also have an understanding of the original transcription error that led to the original code, I'm pretty confident in this iteration.

Nevertheless, especially since we've been bitten once, we should make sure this code is tested at least as much as the first attempted fix was. Remember, the discovered test failures were non-deterministic and not expected to always manifest. We need to test this enough times to have reasonable expectation that any failures introduced will have showed up. If we see any such failures, we need to take them seriously (as we did with the previous).

Even though I'm more confident in this than I was in the previous, I'm blocking merging here until I hear that we have successfully tested this (as thoroughly as we eventually did the previous). The reason I am requesting changes rather than approving with 'conditions' is that the latter approach didn't prevent the code from being merged last time. So this is a slight process adjustment based on what we learned in the last round: we need a bit of extra diligence here.

[cryptonemo]

Ran the small lifecycle tests looped overnight without issue. This is where the failure showed up both on CI and on my machine in the past.

[dignifiedquire]

Rebased into clear commits, the different changes.

[cryptonemo]

Did you want me to re-test here, or are we good to go?

[porcuquine]

Based on @cryptonemo's testing — which he believes is sufficient to have caught the previous error but uncovered no problems here — I am approving this now.