[PeerDAS] Parallelize data column sampling

[0x00101010]

What type of PR is this?
Feature

What does this PR do? Why is it needed?

• Parallelize data column sampling
• Add tests

[0x00101010]

This PR is rebased on #14103, will wait for it to be merged first.

[0x00101010]

Thanks @nalepae for the review, it's still in draft (definitely have plans to address the things you commented). Will let you know once ready

[nalepae]

We just merged #14109 into peerDAS.

I guess the global idea now is to modify the for loop in sampleDataColumnsFromPeers by the job done in this PR.

@0x00101010 do you have a target date for this PR?
If you don't want / don't have time any more to work on it I can take care it, no pb.

[0x00101010]

We just merged #14109 into peerDAS.

I guess the global idea now is to modify the for loop in sampleDataColumnsFromPeers by the job done in this PR.

@0x00101010 do you have a target date for this PR? If you don't want / don't have time any more to work on it I can take care it, no pb.

I'll definitely finish this next week. The major blocker right now is the behavior of sample selection, I saw that there's discussions in ethereum/consensus-specs#3782 that there's various in flight changes discussed regarding column sampling.

What's your take on that? Should we implement based on that? Or what would be the idea behavior regarding (at least for this iteration of implementation):

1. sample selection (seems like the consensus is totally random selection)
2. failure handling? (IncrementalDAS? Saw that you're implementing that in a separate PR iirc)
   a. in the spec, it allows to set higher sample number to allow for errors (missing sample, no response, etc), what's prysm's stance on this?
   b. I need to do more calculations on the probability of missing columns (even with error margin) to see if it is feasible to do totally random selection)

[nalepae]

failure handling? (IncrementalDAS? Saw that you're implementing that in a separate PR iirc)

Yes, this is precisely the PR I was referring to in my previous comment.

This new PR should retain the incrementalDAS/lossyDAS approach.

Main changes to bring:

1. Query peers in parallel rather than sequentially,
2. Implement A node SHOULD maintain a diverse set of peers for each column and each slot by verifying responsiveness to sample queries.
3. Implement If more than one candidate peer is found for a given column, a node SHOULD randomize its peer selection to distribute sample query load in the network. and If possible, it is also recommended to avoid requesting many columns from the same peer in order to avoid relying on and exposing the sample selection to a single peer.

2 and 3 come from ethereum/consensus-specs#3782

[0x00101010]

failure handling? (IncrementalDAS? Saw that you're implementing that in a separate PR iirc)

Yes, this is precisely the PR I was referring to in my previous comment.

This new PR should retain the incrementalDAS/lossyDAS approach.

Main changes to bring:

1. Query peers in parallel rather than sequentially,
2. Implement A node SHOULD maintain a diverse set of peers for each column and each slot by verifying responsiveness to sample queries.
3. Implement If more than one candidate peer is found for a given column, a node SHOULD randomize its peer selection to distribute sample query load in the network. and If possible, it is also recommended to avoid requesting many columns from the same peer in order to avoid relying on and exposing the sample selection to a single peer.

2 and 3 come from ethereum/consensus-specs#3782

Cool, make sense, I'll get on that, and get a PR out asap

[nalepae]

Thanks!