Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

P2P Improve sync behavior #1072

Closed
heifner opened this issue Apr 24, 2023 · 4 comments · Fixed by #1187, #1215, #1225 or #1243
Closed

P2P Improve sync behavior #1072

heifner opened this issue Apr 24, 2023 · 4 comments · Fixed by #1187, #1215, #1225 or #1243
Assignees
@heifner
Labels
actionable enhancement New feature or request 👍 lgtm OCI Work exclusive to OCI team

Comments

@heifner
Copy link
Member

heifner commented Apr 24, 2023 •
edited
Loading

  • Track latency of peers and prioritize syncing from those with lower latency
  • Add configuration of number of peers to sync from (default 3)
    • This would be different than current p2p-peer-address list and max-clients
    • Even though many p2p-peer-address could be specified and connected to, only the top configured number of peers would be actively synced from.
  • Change default sync-fetch-span from 100 to 1000
  • Peer should include block start/end range in notice_message and peers should only attempt to sync from peers that have range of blocks needed. Currently only HEAD info is included. Especially useful for peers with no block log.
  • Do not serve peers until in-sync.
@heifner heifner added enhancement New feature or request actionable and removed triage labels Apr 24, 2023
@heifner
Copy link
Member Author

heifner commented May 3, 2023

eosnetworkfoundation/mandel#792 suggested indicating the block start/end range as part of a handshake_v2. I now think using notice_message would be cleaner.

@heifner heifner mentioned this issue May 3, 2023
Closed
@heifner heifner self-assigned this May 8, 2023
@heifner heifner added the OCI Work exclusive to OCI team label May 8, 2023
heifner added a commit that referenced this issue May 17, 2023
@heifner
GH-1072 Refactor connections set into a connections_manager class
75ee2e2
heifner added a commit that referenced this issue May 17, 2023
@heifner
GH-1072 Use uint32_t for max-clients
31983be
@heifner heifner mentioned this issue May 17, 2023
Merged
@heifner heifner linked a pull request May 17, 2023 that will close this issue
P2P Refactor connections management #1187
Merged
@BenjaminGormanPMP BenjaminGormanPMP moved this from Todo to In Progress in Team Backlog May 17, 2023
heifner added a commit that referenced this issue May 18, 2023
@heifner
Merge branch 'main' into GH-1072-refactor
caf601d
heifner added a commit that referenced this issue May 18, 2023
@heifner
GH-1072 Use better heuristic for determing who to sync from
548692e
heifner added a commit that referenced this issue May 22, 2023
@heifner
GH-1072 Use flat_set and some other misc cleanup
0efab27
heifner added a commit that referenced this issue May 22, 2023
@heifner
GH-1072 Rename _impl methods to _i to distinguish from _impl types.
30d6fd5
heifner added a commit that referenced this issue May 22, 2023
@heifner
GH-1072 Fix indentation
a80f5bc
heifner added a commit that referenced this issue May 22, 2023
@heifner
GH-1072 Add any_of_connection and use it for duplicate implementation
a6b5259
heifner added a commit that referenced this issue May 22, 2023
@heifner
GH-1072 Remove unneeded check
7470c68
heifner added a commit that referenced this issue May 22, 2023
@heifner
GH-1072 Add any_of_block_connections and change for_each_connection a…
6a5bf21 
…nd for_each_block_connection to not short circuit
heifner added a commit that referenced this issue May 22, 2023
@heifner
GH-1072 Fix any_of_block_connections to verify not transaction only c…
01dae4e 
…onnection
heifner added a commit that referenced this issue May 22, 2023
@heifner
Merge remote-tracking branch 'origin/main' into GH-1072-refactor
0af651b
heifner added a commit that referenced this issue May 22, 2023
@heifner
Merge remote-tracking branch 'origin/main' into GH-1072-refactor
44820e9
heifner added a commit that referenced this issue May 22, 2023
@heifner
Merge pull request #1187 from AntelopeIO/GH-1072-refactor
2d06cfb 
P2P Refactor connections management
@github-project-automation github-project-automation bot moved this from In Progress to Done in Team Backlog May 22, 2023
@heifner heifner mentioned this issue Jun 2, 2023
Merged
heifner added a commit that referenced this issue Jun 2, 2023
@heifner
GH-1072 Update log message
ce78703
heifner added a commit that referenced this issue Jun 5, 2023
@heifner
GH-1072 Comment sync_ members
644ded8
heifner added a commit that referenced this issue Jun 5, 2023
@heifner
GH-1072 Simplify logic
8756419
heifner added a commit that referenced this issue Jun 5, 2023
@heifner
Merge pull request #1225 from AntelopeIO/GH-1072-sync-ahead
d04757a 
P2P sync ahead while applying received blocks
@github-project-automation github-project-automation bot moved this from In Progress to Done in Team Backlog Jun 5, 2023
@heifner
Copy link
Member Author

heifner commented Jun 5, 2023

Re-open for a follow on PR to address peer review comment of better calculation of peer latency.

@heifner heifner reopened this Jun 5, 2023
@BenjaminGormanPMP BenjaminGormanPMP moved this from Done to Awaiting Review in Team Backlog Jun 5, 2023
@BenjaminGormanPMP BenjaminGormanPMP moved this from Awaiting Review to In Progress in Team Backlog Jun 5, 2023
heifner added a commit that referenced this issue Jun 6, 2023
@heifner
GH-1072 No reason to post if syncing from peer
c776ac9
heifner added a commit that referenced this issue Jun 6, 2023
@heifner
GH-1072 Use round trip time_message for calculation of latency.
703f8eb
@heifner heifner mentioned this issue Jun 6, 2023
Closed
heifner added a commit that referenced this issue Jun 6, 2023
@heifner
GH-1072 Fix for net_latency_ns not being calculated yet.
1341215
heifner added a commit that referenced this issue Jun 6, 2023
@heifner
GH-1072 Do not print out complete blocklog on failure as it is way to…
cdd05a8 
…o much output for ci/cd.
heifner added a commit that referenced this issue Jun 7, 2023
@heifner
GH-1072 Update comments
668d2a4
heifner added a commit that referenced this issue Jun 7, 2023
@heifner
GH-1072 Test removing some logging
ed326cb
heifner added a commit that referenced this issue Jun 7, 2023
@heifner
GH-1072 Revert check for in sync as we need to always attempt a start…
2fab062 
… sync on a lib notice.
heifner added a commit that referenced this issue Jun 7, 2023
@heifner
GH-1072 No reason to post if syncing from peer
8e45db5
heifner added a commit that referenced this issue Jun 7, 2023
@heifner
GH-1072 Revert: No reason to post if syncing from peer.
adc28f2
heifner added a commit that referenced this issue Jun 7, 2023
@heifner
GH-1072 Revert: No reason to post if syncing from peer.
73644fd
heifner added a commit that referenced this issue Jun 7, 2023
@heifner
Merge branch 'main' into GH-1072-latency
fed0e8a
@heifner heifner mentioned this issue Jun 7, 2023
Merged
heifner added a commit that referenced this issue Jun 12, 2023
@heifner
Merge branch 'main' into GH-1072-send-time-latency
84fa137
@github-project-automation github-project-automation bot moved this from In Progress to Done in Team Backlog Jun 12, 2023
heifner added a commit that referenced this issue Jun 12, 2023
@heifner
Merge pull request #1243 from AntelopeIO/GH-1072-send-time-latency
c2026ce 
P2P use round trip of time_message to calculate latency
This was referenced Jun 12, 2023
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@heifner heifner

Labels
actionable enhancement New feature or request 👍 lgtm OCI Work exclusive to OCI team
Projects
Team Backlog
Archived in project
Milestone
No milestone
3 participants
@bhazzard @heifner @enf-ci-bot