Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Gossamer sometimes don't start syncing when starting #3622

Closed
EclesioMeloJunior opened this issue Dec 4, 2023 · 1 comment · Fixed by #3695
Closed

Gossamer sometimes don't start syncing when starting #3622

EclesioMeloJunior opened this issue Dec 4, 2023 · 1 comment · Fixed by #3695
Assignees
@EclesioMeloJunior
Labels
P-critical this must be fixed immediately or contributors or users will be severely impacted. S-sync-westend related to particular network syncing.

Comments

@EclesioMeloJunior
Copy link
Member

Issue summary

Gossamer is failing to connect to bootnodes at startup, facing errors like:

failed to dial: failed to dial 12D3KooWAq2A7UNFS6725XFatD5QW7iYBezTLdAUx1SmRkxN79Ne: no good addresses pkg=network

failed to open connection for peer 12D3KooWByVpK92hMi9CzTjyFg9cPHDU5ariTM3EPMq9vdh5S5Po: failed to dial: context deadline exceeded pkg=network

[/ip4/34.141.212.103/tcp/30334/ws] dial tcp 34.141.212.103:30334: i/o timeout pkg=network

[/ip4/34.141.212.103/tcp/30333] dial tcp4 34.141.212.103:30333: connect: connection refused

2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWGi1tCpKXLMYED9y28QXLnwgD4neYb1Arqq4QpeV1Sv3K: failed to dial: failed to dial 12D3KooWGi1tCpKXLMYED9y28QXLnwgD4neYb1Arqq4QpeV1Sv3K: all dials failed

The following log stream shows the errors in ordered by time

December 04, 2023 at 08:28 (UTC-4:00) | 2023-12-04T12:28:51Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
-- | -- | -- | --
December 04, 2023 at 08:28 (UTC-4:00) | 2023-12-04T12:28:21Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | 2023-12-04T12:27:51Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | 2023-12-04T12:27:21Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | 2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWAq2A7UNFS6725XFatD5QW7iYBezTLdAUx1SmRkxN79Ne: failed to dial: failed to dial 12D3KooWAq2A7UNFS6725XFatD5QW7iYBezTLdAUx1SmRkxN79Ne: no good addresses pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | 2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWNg8iUqhux7X7voNU9Nty5pzehrFJwkQwg1CJnqN3CTzE: failed to dial: failed to dial 12D3KooWNg8iUqhux7X7voNU9Nty5pzehrFJwkQwg1CJnqN3CTzE: no good addresses pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | 2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWGi1tCpKXLMYED9y28QXLnwgD4neYb1Arqq4QpeV1Sv3K: failed to dial: failed to dial 12D3KooWGi1tCpKXLMYED9y28QXLnwgD4neYb1Arqq4QpeV1Sv3K: all dials failed | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | * [/ip4/34.141.212.103/tcp/30333] dial tcp4 34.141.212.103:30333: connect: connection refused | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | * [/ip4/34.141.212.103/tcp/30334/ws] dial tcp 34.141.212.103:30334: i/o timeout pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | 2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWByVpK92hMi9CzTjyFg9cPHDU5ariTM3EPMq9vdh5S5Po: failed to dial: context deadline exceeded pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00) | 2023-12-04T12:27:00Z WARN failed to open connection for peer 12D3KooWPVPzs42GvRBShdUMtFsk4SvnByrSdWqb6aeAAHvLMSLS: failed to dial: context deadline exceeded pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:26 (UTC-4:00) | 2023-12-04T12:26:55Z WARN failed to open connection for peer 12D3KooWKer94o1REDPtAhjtYR4SdLehnSrN8PEhBnZm5NBoCrMC: failed to dial: context deadline exceeded pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:26 (UTC-4:00) | 2023-12-04T12:26:51Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend
December 04, 2023 at 08:26 (UTC-4:00) | 2023-12-04T12:26:50Z INFO started network service with supported protocols /ipfs/ping/1.0.0, /ipfs/id/1.0.0, /ipfs/id/push/1.0.0, /e143f23803ac50e8f6f8e62695d1ce9e4e1d68aa36c1cd2cfd15340213f3423e/grandpa/1, /wnd2/sync/2, /wnd2/light/2, /wnd2/block-announces/1, /wnd2/transactions/1, /wnd2/kad pkg=network | 08f43eb0b5f2465fa89f813d0846918f | gossamer-westend

December 04, 2023 at 08:28 (UTC-4:00)	2023-12-04T12:28:51Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:28 (UTC-4:00)	2023-12-04T12:28:21Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	2023-12-04T12:27:51Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	2023-12-04T12:27:21Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWAq2A7UNFS6725XFatD5QW7iYBezTLdAUx1SmRkxN79Ne: failed to dial: failed to dial 12D3KooWAq2A7UNFS6725XFatD5QW7iYBezTLdAUx1SmRkxN79Ne: no good addresses pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWNg8iUqhux7X7voNU9Nty5pzehrFJwkQwg1CJnqN3CTzE: failed to dial: failed to dial 12D3KooWNg8iUqhux7X7voNU9Nty5pzehrFJwkQwg1CJnqN3CTzE: no good addresses pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWGi1tCpKXLMYED9y28QXLnwgD4neYb1Arqq4QpeV1Sv3K: failed to dial: failed to dial 12D3KooWGi1tCpKXLMYED9y28QXLnwgD4neYb1Arqq4QpeV1Sv3K: all dials failed	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	* [/ip4/34.141.212.103/tcp/30333] dial tcp4 34.141.212.103:30333: connect: connection refused	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	* [/ip4/34.141.212.103/tcp/30334/ws] dial tcp 34.141.212.103:30334: i/o timeout pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	2023-12-04T12:27:05Z WARN failed to open connection for peer 12D3KooWByVpK92hMi9CzTjyFg9cPHDU5ariTM3EPMq9vdh5S5Po: failed to dial: context deadline exceeded pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:27 (UTC-4:00)	2023-12-04T12:27:00Z WARN failed to open connection for peer 12D3KooWPVPzs42GvRBShdUMtFsk4SvnByrSdWqb6aeAAHvLMSLS: failed to dial: context deadline exceeded pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:26 (UTC-4:00)	2023-12-04T12:26:55Z WARN failed to open connection for peer 12D3KooWKer94o1REDPtAhjtYR4SdLehnSrN8PEhBnZm5NBoCrMC: failed to dial: context deadline exceeded pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:26 (UTC-4:00)	2023-12-04T12:26:51Z WARN failed to advertise in the DHT: failed to find any peer in table pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend
December 04, 2023 at 08:26 (UTC-4:00)	2023-12-04T12:26:50Z INFO started network service with supported protocols /ipfs/ping/1.0.0, /ipfs/id/1.0.0, /ipfs/id/push/1.0.0, /e143f23803ac50e8f6f8e62695d1ce9e4e1d68aa36c1cd2cfd15340213f3423e/grandpa/1, /wnd2/sync/2, /wnd2/light/2, /wnd2/block-announces/1, /wnd2/transactions/1, /wnd2/kad pkg=network	[08f43eb0b5f2465fa89f813d0846918f](https://us-east-2.console.aws.amazon.com/ecs/v2/clusters/gossamer-stg/services/gossamer-westend-svc/tasks/08f43eb0b5f2465fa89f813d0846918f?region=us-east-2)	gossamer-westend

Other information and links
@EclesioMeloJunior EclesioMeloJunior self-assigned this Jan 12, 2024
@P1sar P1sar removed the Sync label Jan 15, 2024
@EclesioMeloJunior EclesioMeloJunior changed the title Gossamer node cannot connect to bootnodes Gossamer sometimes don't start syncing when starting Jan 15, 2024
@EclesioMeloJunior EclesioMeloJunior added P-critical this must be fixed immediately or contributors or users will be severely impacted. S-sync-westend related to particular network syncing. labels Jan 15, 2024
@EclesioMeloJunior EclesioMeloJunior mentioned this issue Jan 15, 2024
Merged
@q9f
Copy link
Contributor

q9f commented Jan 16, 2024

@P1sar we need a networking or p2p label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@EclesioMeloJunior EclesioMeloJunior

Labels
P-critical this must be fixed immediately or contributors or users will be severely impacted. S-sync-westend related to particular network syncing.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(dot/sync): execute p2p handshake when there is no target ChainSafe/gossamer
3 participants
@P1sar @EclesioMeloJunior @q9f