Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Note to Validators #520

Closed
faddat opened this issue Oct 11, 2021 · 7 comments
Closed

Note to Validators #520

faddat opened this issue Oct 11, 2021 · 7 comments

Comments

@faddat
Copy link
Member

faddat commented Oct 11, 2021

Hi,

I'm operating a relayer. If you are operating a validator, it is very important that you:

systemctl enable systemd-timesyncd
systemctl start systemd-timesyncd

please.

If you do not, there are time drift errors, and well, the Hermes default of five seconds is really pretty generous. We should be less than five seconds apart.

@faddat
Copy link
Member Author

faddat commented Oct 12, 2021

Oct 12 09:04:56 archive hermes[741991]:    0: GRPC call return error status status: InvalidArgument, message: "failed to execute message; message index: 0: cannot update client with ID 07-tendermint-1457: failed to verify header: invalid header: new header has a time from the future 2021-10-12 07:04:50.046283195 +0000 UTC (now: 2021-10-12 07:04:43.209109453 +0000 UTC; max clock drift: 5s): invalid request", details: [], metadata: MetadataMap { headers: {"content-type": "application/grpc"} }

@ValarDragon
Copy link
Member

Can we raise the max clock drift allowed? Seems unsafe to make the max clock drift under a block time, and you can get block times of 10s with 1-2 round increments

@faddat
Copy link
Member Author

faddat commented Oct 14, 2021

I think that we can raise it to 7s...... maybe?

This is a judgement call which may require prayer / meditation.

Seems like one could, but also seems like this could make IBC overall more risky.

When looking into shared security stuff, I've been imagining a faster parent chain, slower child chain.

These errors, overall, could be taken as declaring "lower block time gonna do better". Maybe. Except when ibcing with a slower chain to whom you may seem to be in the future. Time is harsh, code-wise.

Only way to test this might be to build some interconnected network of chains-- and then grow it.... 🤔

@ValarDragon
Copy link
Member

Per that ibc-rs issue, my reading is that increasing this time moderately has ~no risks / downsides. Its mostly about tolerating clock drift on smaller scales. (Or at least thats my reading, I am admittedly horribly under-informed here)

@faddat
Copy link
Member Author

faddat commented Oct 16, 2021

I am admittedly horribly under-informed here

I feel the same. I suppose that we could try it for the next release.

@ValarDragon
Copy link
Member

Whats the sitaution like now? I think we're going to be more able to get this resolved via pinned announcements on the discord/telegram. We can coordinate if you'd like to make a discord announcement

@faddat
Copy link
Member Author

faddat commented Nov 17, 2021

This is still happening, but less often, and may have been related to fee calculation in hermes..... but the same thing would happen with the Go relayer, too.

I'm going to close this, and shoot out a missive in discord + survey, to see if validators are using ntp or systemd-timesyncd or similar.

Maybe this was improved in ibc-go 2.0?

@faddat faddat closed this as completed Nov 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

No branches or pull requests

2 participants