refactor: Make SystemTimeTools::shift() async

[iequidoo]

It's a more stable interface, if shift() suddenly will need to wait for smth like finalisation of
parallel running critical sections assuming there are no time jumps, we don't need to fix all its
usages.

[link2xt]

Using tokio Mutex instead of sync Mutex is probably unnecessary?
https://docs.rs/tokio/latest/tokio/sync/struct.Mutex.html#which-kind-of-mutex-should-you-use

[iequidoo]

Using tokio Mutex instead of sync Mutex is probably unnecessary? https://docs.rs/tokio/latest/tokio/sync/struct.Mutex.html#which-kind-of-mutex-should-you-use

Probably you're talking about RwLock-s. Just forgot to change it back after my experiments when it appeared that the problem can't be solved with the only Tokio's RwLock. Done now, added a comment on that also.

[iequidoo]

And also replaced Tokio's Mutex with RwLock so that parallel critical sections using SystemTimeTools::lock() are possible.

[Hocuri]

The fact that cargo test runs multiple tests in the same process has caused us so many problems (also #2901, and I think that there was some other problem that I can't remember right now) that I'm wondering whether it would be better to say that everyone needs to run tests via cargo-nextest. Which would have the additional advantage of running the tests slightly faster (for faster CI response time).

OTOH, there would probably a lot of things to be taken care of, like installing nextest in the CI, running doc tests with cargo test --doc, making sure that tests in all workspaces are run, and making nextest work with https://github.com/dignifiedquire/cross which we are using apparently (TIL by reading scripts/run-rust-test.sh).

What do you think?

[link2xt]

Alternative solution could be to store separate shift for each test by storing a map from tokio runtime ID (https://docs.rs/tokio/1.37.0/tokio/runtime/struct.Handle.html#method.id) to time shift. Then tests have independent time shift even if they run concurrently and there is no need to take care of "locking time" in the tests. Using Id unfortunately requires enabling tokio_unstable feature, but we have a Cargo.lock file, this will not accidentally break.

Switching CI to cargo nextest probably makes sense, iroh is using it: n0-computer/iroh#1867
But I am not sure about breaking compatibility with cargo test now if this issue can be fixed.

[link2xt]

and making nextest work with https://github.com/dignifiedquire/cross which we are using apparently (TIL by reading scripts/run-rust-test.sh).

We don't run cross anywhere in CI and don't use it for cross-compilation, it could be dropped everywhere.

[link2xt]

For switching to nextest I opened #5457

[iequidoo]

Removed SystemTimeTools::lock() as it's useless currently, but left shift() async just in case

[iequidoo]

Let's close this. Can be reopened if will be really needed