Test fixes #5415
Test fixes #5415
Conversation
| var TestTimeoutShort = 1 * time.Minute | ||
| var TestTimeoutMedium = 5 * time.Minute | ||
| var TestTimeoutLong = 15 * time.Minute | ||
| var TestTimeoutShort = 5 * time.Minute |
There was a problem hiding this comment.
I think these values are too high. If we need to change them specifically for certain tests, then, we should change the test and include the specific timeout instead.
There was a problem hiding this comment.
According to my empirical tests these are the values that minimize test runs failing due to small perturbations in load or network speed. Honestly I don't see may cons in bumping them, the only one is that actually failing tests will take longer. But that is a condition that should happens rarely since most of the runs should be of passing tests.
There was a problem hiding this comment.
That's exactly the problem. In case of an unstable change we may introduce, the time to complete a test would be at least duplicated, having to wait for a feedback too much time. Imho, the goal should be to have faster timeouts. If some test is not stable, then, I don't think the culprit is the timeout. And if it's the timeout, likely it's in the specific flaky test.
There was a problem hiding this comment.
I see both of you having a point here. IMO if the increased timeout really fixes the flaky tests I tend to accept the fact that we need to wait some more time when a test really is broken by the changes in a PR. The feedback loop for a successful test is over 60 minutes anyways and having a flaky test can be really exhausting.
valdar
added
the
trigger native test
|
I think the native tests tend to not work in GH actions, do they? @valdar Maybe you can run the native tests on your machine and give feedback here when they succeeded? |
…rly copy kits in each scenario
… forwarding for debug cmd is now handled properly.
Some fixes mostly to prevent flaky tests run. It is part of the work done in #5119