WaitForAssertion does not observe timeout when debugging

[robalexclark]

Hi @egil, sorry I have another question/issue.

It looks like that the time out TimeSpan is not observed when debugging unit tests. For example:

cut.WaitForAssertion(() => Assert.True(cut.FindAll("td").Count > 12), new TimeSpan(1,0,0));

The 1 hour timeout is observed when running tests normally, but when running the test in debug mode a Xunit.Sdk.TrueException: 'Assert.True() Failure is thrown very quickly.

[egil]

No worries Robin.

This is by design. During debugging the default timeout of one second would ruin the experience.

Why do you need such a long timeout?

[robalexclark]

I don't need such a long timeout, I was using an hour to identify that the timeout was not working during debug. The reason that I am using a timeout (10 seconds usually) is because I am testing at the "blazor app level" - i.e. whole application made up of lots of components, some of which raise events in other components to get data etc. I've been successful in using bunit to test smaller components, but at the app level the tests are becoming "flaky" i.e. sometimes pass, sometimes don't because data has not been retrieved or the state of the app is slightly different so the expected html does not match actual.

I think I need to rethink my strategy at this level - maybe just check for individual pieces of html rather than the whole page.

[egil]

Your strategy might not be wrong. You should definitely be able to do more end-2-end testing.

It might be that you are hitting the concurrent bug I am currently chasing in #118.

[robalexclark]

OK I will wait for beta 7. Just a final illustration of the issue I am hitting is that if I wanted to debug the contents of some html table cells (either through breakpoint or console.writeline) I can't because in debug mode the assertation fails whereas in release mode it passes (but I can't then debug). Hope this helps

  cut.WaitForAssertion(() => Assert.True(cut.FindAll("td").Count > 12)); //assertation passes in release mode but not debug mode

           //Assert
            var tableCell = cut.FindAll("td")[0].InnerHtml;
            Debug.WriteLine(tableCell);

[egil]

So when you compiler in release mode and run the test passes, but when you compile in debug mode and run without a debugger attached, it fails?

Or does it only fall when you run with a debugger attached?

[robalexclark]

Apologies, I should clarify - everything is compiled in debug mode, but when I run a test as debug so as to hit breakpoints the assertion fails. When I run normally, the test passes.

I think (can't be sure though) the test is failing when running debug because the assertion does not observe a timeout - this was my original reason for opening this issue :-) When test is run normally the supplied timeout is observed (takes about 2 seconds for data to be retrieved etc before it displays) and so passes.

[egil]

Ok. So it doesn't make sense that it wouldn't pass when there is no time-out, it just means it will keep retrying the assertion for ever until it passes, so if the data is there after two seconds and remain there for ever then the assertion should pass after 2 seconds or two minutes.

Is it because the data is removed again after e.g. 5 seconds, so that you miss the window between 2 and 5 seconds where the assertion can pass?

[rafael-f]

No worries Robin.

This is by design. During debugging the default timeout of one second would ruin the experience.

Why do you need such a long timeout?

I just came across this issue, trying to debug an async call is impossible because wait doesn't work, it fails before we have time to step through the async method.

What would be the recommended way to debug then?

edit: Maybe I misunderstood...

During debugging there's no 1 second timeout? But I put a 1 hour timeout and wait is definitely not waiting 1 hour... same as OP.

[egil]

No worries Robin.

This is by design. During debugging the default timeout of one second would ruin the experience.

Why do you need such a long timeout?

I just came across this issue, trying to debug an async call is impossible because wait doesn't work, it fails before we have time to step through the async method.

What would be the recommended way to debug then?

edit: Maybe I misunderstood...

During debugging there's no 1 second timeout? But I put a 1 hour timeout and wait is definitely not waiting 1 hour... same as OP.

During debugging timeout is changed to "infinite".