Prevent too many ProcessBacklog() from being scheduled

[devbv]

This PR prevents the situation that a consumer makes the number of backlog 1->0 infinitely and many producers schedule new ProcessBacklog() after they make the number of backlog 0->1.
The situation could lead a starvation of thread pool and a lot of timeout when we use transaction heavily. (#1572)

[devbv]

@mgravell Hello, Could you take a look for this MR or issue #1572 ?

[arsnyder16]

@mgravell I am also seeing these errors in my logs. They are a semi related to the issue solved with #1585 in that in these timeout scenarios seems to cascade and build upon each other, and the library struggles to recover.

I would also be interested in seeing your opinion on getting this change in as well

[devbv]

Additional info for better understanding. This is stacktrace of threads when the timeout is occurred. You can see 9 of 10 worker threads are stopped at MutexSlim.TakeWithTimeout called by PhysicalBridge.ProcessBacklog()
This is reproduced from the sample code on #1572

[NickCraver]

Merging main in here for build check changes

[mgravell]

This is a great find, and could explain a lot, but I'm not sure that the approach is the best way of solving it - I'm playing with an alternative strategy in #1612 - but: even if I merge that, I'll still give credit for the research here; much appreciated!

[mgravell]

Have merged #1612, but credited in release notes:

• fix backlog contention issue (Async backlog #1612, see also Prevent too many ProcessBacklog() from being scheduled #1574 via devbv)