You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
How often have you been able to reproduce it?
I tried multiple times, simulating an OOM scenario in the server side, with no luck.
That said, have you tried to run client.close as part of the teardown function and see if it sill panics? If you're still able to reproduce it, I'd suggest to try that, as arguably one could say that the source of the panic is due to the lack of cleanup, although we could explore a mechanism to prevent that from happening at all, of course.
Any news @gm42? I spent some more time trying to reproduce this by reproducing a few more corner cases, as well as intentionally faking the 14 UNAVAILABLE gRPC status code, with no luck.
Would you be able to share how the script looks like, by striping sensitive details, if any, please?
this seems to be happening because there's some "remaining" code running even after calling close(samples)
the AsyncInvoke code specifically looks pretty safe in terms of leaking something like that (as it adequately initializes a promise and registers & uses the callback)
I guessed: aha, it looks like this is probably "reproducible" with Go code that escapes the "main thread" without actually using promises/callbacks adequately (think of any goroutine that calls metrics.PushIfNotDone, with a reference to the channel and a non-cancelled context; which I may happen by mistake, and would probably be easy to force intentionally? 🤔
Thus, I've been taking a look at code in google.golang.org/grpc, referred from that stack trace, looking for any code that could produce this situation, and I haven't spot any so far.
However, I still think this is "doable" (intentionally for instance). So, do you think we should try to prevent that from k6 surface? Or it is just fine to assume that could happen, and treat each case as a bug fix from the extension code (gRPC in this case)?
In other words, if we assume this is caused by a "bug" in the gRPC code; do you think we should prevent that from happening at all in k6 anyway? Or just consider that this should be fixed in gRPC?
cc/ @olegbespalov@mstoykov 🙇🏻
Brief summary
While using
client.asyncInvoke
I managed to trigger this panic:k6 version
0.51.0
OS
Linux
Docker version and image (if applicable)
Docker image tag 0.51.0
Steps to reproduce the problem
To reproduce this problem you need the gRPC endpoint to crash (OOM), which then causes a code 14 gRPC status:
Afterwards the panic may happen, but it's not 100% reproducible. Perhaps a test with
-race
will find the bug?Expected behaviour
No panics when using
client.asyncInvoke
Actual behaviour
A panic happened
The text was updated successfully, but these errors were encountered: