bigquery: Big Query Managed Writer flow controller can dead lock when connections error out #9540
Labels
api: bigquery
Issues related to the BigQuery API.
priority: p2
Moderately-important priority. Fix may not be included in next release.
type: bug
Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Background
We have a service that reads a message from a pub sub topic, performs some computations and then uploads data to a Bigquery table using the
managedwriter
Big query package.We are periodically getting
io.EOF
errors from theAppendRows
API which eventually causes the API to dead lock. We have an open Google Cloud support ticket to root cause why we are receiving EOF errors from Bigquery backend.Reproduction
I was able to reproduce the issue in the following PR by simulating the connection always returning EOFs with a small in-flight request limit:
#9539
It is likely that there are other scenarios where the connection handling logic can error and lead to in flight requests not being properly released.
Workaround
The deadlock can be avoided by disabling the in-flight request limit.
The text was updated successfully, but these errors were encountered: