Getting random failures when fetching data

[jakeleventhal]

Environment details

• OS: macOS Catalina 10.15.3 Beta (19D75a)
• Node.js version: 13.7
• npm version: 6.13.6
• @google-cloud/firestore version: 3.4.1

Steps to reproduce

Making collection fetch queries I am getting the following errors interchangeably at random:

Error: 13 INTERNAL: 
   at callErrorFromStatus (/Users/myusername/myproject/node_modules/@grpc/grpc-js/src/call.ts:79:24)
   at Http2CallStream.<anonymous> (/Users/myusername/myproject/node_modules/@grpc/grpc-js/src/call.ts:132:28)
   at Http2CallStream.emit (events.js:205:15)
   at Http2CallStream.EventEmitter.emit (domain.js:471:20)
   at /Users/myusername/myproject/node_modules/@grpc/grpc-js/src/call-stream.ts:183:14
   at processTicksAndRejections (internal/process/task_queues.js:82:9) {
   code: 13,
   details: '',
   metadata: Metadata { internalRepr: Map {}, options: {} }
}
Error: 8 RESOURCE_EXHAUSTED: Bandwidth exhausted
   at callErrorFromStatus (/Users/myusername/myproject/node_modules/@grpc/grpc-js/src/call.ts:79:24)
   at Http2CallStream.<anonymous> (/Users/myusername/myproject/node_modules/@grpc/grpc-js/src/call.ts:132:28)
   at Http2CallStream.emit (events.js:205:15)
   at Http2CallStream.EventEmitter.emit (domain.js:471:20)
   at /Users/myusername/myproject/node_modules/@grpc/grpc-js/src/call-stream.ts:183:14
   at processTicksAndRejections (internal/process/task_queues.js:82:9) {
   code: 8,
   details: 'Bandwidth exhausted',
   metadata: Metadata { internalRepr: Map {}, options: {} }
}

[schmidt-sebastian]

@jakeleventhal The two failures are likely unrelated:

• INTERNAL should be retried already unless it happens after we have received some responses from the backend. If you have a small log snippet, I could take a look.

• RESOURCE_EXHAUSTED looks like a bug in Node that should have already been fixed in 13.3 (see Bandwidth Exhausted while doing Firestore calls through NodeJS Server deployed on Google App Engine. #765). If this occurs consistently for you, we have to re-open the investigation.

[jakeleventhal]

I am still getting these errors. This is all running in GCP so none of those errors should be appearing

[schmidt-sebastian]

Thank you for sharing. This data will help us prioritize our internal work items.

[crwilcox]

@schmidt-sebastian is there enough info here for you to dig into the issue with INTERNAL?

[schmidt-sebastian]

Unfortunately, not. Unless these errors are reproducible from the SDK, we need recent timestamps, which will allow us to triage these issues on the backend. Unfortunately, we have run into cases where even that is not successful.

[schmidt-sebastian]

While we acknowledge that this is a real issue, we don't have enough information to triage this further. Since this is not actionable at this time, I am going to close it. We can look again at this in the future.