Retry decorator should support generator functions

[daniel-sanche]

Is your feature request related to a problem? Please describe.

Server streaming libraries return an iterable that can asynchronously yield data from the backend over time. Some of our libraries need to provide a wrapper around the raw stream to do some local processing, before passing the data to the user.

It would be useful to wrap this whole pipeline in a retry decorator, so that if the stream breaks mid-way through, we can recover and continue yielding data through our generator as if nothing happened.

Unfortunately, the current implementation returns the result of the target function directly, so generators will not yield values and exceptions through the retry block

Describe the solution you'd like

Add some code to detect when a target function is a generator. If so, Replace:

python-api-core/google/api_core/retry.py

Line 191 in b8e75cf

return target()

with:

yield from target()

Some extra work will also be needed to detect generator .close(), and async generators

I plan on putting together a PR for this shortly

[daniel-sanche]

I have a solution staged in a branch, but I plan on dogfooding it internally for a while before opening a PR