Kafka producer should raise an exception when it fails to connect to broker #636
Conversation
junhui096
requested review from
davidheryanto,
khorshuheng,
pradithya,
woop and
zhilingc
as code owners
|
Hi @junhui096. Thanks for your PR. I'm waiting for a gojek member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/retest |
|
/test test-end-to-end-batch |
junhui096
force-pushed
the
exception_handling
branch
from
471ccbb
to
0adc1ef
Compare
junhui096
force-pushed
the
exception_handling
branch
from
9c55971
to
57cb981
Compare
woop
changed the title
| @@ -129,7 +129,12 @@ def flush(self, timeout: Optional[int]): | |||
| Returns: | |||
| int: Number of messages still in queue. | |||
| """ | |||
| return self.producer.flush(timeout=timeout) | |||
| messages = self.producer.flush(timeout=timeout) | |||
There was a problem hiding this comment.
Might be better to fail fast using produce() callbacks rather than wait for the timeout/queue to clear.
junhui096
force-pushed
the
exception_handling
branch
from
56381d4
to
b56747a
Compare
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: junhui096, zhilingc The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/lgtm |
|
/lgtm |
|
/test test-end-to-end-batch |
What this PR does / why we need it:
In an ingestion job from the Python SDK, the producer should throw an exception if it fails to push data to the Kafka broker.
Which issue(s) this PR fixes:
Fixes #626
Does this PR introduce a user-facing change?: