-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GH-149: Improve error messages on write thread failure #150
Conversation
...ector/src/main/java/com/wepay/kafka/connect/bigquery/write/batch/KCBQThreadPoolExecutor.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From what I understand, all threads can throw exceptions but only the first gets logged and that bubbles up and kills the task. Since the exception will be thrown, we do not want to reset the encounteredError
atomic ref?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
We do not reset the reference because the exception may be used multiple times. We throw exceptions encountered from write threads in both |
@ddasarathan thanks for taking a look. Given the LGTM I'll merge but if you are unsatisfied by my explanation of the way we use |
Addresses #149.
The
KCBQThreadPoolExecutor
is modified to only track one write thread exception at a time, which allows us to include a complete stack trace of that exception when failing the task.The "Attempted to reduce batch size below 1." error is rewritten to include more information on a potential root cause and make the attached cause of the exception clearer.
Some cleanup of unnecessary exception wrapping when instantiating various
AbstractConfig
subclasses is also made.