Add exception callback to handle async errors #318
Conversation
| @@ -41,6 +42,25 @@ public void testSchedulerExceptionHandling() throws InterruptedException, IOExce | |||
| verify(mockInfluxDB, times(2)).write(any(BatchPoints.class)); | |||
| } | |||
|
|
|||
| @Test | |||
There was a problem hiding this comment.
it looks like the tests are not formatted with the same settings as the production code. I'm not sure if that's on purpose or accidental but I decided to use the same formatting as in the main code.
|
Build is failing because of formatting. |
Codecov Report
@@ Coverage Diff @@
## master #318 +/- ##
============================================
- Coverage 78.62% 78.44% -0.18%
+ Complexity 134 133 -1
============================================
Files 11 11
Lines 711 719 +8
Branches 77 77
============================================
+ Hits 559 564 +5
- Misses 109 112 +3
Partials 43 43
Continue to review full report at Codecov.
|
|
Fixed, @majst01 |
|
Addresses #275 |
|
I like your simple approach, if you mind adding some usage information's to README.md i would like to merge this. |
|
@majst01 cool! Now that I'm writing the documentation, I wondering if I should add another overloaded method void enableBatch(int actions, int flushDuration,
TimeUnit flushDurationTimeUnit, Consumer<Throwable> exceptionHandler)What do you think? Is there another way to enable batching? Or maybe I should just add |
|
Yes, would be a good Idea to have a second variation of enableBatch. |
|
which one? I prefer the first one. |
|
related to: #289 |
|
Per my similar changes referenced in #289 which I commented on, my use case requires retrying to persist the failed batch of points so the exception handling method needs to reference the failed points in the batch so they can be re-tried. Also, flip side to that, when the batch succeeds, my use case also requires other parts of the system to be notified so I added a onSuccess event as well as part of the same interface. Overall, our solutions are very similar. |
|
The only thing I'd point out then is that the Consumer functional interface chosen for this change only accepts one input, in this case the Throwable. Should you want to pass more, Consumer may not be the best choice. |
|
@majst01 agree. The formatting is fixed. @kkarski passing the failed points to the callback sounds good, but I think that for your use case (retrying failed writes) it would be better to have that as a library feature. Maybe the user should be able to define a retry policy like "retry five times with exponential backoff increasing time" |
|
@csokol I agree an automated retry would be a good feature and would love to see that. It doesn't solve the issue 100% however which is why I still think it would be good to have a reference to the failed points in the batch in order to attempt some sort of last ditch effort to no loose the data. Alternatively, if the Throwable was a custom type, the points could be referenced in the exception itself rather than as a second parameter. As I said earlier however, I also have an interest in successful batches. |
|
Should we stick with the current approach and try to implement a automatic recovery of failed batch writes later ? |
|
I think the automated recovery is optional but getting a reference to the failed points is necessary. Just my opinion...or at least I'll have to continue using my fork if the official version doesn't support it. |
|
The introduction of a InfluxDBExceptionHandler which implements Consumer and holds a List of failed Batchpoints will do the trick, bonus we will get a fork user back. @csokol would you like to modify your PR this way ? |
|
The |
|
Sounds good, tbh i am still a bit oldschool and not very familiar with all the glories of the function interfaces in java8 :-) If you mind modifying this PR, or create a separate one to see the differences in implementation. |
|
I'll do it in a separate branch to compare it. Looking at the code, I think I won't be able to get only the failed points (I'd have to keep separate entries for udp/http points and keep removing them from a list or something). Is it ok if in this initial implementation I pass the whole batch to the consumer instead of just the ones that failed? |
|
Just noticed that #289 was actually proposing the same thing (the whole batch, not only the failed points) so I'll go for that. |
|
Replaced by a better implementation in #319 |
This enables the client to do something (other than logging) if a problem happens asynchronously. In our use case, we want to send the error to honeybadger.io to be notified of influxdb issues.