Don't retry on selected statuses

[leonidb]

No point to retry on certain statuses, like 401 or 403 for example.
And there is no option to customize on which statuses to retry.

[asieira]

I have submitted a new pull request that addresses this and also scenarios where the underlying curl code can raise errors as well: #429

[patr1ckm]

It's humorous that this issue is #404

[hadley]

I'm not sure what the API should look like here. Maybe a vector of additional status codes that should terminate the retry cycle?

[asieira]

That is basically what I did on the PR I submitted before, with the addition of the optional use of try to catch exceptions raised by the underlying curl code.

Another option that would be more flexible but IMHO less user-friendly would be to request a function parameter that would be applied to the response object and that should return a logical scalar to indicate if the retries should continue.

[hadley]

@asieira you called it "safe statuses" which is confusing (because many aren't actually safe) and the logic you used ((!status_code(resp) %in% safe_statuses && http_error(resp)) is confusing. You also tangled it up with another unrelated issue.

[leonidb]

In practice the desired behavior would depend a lot on service implementation and client's use case. Probably for some use cases might be useful to decide on the retry based on the response body (not saying it's a recommended practice, but it might be the case).

So, despite what I stated originally when I've opened this issue, I think the more flexible option of a function parameter would probably be preferable.

Additionally, it makes sense and is often useful to retry on connection errors, to overcome a flaky connection. Yet for other possible curl exceptions it makes less sense. Not sure it's worth yet additional parameter - a list of exception types to retry on, but maybe worth to hand pick the ones that are useful for retries.

[patr1ckm]

Being able to provide a vector of additional status codes that terminate the retry cycle is all I need.

[asieira]

@leonidb I've seen my fair share of temporary DNS failures to resolve hostnames be raised as exceptions by curl as well, not only connection failures. So that's why I suggested using try to catch and retry on any of them.

[asieira]

@hadley thank you for the feedback, happy to improve on that and re-submit. 🙇

[hadley]

Let's start with a vector of additional status codes (i.e. codes in the 400-500) range to return on. I'm not sure exactly what it should be called but maybe something like terminate_on?

@asieira another attempt would be great. Can you please use tryCatch() instead of try()?

[asieira]

Question - if the last response after all of the retries are finished is still an error condition (as caught with tryCatch), should RETRY return that or should it call stop with it to remain backwards compatible?

My vote goes to raising it again so as to avoid breaking existing code, but wanted to check what you think.

[asieira]

@hadley hopefully this is better: #459