Expand the set of retryable exceptions in JdkHttpSender

[jack-berg]

@mikelaspina has been running a modified version of JdkHttpSender in production and has found some sharp edges with respect to which exceptions are considered retryable.

This PR expands the set to include all IOExceptions which are not instances of SSLException. Based off his experience running this in prod, all the IOExceptions we ran into qualify as retryable. They're also difficult to identify by their message so this PR opts out of specific exceptions instead of the existing behavior of opting in.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Files	Coverage Δ
...ry/exporter/sender/jdk/internal/JdkHttpSender.java	89.38% <100.00%> (ø)

... and 2 files with indirect coverage changes

📢 Thoughts on this report? Let us know!.

[mikelaspina]

lgtm

[laurit]

should

opentelemetry-java/exporters/sender/okhttp/src/main/java/io/opentelemetry/exporter/sender/okhttp/internal/RetryInterceptor.java

Lines 102 to 110 in 1ecc919

	static boolean isRetryableException(IOException e) {
	if (!(e instanceof SocketTimeoutException)) {
	return false;
	}
	String message = e.getMessage();
	// Connect timeouts can produce SocketTimeoutExceptions with no message, or with "connect timed
	// out"
	return message == null || message.toLowerCase(Locale.ROOT).contains("connect timed out");
	}

be also modified?

[jack-berg]

I'm not sure the behavior of the JDK HttpClient is applicable to OkHttp client. I think in principle we should retry in all the same scenarios for both the JDK HttpClient and OkHttp, but not sure how each failure mode manifests as exceptions in OkHttp.

[trask]

is there any documented preference for err'ing on the side of at-most-once or at-least-once delivery?

[jack-berg]

Yes there's this language:

In edge cases (e.g. on reconnections, network interruptions, etc) the client has no way of knowing if recently sent data was delivered if no acknowledgement was received yet. The client will typically choose to re-send such data to guarantee delivery, which may result in duplicate data on the server side. This is a deliberate choice and is considered to be the right tradeoff for telemetry data.