Fix flaky MetricExporterConfigurationTest

[laurit]

Resolves #5866
My guess is that configureMetricReader_KnownSpiExportersOnClasspath fails with java.net.BindException: Address already in use because configureReader_PrometheusOnClasspath, which is the only test that also starts PrometheusHttpServer in MetricExporterConfigurationTest had not yet shut down PrometheusHttpServer. Shutting down PrometheusHttpServer

opentelemetry-java/exporters/prometheus/src/main/java/io/opentelemetry/exporter/prometheus/PrometheusHttpServer.java

Line 164 in 5ccc7fc

shutdown().join(10, TimeUnit.SECONDS);

waits for 10s for the server to shut down, running time of configureReader_PrometheusOnClasspath is 10.104s. So it think it is plausible that configureMetricReader_KnownSpiExportersOnClasspath started before PrometheusHttpServer from previous test was shut down. Also there is a 10s wait in shutting down of PrometheusHttpServer

opentelemetry-java/exporters/prometheus/src/main/java/io/opentelemetry/exporter/prometheus/PrometheusHttpServer.java

Lines 144 to 160 in 5ccc7fc

	public CompletableResultCode shutdown() {
	CompletableResultCode result = new CompletableResultCode();
	Thread thread =
	THREAD_FACTORY.newThread(
	() -> {
	try {
	server.stop(10);
	executor.shutdownNow();
	} catch (Throwable t) {
	result.fail();
	return;
	}
	result.succeed();
	});
	thread.start();
	return result;
	}

Alternatively reducing that could make test more stable.

[codecov]

Codecov Report

All modified lines are covered by tests ✅

📢 Thoughts on this report? Let us know!.

[breedx-splk]

How does the config/port=0 change address the timing? Or does that just change how the binding happens to bypass the conflict entirely?

[laurit]

How does the config/port=0 change address the timing? Or does that just change how the binding happens to bypass the conflict entirely?

port 0 means bind to any free port

[breedx-splk]

How does the config/port=0 change address the timing? Or does that just change how the binding happens to bypass the conflict entirely?

port 0 means bind to any free port

Got it. Was just surprised that your discussion didn't actually mention the approach of the fix. 🙃

[jack-berg]

All these tests cleanup their exporters / readers. This change will help make this test more resilient against other tests starting prometheus on the default port and not cleaning up, but best case we identify / fix the test failing to clean up.