Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sun.net.www.protocol.http.HttpURLConnection.getOutputStream should tr… #6053

Merged
merged 11 commits into from
May 25, 2022
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
/*
* Copyright The OpenTelemetry Authors
* SPDX-License-Identifier: Apache-2.0
*/

package io.opentelemetry.javaagent.instrumentation.httpurlconnection;

import static io.opentelemetry.context.ContextKey.named;

import io.opentelemetry.context.Context;
import io.opentelemetry.context.ContextKey;
import io.opentelemetry.context.ImplicitContextKeyed;
import java.net.HttpURLConnection;

public class GetOutputStreamContext implements ImplicitContextKeyed {
private static final ContextKey<GetOutputStreamContext> KEY =
named("opentelemetry-http-url-connection-get-output-stream");

private volatile boolean outputStreamMethodOfSunConnectionCalled;

private GetOutputStreamContext() {}

@Override
public Context storeInContext(Context context) {
return context.with(KEY, this);
}

public static Context init(Context context) {
if (context.get(KEY) != null) {
return context;
}
return context.with(new GetOutputStreamContext());
}

public static GetOutputStreamContext get(Context context) {
return context.get(KEY);
}

public static void set(
Context context,
Class<? extends HttpURLConnection> connectionClass,
String methodName,
String requestMethod) {
GetOutputStreamContext getOutputStreamContext = context.get(KEY);
String connectionClassName = connectionClass.getName();
if ("sun.net.www.protocol.http.HttpURLConnection".equals(connectionClassName)
&& "getOutputStream".equals(methodName)
&& "POST"
.equals(
requestMethod) // To be sure that getOutputStream has transformed GET into POST if
// the method raised an exception
trask marked this conversation as resolved.
Show resolved Hide resolved
) {
getOutputStreamContext.outputStreamMethodOfSunConnectionCalled = true;
}
}

public boolean isOutputStreamMethodOfSunConnectionCalled() {
return outputStreamMethodOfSunConnectionCalled;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
/*
* Copyright The OpenTelemetry Authors
* SPDX-License-Identifier: Apache-2.0
*/

package io.opentelemetry.javaagent.instrumentation.httpurlconnection;

import io.opentelemetry.api.common.AttributesBuilder;
import io.opentelemetry.api.trace.Span;
import io.opentelemetry.context.Context;
import io.opentelemetry.instrumentation.api.instrumenter.AttributesExtractor;
import io.opentelemetry.semconv.trace.attributes.SemanticAttributes;
import java.net.HttpURLConnection;
import javax.annotation.Nullable;

public class HttpMethodAttributeExtractor<
REQUEST extends HttpURLConnection, RESPONSE extends Integer>
implements AttributesExtractor<REQUEST, RESPONSE> {

private HttpMethodAttributeExtractor() {}

public static AttributesExtractor<? super HttpURLConnection, ? super Integer> create() {
return new HttpMethodAttributeExtractor<>();
}

@Override
public void onStart(
AttributesBuilder attributes, Context parentContext, HttpURLConnection connection) {}

@Override
public void onEnd(
AttributesBuilder attributes,
Context context,
HttpURLConnection connection,
@Nullable Integer responseCode,
@Nullable Throwable error) {

GetOutputStreamContext getOutputStreamContext = GetOutputStreamContext.get(context);

if (getOutputStreamContext.isOutputStreamMethodOfSunConnectionCalled()) {
String requestMethod = connection.getRequestMethod();
// The getOutputStream() has transformed "GET" into "POST"
attributes.put(SemanticAttributes.HTTP_METHOD, requestMethod);
Span span = Span.fromContext(context);
span.updateName("HTTP " + requestMethod);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,11 @@ public static void methodExit(
callDepth.getAndIncrement();
try {
scope.close();
Class<? extends HttpURLConnection> connectionClass = connection.getClass();

String requestMethod = connection.getRequestMethod();
GetOutputStreamContext.set(
httpUrlState.context, connectionClass, methodName, requestMethod);
Comment on lines +119 to +123
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inlining connectionClass might be nice to get rid of the generic reference above

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would hazzard to guess that weblogic's HttpURLConnection implementation probably had to conform to the sun behavior

Maybe , maybe not. We may discuss this point and create a PR outside this issue? This PR was to fix #4597.


if (throwable != null) {
if (responseCode >= 400) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,10 @@ public final class HttpUrlConnectionSingletons {
.addAttributesExtractor(HttpClientAttributesExtractor.create(httpAttributesGetter))
.addAttributesExtractor(NetClientAttributesExtractor.create(netAttributesGetter))
.addAttributesExtractor(PeerServiceAttributesExtractor.create(netAttributesGetter))
.addAttributesExtractor(HttpMethodAttributeExtractor.create())
.addContextCustomizer(
(context, httpRequestPacket, startAttributes) ->
GetOutputStreamContext.init(context))
.addOperationMetrics(HttpClientMetrics.get())
.newClientInstrumenter(RequestPropertySetter.INSTANCE);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -258,4 +258,69 @@ class HttpUrlConnectionTest extends HttpClientTest<HttpURLConnection> implements
}
}
}

def "sun.net.www.protocol.http.HttpURLConnection.getOutputStream should transform GET into POST"() {
setup:
def url = resolveAddress("/success").toURL()
runWithSpan("someTrace") {

HttpURLConnection connection = url.openConnection()

def connectionClassName = connection.getClass().getName()

assert "sun.net.www.protocol.http.HttpURLConnection".equals(connectionClassName)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will that work on non-OpenJDK JVMs? Can you add some assumption that will skip this test if this class is not present?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tests seem to succeed with the OpenJ9 JVM.

I you want, I could add the following kind of code in case where the sun.net.www.protocol.http.HttpURLConnection class disappears one day from the JDK:

      if (!"sun.net.www.protocol.http.HttpURLConnection".equals(connectionClassName)) {
        return;
      }

I preferred to add only this kind of code if the problem will appear one day, to try to keep things simple.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nowadays everybody uses class library from openjdk so I wouldn't worry about this class being missing. Though I wouldn't probably build the instrumentation around sun.net.www.protocol.http.HttpURLConnection but rather instrument getOutpuStream in all subclasses of java.net.HttpURLConnection because for this use case, as far as I can tell, it isn't really important on which class getOutpuStream is called. Only the fact that it was called matters. Handling this in other classes besides sun.net.www.protocol.http.HttpURLConnection could come handy on weblogic where there is a custom url protocol handler (system property java.protocol.handler.pkgs can be used to change url protocol handlers).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though I wouldn't probably build the instrumentation around sun.net.www.protocol.http.HttpURLConnection but rather instrument getOutpuStream in all subclasses of java.net.HttpURLConnection because for this use case, as far as I can tell, it isn't really important on which class getOutpuStream is called.

As far as I understand, the #4597 issue was created for the sun.net.www.protocol.http.HttpURLConnection class. This issue aims at reflecting the getOutputStream() behavior of the sun.net.www.protocol.http.HttpURLConnection class, extending java.net.HttpURLConnection abstract class. Could we be sure that all classes extending java.net.HttpURLConnection will implement a getOutputStream() method that will transform the GET to POST, as in the sun.net.www.protocol.http.HttpURLConnection class?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I understand, the #4597 issue was created for the sun.net.www.protocol.http.HttpURLConnection class. This issue aims at reflecting the getOutputStream() behavior of the sun.net.www.protocol.http.HttpURLConnection class, extending java.net.HttpURLConnection abstract class. Could we be sure that all classes extending java.net.HttpURLConnection will implement a getOutputStream() method that will transform the GET to POST, as in the sun.net.www.protocol.http.HttpURLConnection class?

This is a good question. I believe we can check the value of getRequestMethod after calling getOutputStream, if it returns POST then it was a POST request even if it returned GET before calling getOutputStream.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good question. I believe we can check the value of getRequestMethod after calling getOutputStream, if it returns POST then it was a POST request even if it returned GET before calling getOutputStream.

I am not sure to have been rather explicit. If I was misunderstood., my question was about the expected behavior, not about implementation.

we can check the value of getRequestMethod after calling getOutputStream, if it returns POST then it was a POST request even if it returned GET before calling getOutputStream.

The getOutputStream() method of sun... class updates the method field. So, the getRequestMethod() should return POST even if the HTTP call was done with a GET method.

In term of expected behavior, do we want all classes extending java.net.HttpURLConnection to have an instrumentation behavior for the getOutputStream() method similar to this of sun.net.www.protocol.http.HttpURLConnection? We can't be sure that all classes extending java.net.HttpURLConnection implement the getOutputStream() method the same way as the sun.net.www.protocol.http.HttpURLConnection does (that is to say, changing GET into POST). As in #4597, it may be disturbing not to have instrumentation aligned with the code behavior?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments:

  1. I committed a new implementation to not depend on the execution order of the connection's methods. Could you please review it?

In term of expected behavior, do we want all classes extending java.net.HttpURLConnection to have an instrumentation behavior for the getOutputStream() method similar to this of sun.net.www.protocol.http.HttpURLConnection? We can't be sure that all classes extending java.net.HttpURLConnection implement the getOutputStream() method the same way as the sun.net.www.protocol.http.HttpURLConnection does (that is to say, changing GET into POST). As in https://github.com/open-telemetry/opentelemetry-java-instrumentation/issues/4597, it may be disturbing not to have instrumentation aligned with the code behavior?

The new implementation could be adjusted once the expected behavior is determined.

The new implementation updates the HTTP method attribute but not the span name. Does span name also need to be updated? If yes, please advise for the implementation. The HttpSpanNameExtractor has no access to Context. I have noticed that the Instrumenter class
span name extractor calls the span name extractor
and has access to the context. So, a possible implementation would be to add Context parameter to the existing method of the SpanNameExtractor interface. It may require updating the code in many places. Another option would be to allow the possibility to define a SpanNameExtractor and call it around here in Instrumenter class. If something needs to be done with the span name, perhaps it could be another PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. If you believe it is best to keep the checks for sun.net.www.protocol.http.HttpURLConnection then that is ok.
  2. Nice catch about the span name. I think we should change it as it too. You could use
Span span = Span.fromContext(context);
span.udateName("HTTP " + requestMethod);

to update the span name too. You could update the span name at the same place where you change the value for HTTP_METHOD attribute.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code updated to set the span name, thank you for the trick


connection.setRequestMethod("GET")

String urlParameters = "q=ASDF&w=&e=&r=12345&t="

// Send POST request
connection.setDoOutput(true)
DataOutputStream wr = new DataOutputStream(connection.getOutputStream())
laurit marked this conversation as resolved.
Show resolved Hide resolved
wr.writeBytes(urlParameters)
wr.flush()
wr.close()

assert connection.getResponseCode() == STATUS

def stream = connection.inputStream
def lines = stream.readLines()
stream.close()
assert lines == [RESPONSE]
}

expect:
assertTraces(1) {
trace(0, 3) {
span(0) {
name "someTrace"
hasNoParent()
attributes {
}
}
span(1) {
name "HTTP POST"
kind CLIENT
childOf span(0)
attributes {
"$SemanticAttributes.NET_TRANSPORT" IP_TCP
"$SemanticAttributes.NET_PEER_NAME" "localhost"
"$SemanticAttributes.NET_PEER_PORT" server.httpPort()
"$SemanticAttributes.HTTP_URL" "$url"
"$SemanticAttributes.HTTP_METHOD" "POST"
"$SemanticAttributes.HTTP_STATUS_CODE" STATUS
"$SemanticAttributes.HTTP_FLAVOR" "1.1"
}
}
span(2) {
name "test-http-server"
kind SERVER
childOf span(1)
attributes {
}
}
}
}
}

}