grpc: Move some stats handler calls to gRPC layer, and add local address to peer.Peer #6716

zasweq · 2023-10-10T23:53:11Z

This PR moves ConnBegin, ConnEnd, and InHeader from the transport layer to the gRPC layer. The other two callouts still in the transport, OutHeader and OutTrailer will require a reworking of the ownership of streaming code and logic with respect to functionality in gRPC layer and transport layer. Continuation of #6624.

RELEASE NOTES:

peer: Add local address to peer.Peer

codecov · 2023-10-11T15:22:06Z

Codecov Report

Merging #6716 (4d2490f) into master (e88e849) will increase coverage by 0.03%.
Report is 2 commits behind head on master.
The diff coverage is 97.72%.

Additional details and impacted files

dfawley · 2023-10-19T21:53:29Z

internal/transport/handler_server.go

@@ -167,6 +167,14 @@ func (ht *serverHandlerTransport) Close(err error) {

 func (ht *serverHandlerTransport) RemoteAddr() net.Addr { return strAddr(ht.req.RemoteAddr) }

+func (ht *serverHandlerTransport) LocalAddr() net.Addr { return nil } // Server Handler transport has no access to local addr (was simply not calling sh with local addr).


I believe this can be obtained with: ht.req.Context().Value(http.LocalAddrContextKey)

Ah ok, done. Since you only want peer though will be added to the Peer() method and this will be deleted.

dfawley · 2023-10-19T22:01:51Z

internal/transport/handler_server.go

+		method:           req.URL.Path,
+		recvCompress:     req.Header.Get("grpc-encoding"),
+		contentSubtype:   ht.contentSubtype,
+		headerWireLength: 0, // doesn't know header wire length, will call into stats handler as 0.


This one doesn't seem to be available. I think we'll need:

golang/go#18997

Please leave a comment in the code referencing the issue.

Done. Thanks.

dfawley · 2023-10-19T22:02:25Z

internal/transport/handler_server.go

+		method:           req.URL.Path,
+		recvCompress:     req.Header.Get("grpc-encoding"),
+		contentSubtype:   ht.contentSubtype,
+		headerWireLength: 0, // doesn't know header wire length, will call into stats handler as 0.
 	}
 	pr := &peer.Peer{


Reuse Peer()?

Ah, good point. Done.

dfawley · 2023-10-19T22:03:41Z

internal/transport/handler_server.go

@@ -347,7 +355,7 @@ func (ht *serverHandlerTransport) WriteHeader(s *Stream, md metadata.MD) error {
 	return err
 }

-func (ht *serverHandlerTransport) HandleStreams(startStream func(*Stream)) {
+func (ht *serverHandlerTransport) HandleStreams(_ context.Context, startStream func(*Stream)) {


Doesn't ignoring the context mess everything up?

Can grpc not pass the http.Request.Context() (with things like Peer added) so that it doesn't need to be ignored here?

It was being ignored in master already, and it uses the requests Context (ctx := ht.req.Context()). I think this is the same functionality, and I don't think I want to add something to gRPC layer using the http.Request since it's a server handler transport specific concept.

Discussed offline; decided to move peer creation to construction time and read it in gRPC before calling this.

dfawley · 2023-10-19T22:14:10Z

internal/transport/transport.go

@@ -698,7 +712,7 @@ type ClientTransport interface {
 // Write methods for a given Stream will be called serially.
 type ServerTransport interface {
 	// HandleStreams receives incoming streams using the given handler.


"the Context given is used as the base context for all streams started on this transport."?

As per comment above, will wait until further discussion on the server handler transport to add this docstring.

Decided to do this, added this comment.

dfawley · 2023-10-19T22:14:50Z

internal/transport/transport.go

+	// Peer returns the peer of the server transport.
+	Peer() *peer.Peer


Should we delete Local/RemoteAddr and only have this?

Sure, done. There's an in flight PR for this but the contributor isn't getting back to my comments so I'll go ahead and just add it to this PR.

dfawley · 2023-10-19T22:15:27Z

server.go

+// GetConnection gets the connection from the context.
+func GetConnection(ctx context.Context) net.Conn {


I don't think we want this to be public.

Hmmm ok I'll plumb it through /internal good point

zasweq

Thanks for the pass. Got to all comments.

arvindbr8 · 2023-10-20T20:16:30Z

@zasweq -- seems like there is a conflict. Adding you to the assignees to take care of that.

zasweq · 2023-10-20T20:43:49Z

Ah thanks resolved

dfawley · 2023-10-23T22:50:25Z

internal/internal.go

@@ -62,6 +62,8 @@ var (
 	// gRPC server. An xDS-enabled server needs to know what type of credentials
 	// is configured on the underlying gRPC server. This is set by server.go.
 	GetServerCredentials any // func (*grpc.Server) credentials.TransportCredentials
+	// GetConnection gets the connection from the context.
+	GetConnection any // func (context.Context) net.Conn


It should be fine to give this a type to avoid the casts, since it's only things from the stdlib.

But, it should also be fine to implement it fully within the internal package. But then.. why not just leave it where it is and export the setter? Or find another place in internal for it?

Thought about this, decided to move back to transport. I don't really think any of the current internal directories make semantical sense, and as per offline discussion I think there's complications about trying to keep file paths consistent.

…side

dfawley

LGTM modulo the last comment.

dfawley · 2023-10-25T19:04:52Z

internal/transport/handler_server.go

@@ -75,11 +75,25 @@ func NewServerHandlerTransport(w http.ResponseWriter, r *http.Request, stats []s
 		return nil, errors.New(msg)
 	}

+	var localAddr net.Addr
+	if la := r.Context().Value(http.LocalAddrContextKey); la != nil {
+		localAddr = la.(net.Addr)


This will panic if the wrong type is here. localAddr, _ = ... is safer.

…ess to peer.Peer (grpc#6716)

zasweq requested a review from dfawley October 10, 2023 23:53

zasweq assigned dfawley Oct 10, 2023

zasweq added the Type: Internal Cleanup Refactors, etc label Oct 10, 2023

zasweq added this to the 1.60 Release milestone Oct 10, 2023

zasweq mentioned this pull request Oct 10, 2023

grpc: Move some stats handler callouts from transport to gRPC layer server side #6624

Closed

Move conn begin conn end and in header to grpc layer

46af5f9

zasweq force-pushed the move-stats-handler-calls-partially-server-side branch from 026fed9 to 46af5f9 Compare October 11, 2023 15:17

zasweq mentioned this pull request Oct 12, 2023

grpc: Add a pointer of server to ctx passed into stats handler #6724

Closed

dfawley reviewed Oct 19, 2023

View reviewed changes

dfawley assigned zasweq and unassigned dfawley Oct 19, 2023

Responded to Doug's comments

1fd9e20

zasweq assigned dfawley and unassigned zasweq Oct 20, 2023

zasweq commented Oct 20, 2023

View reviewed changes

arvindbr8 assigned zasweq Oct 20, 2023

zasweq removed their assignment Oct 20, 2023

dfawley reviewed Oct 23, 2023

View reviewed changes

Responded to offline discussion

8a70332

zasweq force-pushed the move-stats-handler-calls-partially-server-side branch from a57125e to 8a70332 Compare October 23, 2023 23:26

zasweq and others added 5 commits October 23, 2023 19:27

Merge branch 'master' into move-stats-handler-calls-partially-server-…

27c10aa

…side

Test

11a5313

Move getters and setters for conn

1b2cb8c

Vet

18dab87

Fix bug

037ad1e

zasweq assigned dfawley and unassigned dfawley Oct 24, 2023

zasweq mentioned this pull request Oct 24, 2023

Add LocalAddr to peer.Peer (#6577) #6702

Closed

zasweq changed the title ~~grpc: Move some stats handler calls to gRPC layer~~ grpc: Move some stats handler calls to gRPC layer, and add local address to peer.Peer Oct 24, 2023

dfawley approved these changes Oct 25, 2023

View reviewed changes

dfawley assigned zasweq and unassigned dfawley Oct 25, 2023

Responded to Doug's comment

4d2490f

zasweq merged commit c76d75f into grpc:master Oct 25, 2023
14 checks passed

CarlosRDomin mentioned this pull request Nov 3, 2023

add LocalAddr to peer.Peer client side #6577

Closed

arvindbr8 pushed a commit to arvindbr8/grpc-go that referenced this pull request Nov 7, 2023

grpc: Move some stats handler calls to gRPC layer, and add local addr…

15643a2

…ess to peer.Peer (grpc#6716)

CarlosRDomin mentioned this pull request Nov 9, 2023

Add LocalAddr to http2Client.getPeer() #6779

Merged

This was referenced Jan 17, 2024

Collector crashing with invalid memory address or nil pointer dereference on collector exposing a GRPC endpoint behind an AWS ALB open-telemetry/opentelemetry-collector#9296

Closed

stats.Handler's HandleRPC is called with invalid context #6928

Closed

github-actions bot locked as resolved and limited conversation to collaborators Apr 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

grpc: Move some stats handler calls to gRPC layer, and add local address to peer.Peer #6716

grpc: Move some stats handler calls to gRPC layer, and add local address to peer.Peer #6716

zasweq commented Oct 10, 2023 •

edited

Loading

codecov bot commented Oct 11, 2023 •

edited

Loading

dfawley Oct 19, 2023

zasweq Oct 20, 2023

dfawley Oct 19, 2023

zasweq Oct 20, 2023

dfawley Oct 19, 2023

zasweq Oct 20, 2023

dfawley Oct 19, 2023

zasweq Oct 20, 2023

zasweq Oct 23, 2023

dfawley Oct 19, 2023

zasweq Oct 20, 2023

zasweq Oct 23, 2023

dfawley Oct 19, 2023

zasweq Oct 20, 2023

dfawley Oct 19, 2023

zasweq Oct 19, 2023

zasweq left a comment

arvindbr8 commented Oct 20, 2023

zasweq commented Oct 20, 2023

dfawley Oct 23, 2023

zasweq Oct 23, 2023

dfawley left a comment

dfawley Oct 25, 2023

zasweq Oct 25, 2023

		@@ -167,6 +167,14 @@ func (ht *serverHandlerTransport) Close(err error) {

		func (ht *serverHandlerTransport) RemoteAddr() net.Addr { return strAddr(ht.req.RemoteAddr) }

		func (ht *serverHandlerTransport) LocalAddr() net.Addr { return nil } // Server Handler transport has no access to local addr (was simply not calling sh with local addr).

		// Peer returns the peer of the server transport.
		Peer() *peer.Peer

		// GetConnection gets the connection from the context.
		func GetConnection(ctx context.Context) net.Conn {

grpc: Move some stats handler calls to gRPC layer, and add local address to peer.Peer #6716

grpc: Move some stats handler calls to gRPC layer, and add local address to peer.Peer #6716

Conversation

zasweq commented Oct 10, 2023 • edited Loading

codecov bot commented Oct 11, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zasweq left a comment

Choose a reason for hiding this comment

arvindbr8 commented Oct 20, 2023

zasweq commented Oct 20, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dfawley left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zasweq commented Oct 10, 2023 •

edited

Loading

codecov bot commented Oct 11, 2023 •

edited

Loading