Ability to export data from ctx/extensions to the outside before it is dropped

[soundofspace]

Current architecture mainly works like this now:

• Outer layer(ctx, reqeuest) -> Response
• Inner layer(ctx, reqeuest) -> (Ctx, Response...)
• Deeper inner layer(ctx, reqeuest) -> (Ctx, Response...)

This makes it possible for each inner layer/svc to modify ctx, or use an internal ctx for some layers and return an updated ctx to the outside service. This way multiple services can work together while achieving isolation at the same time. This however has one huge drawback, information available in ctx is dropped at the outer service and as such is not available to the outside caller.

One example of where this is currently a pain point: Making a request with an HttpClient and trying to access the server certificate chain stored in the NegotiatedTlsParameters.

Some solutions that were discussed in private already and will not be implemented:

1. Always return ctx, even in outside layers. Will not be used as this makes the return type of outer services complicated and weird
2. Inject ctx in response.extensions. Creative workaround but is kind of a nasty solution, and only works if outer layer is an http client

This issue is here to discuss potential solutions and once a rough solution is ready work for it can be started in a PR.

Some potential solutions:

1. Extend all services that drop ctx with an extra parameter beforeDropCallback |ctx: Ctx, response: Response| -> Response. This way external users can use this callback to move data out of ctx and into the response or into some external storage.

2. Split all services that drop ctx into two services.
   The outer service is very simple and only does this and can be generic over response

ctx, response = self.innner(ctx, request).await;
response

The inner service is basically what the service is right now, without the dropping ctx part. By default when using a service like this it would be constructed with both layers immediately after each other. But if needed they could be split into separate layers which would make it easy to design one or multiple layers/services in between them to extract data. This solution is a bit more flexible then the first one, but does require more work

3. Something completely else

[GlenDC]

For category (3) I've been thinking for a long time already what if one can somehow mark an extension as persistent. This would mean that for any "write" operation in Context/Extensions you would also provide a *_persist variant.

This way a leaf service such as a client or server, can automatically "take" all context extensions that are persistent (e.g. because Context/Extensions would allow one to consume itself into something).

I think that's quiet elegant and basically solves what you want automatically.

But up for debate whether or not this is a desired approach.

[soundofspace]

I kinda like that, but doesn't that also require something like solution 1? It feels like one would have to provide a way for this leaf service to transfer data somewhere. Which brings us back to almost the same problem, how to move the data we want (persist stuff) to some place we can access, or do you also have a way to transfer data in mind?

For http client this is pretty trivial, move all persist extensions to response extensions, but don't know how that would translate to other leaf services. But if that is the main focus, then this is a pretty clean solution.

[GlenDC]

No because you would inject it into the response. Eg http::Response::Extension.

that’s the same as what you already would do today, except that it’s automated

the core solution is not specific to http, so if in future someone has something else, eg ICAP, it’s easy enough for that person or team to use this functionality to inject persistent extensions into their ICAP response