⚡ http client defer CloseIdleConnections #1513

cuisongliu · 2024-05-09T13:01:57Z

when high concurrency occurs, the connection is full and the connection is rejected.

bwplotka

Hm, my understanding is that this change would only makes things slower, will not help with too many concurrent connections (if you mean that).

Perhaps a reproduction test would help? Perhaps there is some http transport settings you can adjust, or you hit some other limits?

bwplotka · 2024-05-09T15:08:12Z

api/client.go

@@ -122,6 +122,7 @@ func (c *httpClient) Do(ctx context.Context, req *http.Request) (*http.Response,
 	if ctx != nil {
 		req = req.WithContext(ctx)
 	}
+	defer c.client.CloseIdleConnections()


Are you sure this is a correct solution?

My understanding is that this closes connections which might be REUSED on the subsequent Do request. So you have to establish them again (overhead)?

Source: https://forum.golangbridge.org/t/when-should-i-use-client-closeidleconnections/19254, https://stackoverflow.com/questions/18697290/apache-httpclient-how-to-auto-close-connections-by-servers-keep-alive-time

Currently, doing an HTTP probe requiring allocating a new http.Client and http.Transport, and the underlying connections managed by http.Transport might be ESTABLISHED indefinitely if no parties (client or server) directly close them. As a result, if the probes continuously happen, we can see TCP connections opened by one process
drastically increase.

I understand that there may be such a problem in high concurrency. Even if it is reused, the connection is limited and cannot meet the requirements. In this case, the connection may be occupied and new connections cannot be applied for. As a result, the connection request cannot be continued.

PS. I appeared in a high-concurrency scenario, with about 10000 of concurrencies.

Nice! What "HTTP probe" do you mean? Is it some OSS code or your own app?

Generally it sounds you might use this client incorrectly if you create new client for single Do, assuming you can share the client.

For cases you cannot share e.g. it's serverless or it's on-off request to adhoc destination, I think you can access Config.Client (which is used to create API client) and invoke this method after Do on your own, which would be a correct thing to do 👍🏽

I am also open to create CloseIdleConnections() method on our client with a clear commentary when it's useful to make it easier (:

Would that help?

in this conde add Close func ?

// Client is the interface for an API client. type Client interface { URL(ep string, args map[string]string) *url.URL Do(context.Context, *http.Request) (*http.Response, []byte, error) }

I think it would be more reasonable if the interface or open functions could be adjusted and allowed to be called by users themselves. Very useful to me too.

@bwplotka i am add code to interface using CloseIdleConnections() func . plz review this code ?

Why do we need interface change? Can we remove this defer? Otherwise new method is good 👍🏽

Ok interface change is needed. Let's kill the defer statement here and LGTM!

bwplotka · 2024-05-16T11:06:05Z

api/client.go

@@ -75,6 +75,7 @@ func (cfg *Config) validate() error {

 // Client is the interface for an API client.
 type Client interface {
+	CloseIdleConnections()


Let's move it down (as 3rd in order)

bwplotka · 2024-05-16T11:06:20Z

api/client.go

@@ -122,6 +122,7 @@ func (c *httpClient) Do(ctx context.Context, req *http.Request) (*http.Response,
 	if ctx != nil {
 		req = req.WithContext(ctx)
 	}
+	defer c.client.CloseIdleConnections()


Ok interface change is needed. Let's kill the defer statement here and LGTM!

bwplotka · 2024-05-16T11:07:26Z

api/prometheus/v1/api.go

@@ -1342,6 +1342,7 @@ type Warnings []string
 // apiClient wraps a regular client and processes successful API responses.
 // Successful also includes responses that errored at the API level.
 type apiClient interface {


Do we need this change on both interfaces really? Ideally those are very small.

api/prometheus/v1/api.go

bwplotka

So sorry for blocking this PR again, but I think we should remove interface changes. It is possible to avoid them. WDYT?

Thanks for patience!

bwplotka · 2024-05-16T17:11:46Z

api/client.go

@@ -77,6 +77,7 @@ func (cfg *Config) validate() error {
 type Client interface {
 	URL(ep string, args map[string]string) *url.URL
 	Do(context.Context, *http.Request) (*http.Response, []byte, error)
+	CloseIdleConnections()


I double check and I would propose we remove CloseIdleConnections() from both interfaces. We don't want to break compatibility for not too important reason (we kind of can because it's an experimental package, but only if we have to).

This method on interface is not needed because you can totally do

type closeIdler interface { CloseIdleConnections() } // .. in some function cl, err := NewClient() // check err cl.(closeIdler).CloseIdleConnection()

Thus to be safe, I would start with this approach first. It unblocks you without impacting potentially many packages in a very painful way (when upgrading client_golang).

bwplotka

Thanks! Almost there 💪🏽

bwplotka · 2024-07-30T07:26:08Z

api/client.go

@@ -79,6 +79,10 @@ type Client interface {
 	Do(context.Context, *http.Request) (*http.Response, []byte, error)
 }

+type closeIdler interface {


Suggested change

type closeIdler interface {

type CloseIdler interface {

This allows us to avoid shallow ClientCloseIdler function.

bwplotka · 2024-07-30T07:26:50Z

api/client.go

+func ClientCloseIdler(cl Client) {
+	cl.(closeIdler).CloseIdleConnections()
+}
+


Suggested change

func ClientCloseIdler(cl Client) {

cl.(closeIdler).CloseIdleConnections()

}

func ClientCloseIdler(cl Client) {

cl.(closeIdler).CloseIdleConnections()

}

No need for that, anyone can do cl.(CloseIdler).CloseIdleConnection()

…tion is rejected. Signed-off-by: cuisongliu <cuisongliu@qq.com>

cuisongliu force-pushed the close_client branch from b392958 to e74129a Compare May 9, 2024 13:04

bwplotka requested changes May 9, 2024

View reviewed changes

cuisongliu force-pushed the close_client branch 3 times, most recently from 274e000 to 05e818a Compare May 13, 2024 02:20

bwplotka reviewed May 16, 2024

View reviewed changes

cuisongliu force-pushed the close_client branch from 05e818a to 9f1e4e5 Compare May 16, 2024 11:26

bwplotka reviewed May 16, 2024

View reviewed changes

api/prometheus/v1/api.go Outdated Show resolved Hide resolved

cuisongliu force-pushed the close_client branch from 9f1e4e5 to e9f972c Compare May 16, 2024 16:08

bwplotka requested changes May 16, 2024

View reviewed changes

cuisongliu force-pushed the close_client branch 2 times, most recently from ac1acdf to 4029570 Compare May 17, 2024 15:13

bwplotka approved these changes Jul 30, 2024

View reviewed changes

cuisongliu force-pushed the close_client branch from 4029570 to f9acbab Compare July 30, 2024 15:06

⚡ when high concurrency occurs, the connection is full and the connec…

a86b18c

…tion is rejected. Signed-off-by: cuisongliu <cuisongliu@qq.com>

cuisongliu force-pushed the close_client branch from f9acbab to a86b18c Compare July 30, 2024 15:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

⚡ http client defer CloseIdleConnections #1513

⚡ http client defer CloseIdleConnections #1513

cuisongliu commented May 9, 2024

bwplotka left a comment

bwplotka May 9, 2024

cuisongliu May 9, 2024

bwplotka May 10, 2024 •

edited

Loading

cuisongliu May 10, 2024

cuisongliu May 13, 2024

bwplotka May 16, 2024

bwplotka May 16, 2024

bwplotka May 16, 2024

bwplotka May 16, 2024

bwplotka May 16, 2024

bwplotka left a comment

bwplotka May 16, 2024

bwplotka left a comment

bwplotka Jul 30, 2024

bwplotka Jul 30, 2024

bwplotka Jul 30, 2024

cuisongliu Jul 30, 2024

⚡ http client defer CloseIdleConnections #1513

Are you sure you want to change the base?

⚡ http client defer CloseIdleConnections #1513

Conversation

cuisongliu commented May 9, 2024

bwplotka left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bwplotka May 10, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bwplotka left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bwplotka left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bwplotka May 10, 2024 •

edited

Loading