Use ExecutorService instead of event loop for Netty connection #13904

hanpen24 · 2024-03-11T12:43:33Z

This change addresses the issue of the event loop being blocked for an extended period, improving overall performance and responsiveness.

What is the purpose of the change

This PR addresses the issue #13853 where connectionClient.doConnect() was blocking the event loop for up to 3 seconds.
To resolve this, the connection process has been moved to a separate thread using an ExecutorService.

Brief changelog

Changes:

Removed the scheduling of connectionClient.doConnect() on the event loop.
Added submission of connectionClient.doConnect() to an ExecutorService to execute in a separate thread.

Verifying this change

Checklist

Make sure there is a GitHub_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GitHub issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
Each commit in the pull request should have a meaningful subject line and body.
Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
Check if is necessary to patch to Dubbo 3 if you are work on Dubbo 2.7
Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.
Add some description to dubbo-website project if you are requesting to add a feature.
GitHub Actions works fine on your own branch.
If this contribution is large, please follow the Software Donation Guide.

This change addresses the issue of the event loop being blocked for an extended period, improving overall performance and responsiveness.

AlbumenJ · 2024-03-12T09:40:20Z

Please fix failed case

hanpen24 · 2024-03-12T10:04:41Z

Ok. After fixing failed case, I'll make this ready.

AlbumenJ · 2024-03-12T12:47:15Z

...g-netty4/src/main/java/org/apache/dubbo/remoting/transport/netty4/NettyConnectionClient.java

-                    },
-                    1L,
-                    TimeUnit.SECONDS);
+            executor.submit(() -> {


Also, please create a new isolated executor to reconnect

Thank you very much.
I resolved the error by defining the ExecutorService within the method. However, a new error has occurred, so I will check it.
Considering the current modification, instantiating it inside the method might be costly. Would it be better to have an instance of the ExecutorRepository class as a field in the NettyConnectionClient class?

Yes, you're right. We need to maintain this executor inExecutorRepository

codecov-commenter · 2024-03-17T12:53:43Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.42%. Comparing base (2f6b4c0) to head (1ba41c1).
Report is 25 commits behind head on 3.2.

Additional details and impacted files

@@            Coverage Diff             @@
##              3.2   #13904      +/-   ##
==========================================
+ Coverage   70.39%   70.42%   +0.03%     
==========================================
  Files        1607     1607              
  Lines       70070    70094      +24     
  Branches    10098    10102       +4     
==========================================
+ Hits        49327    49367      +40     
+ Misses      16116    16084      -32     
- Partials     4627     4643      +16

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

AlbumenJ · 2024-03-18T02:33:22Z

...g-netty4/src/main/java/org/apache/dubbo/remoting/transport/netty4/NettyConnectionClient.java

@@ -90,6 +95,8 @@ protected void initConnectionClient() {
        this.closePromise = new DefaultPromise<>(GlobalEventExecutor.INSTANCE);
        this.init = new AtomicBoolean(false);
        this.increase();
+        this.isReconnecting = new AtomicBoolean(false);
+        scheduledExecutorService = Executors.newSingleThreadScheduledExecutor();


This executor should be managed by org.apache.dubbo.common.threadpool.manager.FrameworkExecutorRepository

Thank you for the suggestion. I will make the necessary modifications.

hanpen24 · 2024-03-19T10:08:09Z

@icodening @AlbumenJ
I have made the revisions. Could you please review them

AlbumenJ · 2024-03-20T02:10:31Z

...g-netty4/src/main/java/org/apache/dubbo/remoting/transport/netty4/NettyConnectionClient.java

@@ -374,7 +385,7 @@ public void operationComplete(ChannelFuture future) {
                                    "Failed to connect to server: " + getConnectAddress());
                        }
                    },
-                    1L,
+                    1,


Please sync with org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeClient#calculateReconnectDuration to get the duration

@AlbumenJ
Thank you for your review.
I have modified the code to get the idle time before reconnection. By default, it will now wait for 60000ms.

…ration

AlbumenJ · 2024-03-22T02:26:03Z

...ing/dubbo-remoting-api/src/main/java/org/apache/dubbo/remoting/transport/AbstractClient.java

    public AbstractClient(URL url, ChannelHandler handler) throws RemotingException {
        super(url, handler);
        // set default needReconnect true when channel is not connected
        needReconnect = url.getParameter(Constants.SEND_RECONNECT_KEY, true);

+        applicationModel = url.getOrDefaultApplicationModel();


Directly get Framework Model here would be better

@AlbumenJ
Thank you for your review.
I've made the adjustments based on your suggestions. Could you please check if it aligns with your expectations?

AlbumenJ · 2024-03-22T02:26:37Z

...remoting-netty4/src/test/java/org/apache/dubbo/remoting/transport/netty4/ConnectionTest.java

@@ -138,6 +138,7 @@ void connectSyncTest() throws Throwable {

        nettyPortUnificationServer.bind();
        // auto reconnect
+        Thread.sleep(60000);


Use awaitility to verify

I've made the adjustments that use awaitility.

use awaitility

sonarqubecloud · 2024-03-22T10:56:06Z

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
25.9% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

AlbumenJ · 2024-03-26T11:00:05Z

@icodening PTAL

icodening

LGTM

AlbumenJ · 2024-03-29T03:13:03Z

@EarthChen PTAL

EarthChen

LGTM

…e#13904) * Use ExecutorService instead of event loop for Netty connection This change addresses the issue of the event loop being blocked for an extended period, improving overall performance and responsiveness. * fix log getConnectAddress * applay format * applay format * applay format * applay format * applay format * Use an independent ExecutorService * Use ScheduledExecutorService for scheduling tasks * delete unnecessary files * Modify to stop ExecutorService using shutdownNow * Modify to use ScheduledExecutor managed by FrameworkExecutorRepository * Synchronize reconnectDuration with HeaderExchangeClient's reconnectDuration * get framework model directly use awaitility

hanpen24 added 8 commits March 11, 2024 21:39

Use ExecutorService instead of event loop for Netty connection

38d438e

This change addresses the issue of the event loop being blocked for an extended period, improving overall performance and responsiveness.

fix log getConnectAddress

d3cdfc1

applay format

f9a8fa0

applay format

3146aa6

applay format

392267f

applay format

6f26bc8

applay format

0ac828a

applay format

abdfe13

AlbumenJ requested a review from icodening March 12, 2024 09:40

hanpen24 marked this pull request as draft March 12, 2024 10:00

AlbumenJ requested changes Mar 12, 2024

View reviewed changes

Use an independent ExecutorService

ba4ce96

hanpen24 marked this pull request as ready for review March 13, 2024 15:00

hanpen24 marked this pull request as draft March 13, 2024 15:28

AlbumenJ mentioned this pull request Mar 14, 2024

[Bug] all channels of the EventLoop delay time #13853

Open

4 tasks

hanpen24 added 3 commits March 17, 2024 19:55

Use ScheduledExecutorService for scheduling tasks

dfba18a

delete unnecessary files

bb5d587

Modify to stop ExecutorService using shutdownNow

99f354f

AlbumenJ requested changes Mar 18, 2024

View reviewed changes

Modify to use ScheduledExecutor managed by FrameworkExecutorRepository

d1200f2

hanpen24 marked this pull request as ready for review March 19, 2024 10:01

AlbumenJ requested changes Mar 20, 2024

View reviewed changes

Synchronize reconnectDuration with HeaderExchangeClient's reconnectDu…

2ff51a6

…ration

AlbumenJ requested changes Mar 22, 2024

View reviewed changes

get framework model directly

1ba41c1

use awaitility

AlbumenJ approved these changes Mar 26, 2024

View reviewed changes

icodening approved these changes Mar 26, 2024

View reviewed changes

EarthChen approved these changes Apr 7, 2024

View reviewed changes

EarthChen merged commit f03823e into apache:3.2 Apr 7, 2024
19 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use ExecutorService instead of event loop for Netty connection #13904

Use ExecutorService instead of event loop for Netty connection #13904

hanpen24 commented Mar 11, 2024

AlbumenJ commented Mar 12, 2024

hanpen24 commented Mar 12, 2024

AlbumenJ Mar 12, 2024

hanpen24 Mar 13, 2024

AlbumenJ Mar 14, 2024

codecov-commenter commented Mar 17, 2024 •

edited

Loading

AlbumenJ Mar 18, 2024

hanpen24 Mar 19, 2024

hanpen24 commented Mar 19, 2024

AlbumenJ Mar 20, 2024

hanpen24 Mar 21, 2024

AlbumenJ Mar 22, 2024

hanpen24 Mar 22, 2024

AlbumenJ Mar 22, 2024

hanpen24 Mar 22, 2024

sonarqubecloud bot commented Mar 22, 2024

AlbumenJ commented Mar 26, 2024

icodening left a comment

AlbumenJ commented Mar 29, 2024

EarthChen left a comment

Use ExecutorService instead of event loop for Netty connection #13904

Use ExecutorService instead of event loop for Netty connection #13904

Conversation

hanpen24 commented Mar 11, 2024

What is the purpose of the change

Brief changelog

Verifying this change

Checklist

AlbumenJ commented Mar 12, 2024

hanpen24 commented Mar 12, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented Mar 17, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hanpen24 commented Mar 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Mar 22, 2024

Quality Gate passed

AlbumenJ commented Mar 26, 2024

icodening left a comment

Choose a reason for hiding this comment

AlbumenJ commented Mar 29, 2024

EarthChen left a comment

Choose a reason for hiding this comment

codecov-commenter commented Mar 17, 2024 •

edited

Loading