Implement service-level payload

[mytang0]

What is the purpose of the change

Implement service-level payload. Issue #10773

Brief changelog

Verifying this change

Checklist

• Make sure there is a GitHub_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GitHub issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.

• Each commit in the pull request should have a meaningful subject line and body.

• Write a pull request description that is detailed enough to understand what the pull request does, how, and why.

• Check if is necessary to patch to Dubbo 3 if you are work on Dubbo 2.7

• Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.

• Add some description to dubbo-website project if you are requesting to add a feature.

• GitHub Actions works fine on your own branch.

• If this contribution is large, please follow the Software Donation Guide.

[codecov-commenter]

Codecov Report

Merging #11517 (68f37e8) into 3.2 (638b193) will increase coverage by 0.04%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##                3.2   #11517      +/-   ##
============================================
+ Coverage     69.34%   69.39%   +0.04%     
  Complexity        2        2              
============================================
  Files          1452     1452              
  Lines         61350    61382      +32     
  Branches       9056     9060       +4     
============================================
+ Hits          42542    42593      +51     
+ Misses        14630    14624       -6     
+ Partials       4178     4165      -13     

Impacted Files	Coverage Δ
.../registry/zookeeper/ZookeeperServiceDiscovery.java	70.51% <0.00%> (-3.85%)	⬇️
.../dubbo/monitor/support/AbstractMonitorFactory.java	80.00% <0.00%> (-3.34%)	⬇️
...org/apache/dubbo/config/AbstractServiceConfig.java	94.25% <0.00%> (-2.18%)	⬇️
...bo/rpc/cluster/support/AbstractClusterInvoker.java	80.92% <0.00%> (-0.66%)	⬇️
...ubbo/config/deploy/DefaultApplicationDeployer.java	74.06% <0.00%> (-0.36%)	⬇️
...java/org/apache/dubbo/rpc/cluster/RouterChain.java	80.28% <0.00%> (ø)
...n/java/org/apache/dubbo/common/utils/NetUtils.java	66.84% <0.00%> (+0.26%)	⬆️
...exchange/support/header/HeaderExchangeChannel.java	83.01% <0.00%> (+0.32%)	⬆️
...mon/src/main/java/org/apache/dubbo/common/URL.java	52.02% <0.00%> (+0.43%)	⬆️
...va/org/apache/dubbo/remoting/exchange/Request.java	94.91% <0.00%> (+0.47%)	⬆️
... and 16 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[wxbty]

Adding a payload to the request at the communication layer is unnecessary and risky. The verification of the payload is in the following fragment. The payload itself is obtained through the url. We may modify the corresponding value in the current url through filter or other interception.

protected static int getPayload(Channel channel) {
    if (channel != null && channel.getUrl() != null) {
        return channel.getUrl().getParameter(Constants.PAYLOAD_KEY, Constants.DEFAULT_PAYLOAD);
    }
    return Constants.DEFAULT_PAYLOAD;
}

[mytang0]

Adding a payload to the request at the communication layer is unnecessary and risky. The verification of the payload is in the following fragment. The payload itself is obtained through the url. We may modify the corresponding value in the current url through filter or other interception.

protected static int getPayload(Channel channel) {
    if (channel != null && channel.getUrl() != null) {
        return channel.getUrl().getParameter(Constants.PAYLOAD_KEY, Constants.DEFAULT_PAYLOAD);
    }
    return Constants.DEFAULT_PAYLOAD;
}

I am thinking about the following:

1. Channel is shared, so modifying it is complicated and risky
2. Channel's URL payload is a protocol payload
3. The payload property of Request is not passed over the network , so it will not increase network overhead and affect the packaging and unpacking of messages

[wxbty]

channel.getUrl() The attributes here come from app-config, service-config and other external configurations, so at least they are shared at the service level.
Since the parameters are obtained from the url, maybe we can achieve the payload configuration at the service level without doing anything. As the following example:

@DubboService(parameters = {"payload", "3000"})
public class GameServiceImpl2 implements GameService {
    public String play(String name) {
        return "hello 2181:" + name;
    }
}

[wxbty]

The configuration of parameters parameters may not be obvious to users, so it should be necessary to add attributes to @DubboService

[mytang0]

Thank you very much for your suggestion. It is true that the solution you suggested is the best implementation when the channel is not shared. Consumers share channels through provider address (host + port), each channel holds the url of the service that created it. Therefore, in the sharing mode, it cannot be guaranteed that the payload obtained through the channel is accurate.

[wxbty]

/**
 * Get dubbo channel by netty channel through channel cache.
 * Put netty channel into it if dubbo channel don't exist in the cache.
 *
 * @param ch      netty channel
 * @param url
 * @param handler dubbo handler that contain netty's handler
 * @return
 */
static NettyChannel getOrAddChannel(Channel ch, URL url, ChannelHandler handler) {
    if (ch == null) {
        return null;
    }
    NettyChannel ret = CHANNEL_MAP.get(ch);
    if (ret == null) {
        NettyChannel nettyChannel = new NettyChannel(ch, url, handler);
        if (ch.isActive()) {
            nettyChannel.markActive(true);
            ret = CHANNEL_MAP.putIfAbsent(ch, nettyChannel);
        }
        if (ret == null) {
            ret = nettyChannel;
        }
    } else {
        ret.markActive(true);
    }
    return ret;
}

@AlbumenJ NettyChannel seems to have a bug here. The key of CHANNEL_MAP is indeed the original channel of netty, and the url is not included, which means that when calling service2, the channel(created by first service) obtained contains the url of service1. The same goes for actual testing. Maybe we can solve it like this: the key of CHANNEL_MAP uses Channel+url instead of Channel

[mytang0]

private List<ReferenceCountExchangeClient> getSharedClient(URL url, int connectNum) {

        String key = url.getAddress();

        Object clients = referenceClientMap.get(key);
        if (clients instanceof List) {
            List<ReferenceCountExchangeClient> typedClients = (List<ReferenceCountExchangeClient>) clients;
            if (checkClientCanUse(typedClients)) {
                batchClientRefIncr(typedClients);
                return typedClients;
            }
        }
        ......
}

In this place, cause the url may already be inaccurate. I think Url is used as a property container, and the remoting layer mainly transmits the configuration properties of the remoting layer. Therefore, when dealing with sharing, Url seems to be less important.

[AlbumenJ]

/**
 * Get dubbo channel by netty channel through channel cache.
 * Put netty channel into it if dubbo channel don't exist in the cache.
 *
 * @param ch      netty channel
 * @param url
 * @param handler dubbo handler that contain netty's handler
 * @return
 */
static NettyChannel getOrAddChannel(Channel ch, URL url, ChannelHandler handler) {
    if (ch == null) {
        return null;
    }
    NettyChannel ret = CHANNEL_MAP.get(ch);
    if (ret == null) {
        NettyChannel nettyChannel = new NettyChannel(ch, url, handler);
        if (ch.isActive()) {
            nettyChannel.markActive(true);
            ret = CHANNEL_MAP.putIfAbsent(ch, nettyChannel);
        }
        if (ret == null) {
            ret = nettyChannel;
        }
    } else {
        ret.markActive(true);
    }
    return ret;
}

@AlbumenJ NettyChannel seems to have a bug here. The key of CHANNEL_MAP is indeed the original channel of netty, and the url is not included, which means that when calling service2, the channel(created by first service) obtained contains the url of service1. The same goes for actual testing. Maybe we can solve it like this: the key of CHANNEL_MAP uses Channel+url instead of Channel

We should prevent to get parameters from channel url. Channel should be shared among services.

[CrazyHZM]

@mytang0 Regarding the payload configuration of this service level, is it possible to bind different services to different protocol configs to achieve the purpose?

[mytang0]

@mytang0 Regarding the payload configuration of this service level, is it possible to bind different services to different protocol configs to achieve the purpose?

Yes, it can be achieved by adding channels, but there is a little violation of channel sharing.

[mytang0]

PTAL @chickenlj

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
3 Code Smells

75.5% Coverage
0.0% Duplication

[chickenlj]

LGTM.