-
Notifications
You must be signed in to change notification settings - Fork 877
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: zishen <softwarebtg@163.com>
- Loading branch information
Showing
2 changed files
with
136 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,136 @@ | ||
# Remove "derived-" prefix of MCS | ||
|
||
## Summary | ||
|
||
As mentioned in [issue-2384](https://github.com/karmada-io/karmada/issues/2384), currently karmada uses ServiceExport, ServiceImport and other CR to form mcs-controller and manage multi-cluster services. But in the cluster where the service is not located, we need to use a different IP or a service name with a "derived-" prefix to access. We hope to reduce this difference so that different clusters can use the same domain name to access the same service. | ||
|
||
## Motivation | ||
|
||
Currently, in different cluster members of Karmada, if you need to access the same service across clusters, you need to use different ip or domain names with a "derived-" prefix to access. This will increase the complexity of the user's use. We hope that users can use the same domain name to achieve cross-cluster access. | ||
|
||
### Goals | ||
|
||
* Keep the original access method. Avoid the need to make modifications to the customer's system after the upgrade. | ||
* Added the use of a unified domain name for cross-cluster access. The domain name is the access domain name of the cluster where the service resides. | ||
|
||
### Non-Goals | ||
|
||
none | ||
|
||
## Proposal | ||
|
||
### User Stories | ||
|
||
According to the current Karmada's [mcs configuration](https://karmada.io/docs/userguide/service/multi-cluster-service) , if the service is currently in member1, the user needs to access the service in member1, needs to use the domain name: "serve.default.svc.cluster.local", and the domain name in member2 will be: "derived-serve.default.svc.cluster.local". As a result, when customers use the service, they need to add the prefix "derived-" to access the domain name of the service. This brings additional inconvenience. So I hope to be able to: | ||
|
||
1. Provide a unified access method across clusters. | ||
|
||
2. The original access method remains unchanged. So that existing business can also work normally. | ||
|
||
### Notes/Constraints/Caveats (Optional) | ||
|
||
1. The coredns version must be v1.9.3 or above, and the matching k8s verison is at v1.21.0 or above, otherwise the multicluster feature is not supported | ||
|
||
### Risks and Mitigations | ||
|
||
|
||
|
||
## Design Details | ||
|
||
### API change | ||
|
||
none | ||
|
||
### Components change | ||
|
||
Here is the flow diagram: | ||
|
||
![diagrammatic drawing of remove mcs](remove-derived-mcs.png) | ||
|
||
|
||
|
||
1) . Given that the `Service` named `foo` exists on cluster member1 and it has endpointSlice and two pods. | ||
|
||
2). Kamarda make `derived-foo` service and its endpointSlice on member2 through `serviceExport` and `serviceImport`. We will not change these, but add value to `serviceImport`'s `ips`, and this value comes from `derived-foo`'s clusterIP. | ||
|
||
3). Add `coreDNS` plugin `multicluster`, and config ``coreDNS` ` as: https://github.com/coredns/multicluster | ||
|
||
Here's an example using "clusterset.local" domain: | ||
|
||
* Add multicluster processing rules in corefile in coredns | ||
|
||
```shell | ||
kubectl edit configmap coredns -n kube-system | ||
.... | ||
Corefile: | | ||
.:53 { | ||
errors | ||
health { | ||
lameduck 5s | ||
} | ||
ready | ||
multicluster clusterset.local | ||
kubernetes cluster.local in-addr.arpa ip6.arpa { | ||
pods insecure | ||
fallthrough in-addr.arpa ip6.arpa | ||
ttl 30 | ||
} | ||
prometheus :9153 | ||
forward . /etc/resolv.conf { | ||
max_concurrent 1000 | ||
} | ||
cache 30 | ||
loop | ||
reload | ||
loadbalance | ||
} | ||
... | ||
``` | ||
|
||
* RBAC permissions need to add read permissions to `serviceImports` | ||
|
||
excute the cmd: | ||
|
||
```shell | ||
kubectl edit clusterrole system:coredns | ||
``` | ||
|
||
to add the following: | ||
|
||
```shell | ||
... | ||
- apiGroups: | ||
- multicluster.x-k8s.io | ||
resources: | ||
- serviceimports | ||
verbs: | ||
- list | ||
- watch | ||
``` | ||
|
||
* add "clusterset.local" for dns search. | ||
|
||
Edit the /etc/resolv.conf file to add "clusterset.local" | ||
|
||
```shell | ||
root@btg:/# cat /etc/resolv.conf | ||
search default.svc.cluster.local svc.cluster.local cluster.local clusterset.local | ||
nameserver 10.13.0.10 | ||
options ndots:5 | ||
``` | ||
|
||
### Test Plan | ||
|
||
- All current testing should be passed, no break change would be involved by this feature. | ||
- Add new E2E tests to cover the feature, the scope should include: | ||
- Use the same domain name to access service in different clusters . | ||
|
||
|
||
|
||
## Alternatives | ||
|
||
Here are the two ways I know: | ||
|
||
First, using the same service name, not service with`derived-` prefix on other cluster. More details see **[bivas](https://github.com/bivas)** 's pr [proposal for native service discovery](https://github.com/karmada-io/karmada/pull/3694) | ||
|
||
Second, install and using [submariner](https://submariner.io/) . |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.