Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to change service from type=NodePort to type=ClusterIP with kubectl #221

Closed
MrHohn opened this issue Jan 23, 2018 · 24 comments
Closed
Labels
kind/bug Categorizes issue or PR as related to a bug. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. sig/cli Categorizes an issue or PR as relevant to SIG CLI.

Comments

@MrHohn
Copy link
Member

MrHohn commented Jan 23, 2018

Is this a BUG REPORT or FEATURE REQUEST? (choose one):
/kind bug

Kubernetes version (use kubectl version):

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.6", GitCommit:"6260bb08c46c31eea6cb538b34a9ceb3e406689c", GitTreeState:"clean", BuildDate:"2017-12-21T06:34:11Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"10+", GitVersion:"v1.10.0-alpha.1.930+0d1986ad8219bf-dirty", GitCommit:"0d1986ad8219bf1a64fc657eac407cc84d3ad647", GitTreeState:"dirty", BuildDate:"2018-01-17T03:10:36Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}

What happened:

  • Create a service with type=ClusterIP. Let's say kube-dns:
apiVersion: v1
kind: Service
metadata:
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
    k8s-app: kube-dns
    kubernetes.io/cluster-service: "true"
    kubernetes.io/name: KubeDNS
  name: kube-dns
  namespace: kube-system
spec:
  clusterIP: 10.0.0.10
  ports:
  - name: dns
    port: 53
    protocol: UDP
    targetPort: 53
  - name: dns-tcp
    port: 53
    protocol: TCP
    targetPort: 53
  selector:
    k8s-app: kube-dns
  type: ClusterIP
  • Edit type from ClusterIP to NodePort, and now we have:
apiVersion: v1
kind: Service
metadata:
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
    k8s-app: kube-dns
    kubernetes.io/cluster-service: "true"
    kubernetes.io/name: KubeDNS
  name: kube-dns
  namespace: kube-system
spec:
  clusterIP: 10.0.0.10
  ports:
  - name: dns
    nodePort: 31716
    port: 53
    protocol: UDP
    targetPort: 53
  - name: dns-tcp
    nodePort: 30492
    port: 53
    protocol: TCP
    targetPort: 53
  selector:
    k8s-app: kube-dns
  type: NodePort
  • Edit type from NodePort to ClusterIP with all the auto-allocated nodePorts removed, and we hit this error:
# services "kube-dns" was not valid:
# * spec.ports[1].nodePort: Forbidden: may not be used when `type` is 'ClusterIP'
  • Try to use kubectl apply -f with a similar kube-dns yaml (type=ClusterIP with all nodePorts removed) and hit the same error:
$ kubectl apply -f kube-dns.yaml 
The Service "kube-dns" is invalid: spec.ports[1].nodePort: Forbidden: may not be used when `type` is 'ClusterIP'

What you expected to happen:
Users should be able to modify service type from NodePort to CluserIP.

Anything else we need to know:
Is this a regression? There was a similar bug before (kubernetes/kubernetes#42282), which was fixed.

@mengqiy @kubernetes/sig-cli-bugs

@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. sig/cli Categorizes an issue or PR as relevant to SIG CLI. labels Jan 23, 2018
@mengqiy
Copy link
Member

mengqiy commented Jan 23, 2018

This is caused by a known issue: mergeKey in ports is portNumber (both are 53) cannot uniquely identify an entry in this case.

@cristianrgreco
Copy link

I've just run into this issue, is there a workaround?

@mengqiy
Copy link
Member

mengqiy commented Mar 16, 2018

kubectl replace can be the workaround.

@Vikash082
Copy link

Hitting similar issue. Any workaround which doesn't delete service ?

@mengqiy
Copy link
Member

mengqiy commented Apr 24, 2018

@Vikash082 Does kubectl replace work for you?

@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 23, 2018
@jaitaiwan
Copy link

Just came across this issue so definitely not stale.

@njzhangyifei
Copy link

kubectl replace worked for me

Note: it requires specifying the clusterIP field to be the same IP as the allocated ClusterIP of this service

@fejta-bot
Copy link

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

@k8s-ci-robot k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Aug 31, 2018
@mabushey
Copy link

Not fixed:

$ kubectl replace -f grav-deployment.yml
deployment.apps/grav replaced
The Service "grav-service" is invalid: spec.clusterIP: Invalid value: "": field is immutable
$ kubectl delete -f grav-deployment.yml                                                                                                                                                                                                           
deployment.apps "grav" deleted
service "grav-service" deleted
$ kubectl apply -f grav-deployment.yml                                                                                                                                                                                                            
deployment.apps/grav created
service/grav-service created

@dixudx
Copy link
Member

dixudx commented Sep 16, 2018

Dup of kubernetes/kubernetes#66390, which has been fixed in kubernetes/kubernetes#66602.

Please try using kubectl apply -f test-service.yaml --force.

/close

Please reopen if needed.

@k8s-ci-robot
Copy link
Contributor

@dixudx: Closing this issue.

In response to this:

Dup of kubernetes/kubernetes#66390, which has been fixed in kubernetes/kubernetes#66602.

Please try using kubectl apply -f test-service.yaml --force.

/close

Please reopen if needed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@willzhang
Copy link

it workd for me
kubectl apply -f test-service.yaml --force.

@zgfh
Copy link

zgfh commented Feb 12, 2019

The Service "ingress-nginx" is invalid:
* spec.ports[0].nodePort: Forbidden: may not be used when `type` is 'ClusterIP'
* spec.ports[1].nodePort: Forbidden: may not be used when `type` is 'ClusterIP'

Should we update the error message?
 

@mpgarate
Copy link

is there a workaround for this that does not require the use of --force?

@MrHohn
Copy link
Member Author

MrHohn commented Feb 21, 2019

@mpgarate Have you tried kubectl replace?

Ref #221 (comment).

@mpgarate
Copy link

I ran into this message with kubectl replace:
The Service ... is invalid: spec.clusterIP: Invalid value: "": field is immutable

I suspect this is because I do not have the ip address hard-coded as mentioned in comment 221. In my case is is not practical to do that for every service, and instead I am using a NodePort type as a workaround.

@MrHohn
Copy link
Member Author

MrHohn commented Feb 21, 2019

@mpgarate Agreed that hardcoding the IP for every service on creation is not practical. But it may be doable to simply copy over the randomly allocated clusterIP when you try to update the service (e.g. via kubectl replace)?

@Kenzku
Copy link

Kenzku commented Mar 12, 2019

use edit service, and
comment out the # nodePort: 30492
change to ClusterIP
save it

@aserrallerios
Copy link

aserrallerios commented Jul 23, 2019

I use helm and this helped me fix the issue:

spec:
  type: {{ .Values.service.type }}
  ports:
    - port: {{ .Values.service.externalPort }}
{{- if (eq .Values.service.type "ClusterIP") }}
      nodePort: null
{{- end }}
      targetPort: {{ .Values.service.internalPort }}
      protocol: TCP
      name: {{ .Values.service.name }}

Edit: this only works after the first install. So there's no "declarative" workaround, you need to change your service manually.

luong-komorebi added a commit to luong-komorebi/kafdrop that referenced this issue Jan 30, 2020
In some use cases, users of kafdrop may opt for clusterIP or other K8s
Service Types.
This help avoid errors from helm when installing the service.
Such error may be expressed as:
```
Forbidden: ... may not be used when `type` is 'ClusterIP'
```
Ref:
1. kubernetes/kubectl#221
2. Similar issue: helm/charts#16940

Signed-off-by: Luong Vo <vo.tran.thanh.luong@gmail.com>
@uhaciogullari
Copy link

nodePort: null definitely works declaratively. Thanks @aserrallerios.

github-actions bot pushed a commit to shantanualsi/kafdrop that referenced this issue Apr 26, 2020
In some use cases, users of kafdrop may opt for clusterIP or other K8s
Service Types.
This help avoid errors from helm when installing the service.
Such error may be expressed as:
```
Forbidden: ... may not be used when `type` is 'ClusterIP'
```
Ref:
1. kubernetes/kubectl#221
2. Similar issue: helm/charts#16940

Signed-off-by: Luong Vo <vo.tran.thanh.luong@gmail.com>
github-actions bot pushed a commit to shantanualsi/kafdrop that referenced this issue Apr 26, 2020
In some use cases, users of kafdrop may opt for clusterIP or other K8s
Service Types.
This help avoid errors from helm when installing the service.
Such error may be expressed as:
```
Forbidden: ... may not be used when `type` is 'ClusterIP'
```
Ref:
1. kubernetes/kubectl#221
2. Similar issue: helm/charts#16940

Signed-off-by: Luong Vo <vo.tran.thanh.luong@gmail.com>
github-actions bot pushed a commit to shantanualsi/kafdrop that referenced this issue Apr 26, 2020
In some use cases, users of kafdrop may opt for clusterIP or other K8s
Service Types.
This help avoid errors from helm when installing the service.
Such error may be expressed as:
```
Forbidden: ... may not be used when `type` is 'ClusterIP'
```
Ref:
1. kubernetes/kubectl#221
2. Similar issue: helm/charts#16940

Signed-off-by: Luong Vo <vo.tran.thanh.luong@gmail.com>
@gopipalamalai
Copy link

nodePort: null definitely works declaratively. Thanks @aserrallerios.

The following worked for me kubectl patch svc my-service --type='json' -p '[{"op":"replace","path":"/spec/type","value":"ClusterIP"},{"op":"replace","path":"/spec/ports/0/nodePort","value":null}]'.

@tarunkhannan
Copy link

Check for indentations too. In my case its the issue :p

yuvipanda added a commit to yuvipanda/datahub-old-fork that referenced this issue Feb 9, 2021
We wanna release our unused static IP addresses,
since all traffic into most hubs now comes in
via the ingress controller. The exceptions are
the data8x hubs, cs194 prod and the datahubs. We
explicitly mark those as LoadBalancer to keep their
public IPs intact. I've already removed their DNS
entries.

You can't actually just change the type from
LoadBalancer to ClusterIP (kubernetes/kubectl#221),
so this command was used to patch them manually
k get ns | rg staging | rg -v datahub | awk '{ print $1; }' | xargs -L1 -I{} kubectl -n {} patch svc proxy-public --type='json' -p '[{"op":"replace","path":"/spec/type","value":"ClusterIP"},{"op":"replace","path":"/spec/ports/0/nodePort","value":null},{"op":"replace","path":"/spec/ports/1/nodePort","value":null},{"op":"replace","path":"/spec/ports/2/nodePort","value":null}]'

Ref #2167
yuvipanda added a commit to yuvipanda/datahub-old-fork that referenced this issue Feb 9, 2021
We wanna release our unused static IP addresses,
since all traffic into most hubs now comes in
via the ingress controller. The exceptions are
the data8x hubs, cs194 prod and the datahubs. We
explicitly mark those as LoadBalancer to keep their
public IPs intact. I've already removed their DNS
entries.

You can't actually just change the type from
LoadBalancer to ClusterIP (kubernetes/kubectl#221),
so this command was used to patch them manually
k get ns | rg staging | rg -v datahub | awk '{ print $1; }' | xargs -L1 -I{} kubectl -n {} patch svc proxy-public --type='json' -p '[{"op":"replace","path":"/spec/type","value":"ClusterIP"},{"op":"replace","path":"/spec/ports/0/nodePort","value":null},{"op":"replace","path":"/spec/ports/1/nodePort","value":null},{"op":"replace","path":"/spec/ports/2/nodePort","value":null}]'

Ref #2167
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. sig/cli Categorizes an issue or PR as relevant to SIG CLI.
Projects
None yet
Development

No branches or pull requests