-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Failed to create listener: bind: permission denied #782
Comments
There was some discussion of this same issue on #725 after it was closed. |
@serathius could you please take a look? |
Even with permissive PSP the container files to bind the port, changing the container port,LP & RP to 4443 , gets the pod up but it doesnt return metrics.Please help to resolve this issue, we will revert back to older version till this issue is fixed. PSP setting:- spec:
|
@yangjunmyfm192085 @dgrisonnet @maxbrunet @serathius @ikarldasan could you please take a look? |
I'm interested as to why it's desirable to use port |
@pierluigilenoci @hetpats , I think you can do the following check first.
|
@stevehipwell it doesn't really make sense to me to use a port < 1024 for multiple reasons. It just creates complications with no real benefit. @yangjunmyfm192085 I installed the software with this chart #670 and CAP_NET_BIND_SERVICE is not configured if I have seen correctly. @stevehipwell could you please confirm? |
@pierluigilenoci if you want to use a port lower than 1024 you need to setup the security context in the chart to support this. |
@stevehipwell I absolutely don't want to do it, the metrics-server maintainers want that #730. 😄 |
@yangjunmyfm192085 obviously I didn't mention the fact that the problem occurs using the official manifest https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.5.0/components.yaml because it was in the issue text itself. |
@pierluigilenoci I've just looked at the official manifest and as far as I can tell it's not going to work as the capabilities haven't been set in the security context. Have you tried the following security context? securityContext:
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 1000
capabilities:
drop:
- all
add:
- NET_BIND_SERVICE @hetpats it's not the PSP, could you try the security context above? |
@yangjunmyfm192085 the Docker image is set to run as user |
Hi @stevehipwell,
|
@pierluigilenoci What kind of runtime you are using? |
@yangjunmyfm192085 I know the user can be overridden by |
@stevehipwell, I think |
@stevehipwell I didn't do it because I wasn't interested in finding a workaround but that it was fixed at the source. So I have not made any attempt with the @yangjunmyfm192085 I did not understand in detail what you are asking but I used the manifest without any changes. The details of the environment are specified in the issue text. What other detail are you interested in? |
I use the @stevehipwell chart to install the metrics-server and I was able to install it properly after this chart release stevehipwell/helm-charts@9eca9cc#diff-075666af5967de738291733584934578239841b3eaf0caf7965207aba168ed75 |
@yangjunmyfm192085 after looking through #725 I can see that My personal suggestion of what's missing in the official manifest is below. securityContext:
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 65534
runAsGroup: 65534
capabilities:
drop:
- all
add:
- NET_BIND_SERVICE @serathius could you take a look at this thread and offer your thoughts? |
Yeah, I think this chart use the container port 4443 here, so you can install it properly @stevehipwell . |
@yangjunmyfm192085 I use container port |
any way, you solved @pierluigilenoci's issue, thanks. |
@yangjunmyfm192085 I assume that anyone running the official manifests would be getting this issue and I'm saying that it will work if the security context is updated or if the port is made >1024. |
I run the official manifests locally and it’s normal, So I want to know if it will work if the security context is updated in pierluigilenoci's scene |
Main motivation for using port 443 is backward compatibility with https://github.com/kubernetes/kubernetes/blob/2a88664eccdf02cd01777b48171873a028117737/cluster/addons/metrics-server/metrics-server-deployment.yaml#L61 In v0.4.0 stopped using root user in the image, which broke K8s e2e tests. We could not change the port as 443 as this would be a breaking change for firewall rules in common K8s setups (discussion kubernetes/kubernetes#103713 (comment) ). To mitigate it @x13 proposed to add capability on binary to allow binding priviliged ports even for non-root users. This unfortunately impacted also other users, as even though they don't bind privileged port, by giving binary this capability it will always require this to run. As solution I think we should start working on migrating K8s e2e test components out of privileged port. I'm for that, but we need to convince some people. I would start with @liggitt |
@yangjunmyfm192085 I don't understand how the official manifest can work for you. |
use kind or my local machine both ok. |
@serathius I'm not sure why switching the deployment container port to Based on everything said here the release manifests seem to be incorrect and need the capability adding, would you agree @serathius? Also if the capability is always required to run the binary I'm not sure how my chart is currently working as that doesn't have the capability set which makes me think that either Docker isn't copying across the extended attributes or this isn't the case? @yangjunmyfm192085 I've seen it work in Kind too, I think this is because containers are run as privileged due to it being DinD; this caused the first v0.5.0 version of my chart to break as the CI passed. By local what do you mean? |
I would like to point out that for some time Azure AKS has switched to |
@pierluigilenoci that's probably not relevant here as the DinD bit here is to explain why it works on a local machine, but EKS are making the same transition and it's worth knowing. |
I mean, kubernetes runs on my local machine |
@pierluigilenoci Tried running it with above SecurityContext and it still fails. |
Tried it with securitycontext suggested by @pierluigilenoci at it still wouldnt bind to 443 |
Tried this too and still couldnt bind to 443 |
I've just been looking at something completely un-related and have noticed that if you're running with |
Can't wait for the new versions to be released! ❤️ |
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
@pierluigilenoci I think this should be able to be closed now? |
@stevehipwell yep! |
What happened:
I tried to install metrics-server on an AWS EKS cluster with the official manifest
The pod goes into CrashLoopBack with this error:
The only changes between versions 0.4.4 and version 0.5.0 of the manifest are the container port (from
4443
to443
).@stevehipwell also had the same problem, again on EKS.
Ref: #670 (comment)
What you expected to happen:
The pod starts without problems.
Anything else we need to know?:
Environment:
Kubernetes distribution: AWS EKS
Container Network Setup (flannel, calico, etc.): AWS Calico
Kubernetes version (use
kubectl version
): Server Version: version.Info{Major:"1", Minor:"20+", GitVersion:"v1.20.4-eks-6b7464", GitCommit:"6b746440c04cb81db4426842b4ae65c3f7035e53", GitTreeState:"clean", BuildDate:"2021-03-19T19:33:03Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}Metrics Server manifest
https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.5.0/components.yaml
/kind bug
The text was updated successfully, but these errors were encountered: