Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ports not exposed on bare metal installation #7547

Closed
darox opened this issue Aug 26, 2021 · 3 comments
Closed

Ports not exposed on bare metal installation #7547

darox opened this issue Aug 26, 2021 · 3 comments
Assignees
@kundan2707
Labels
kind/support Categorizes issue or PR as a support question. needs-priority needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.

Comments

@darox
Copy link

darox commented Aug 26, 2021 •
edited
Loading

NGINX Ingress controller version: v1.0.0

Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.0", GitCommit:"c2b5237ccd9c0f1d600d3072634ca66cefdf272f", GitTreeState:"clean", BuildDate:"2021-08-04T18:03:20Z", GoVersion:"go1.16.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.0", GitCommit:"c2b5237ccd9c0f1d600d3072634ca66cefdf272f", GitTreeState:"clean", BuildDate:"2021-08-04T17:57:25Z", GoVersion:"go1.16.6", Compiler:"gc", Platform:"linux/amd64"}

Environment:

  • Cloud provider or hardware configuration: VMs on Esxi 7.x
  • OS (e.g. from /etc/os-release): Ubuntu 20.04
  • Kernel (e.g. uname -a): Linux kub-master-1 5.4.0-81-generic Fix x-forwarded-port mapping #91-Ubuntu SMP Thu Jul 15 19:09:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
  • Install tools: Via kubeadm
  • Basic cluster related info:
NAME           STATUS   ROLES                  AGE     VERSION
kub-master-1   Ready    control-plane,master   6d22h   v1.22.0
kub-worker-1   Ready    <none>                 6d22h   v1.22.0

NAME                                       READY   STATUS      RESTARTS   AGE   IP           NODE           NOMINATED NODE   READINESS GATES
ingress-nginx-admission-create--1-vsdxw    0/1     Completed   0          34m   10.0.0.9     kub-worker-1   <none>           <none>
ingress-nginx-admission-patch--1-9z2cd     0/1     Completed   0          34m   10.0.0.140   kub-worker-1   <none>           <none>
ingress-nginx-controller-8cf5559f8-2fl97   1/1     Running     0          34m   10.0.0.95    kub-worker-1   <none>           <none>

  • How was the ingress-nginx-controller installed:
    According to: https://kubernetes.github.io/ingress-nginx/deploy/
    kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.0.0/deploy/static/provider/baremetal/deploy.yaml

  • Current State of the controller:

  NAMESPACE        NAME                                           READY   STATUS      RESTARTS        AGE     IP             NODE           NOMINATED NODE   READINESS GATES
emojivoto        pod/emoji-66ccdb4d86-fp9hl                     1/1     Running     0               6d19h   10.0.0.110     kub-worker-1   <none>           <none>
emojivoto        pod/vote-bot-69754c864f-754xv                  1/1     Running     0               6d19h   10.0.0.61      kub-worker-1   <none>           <none>
emojivoto        pod/voting-f999bd4d7-sxgwm                     1/1     Running     0               6d19h   10.0.0.222     kub-worker-1   <none>           <none>
emojivoto        pod/web-79469b946f-rn922                       1/1     Running     0               6d19h   10.0.0.212     kub-worker-1   <none>           <none>
ingress-nginx    pod/ingress-nginx-admission-create--1-vsdxw    0/1     Completed   0               35m     10.0.0.9       kub-worker-1   <none>           <none>
ingress-nginx    pod/ingress-nginx-admission-patch--1-9z2cd     0/1     Completed   0               35m     10.0.0.140     kub-worker-1   <none>           <none>
ingress-nginx    pod/ingress-nginx-controller-8cf5559f8-2fl97   1/1     Running     0               35m     10.0.0.95      kub-worker-1   <none>           <none>
kube-system      pod/cilium-m2994                               1/1     Running     0               30m     192.168.2.86   kub-worker-1   <none>           <none>
kube-system      pod/cilium-operator-65d464c5f6-kpxzv           1/1     Running     0               6d18h   192.168.2.52   kub-master-1   <none>           <none>
kube-system      pod/cilium-operator-65d464c5f6-m9wzl           1/1     Running     0               6d18h   192.168.2.86   kub-worker-1   <none>           <none>
kube-system      pod/cilium-qvrwd                               1/1     Running     0               30m     192.168.2.52   kub-master-1   <none>           <none>
kube-system      pod/coredns-78fcd69978-gtjvd                   1/1     Running     1 (6d22h ago)   6d22h   10.0.1.199     kub-master-1   <none>           <none>
kube-system      pod/coredns-78fcd69978-rwz8b                   1/1     Running     1 (6d22h ago)   6d22h   10.0.1.13      kub-master-1   <none>           <none>
kube-system      pod/etcd-kub-master-1                          1/1     Running     1               6d22h   192.168.2.52   kub-master-1   <none>           <none>
kube-system      pod/hubble-relay-6fbb7b8d7c-2pl9f              1/1     Running     0               6d18h   10.0.0.159     kub-worker-1   <none>           <none>
kube-system      pod/hubble-ui-5df5fb587d-wbfkb                 3/3     Running     0               6d18h   10.0.0.165     kub-worker-1   <none>           <none>
kube-system      pod/kube-apiserver-kub-master-1                1/1     Running     1               6d22h   192.168.2.52   kub-master-1   <none>           <none>
kube-system      pod/kube-controller-manager-kub-master-1       1/1     Running     1               6d22h   192.168.2.52   kub-master-1   <none>           <none>
kube-system      pod/kube-scheduler-kub-master-1                1/1     Running     1               6d22h   192.168.2.52   kub-master-1   <none>           <none>
linkerd-jaeger   pod/collector-7b5b896747-fvz9w                 2/2     Running     0               6d21h   10.0.0.63      kub-worker-1   <none>           <none>
linkerd-jaeger   pod/jaeger-6fdd6746c8-98227                    2/2     Running     0               6d21h   10.0.0.220     kub-worker-1   <none>           <none>
linkerd-jaeger   pod/jaeger-injector-696b84c4cf-n4sss           2/2     Running     0               6d21h   10.0.0.39      kub-worker-1   <none>           <none>
linkerd-viz      pod/grafana-5c47556496-h4kwf                   2/2     Running     0               6d21h   10.0.0.8       kub-worker-1   <none>           <none>
linkerd-viz      pod/metrics-api-86b88fddc-rhz62                2/2     Running     0               6d21h   10.0.0.237     kub-worker-1   <none>           <none>
linkerd-viz      pod/prometheus-554c6cc89f-hk8jt                2/2     Running     0               6d21h   10.0.0.17      kub-worker-1   <none>           <none>
linkerd-viz      pod/tap-b6cb479d-mkgrs                         2/2     Running     0               6d21h   10.0.0.92      kub-worker-1   <none>           <none>
linkerd-viz      pod/tap-injector-86cf45cf7f-rr4l9              2/2     Running     0               6d21h   10.0.0.236     kub-worker-1   <none>           <none>
linkerd-viz      pod/web-6b8894468b-8nwxs                       2/2     Running     0               6d21h   10.0.0.251     kub-worker-1   <none>           <none>
linkerd          pod/linkerd-controller-7c54c8f4db-qwtkt        2/2     Running     1 (6d21h ago)   6d21h   10.0.0.120     kub-worker-1   <none>           <none>
linkerd          pod/linkerd-destination-6d6bdb88c8-v8mcv       2/2     Running     1 (6d21h ago)   6d21h   10.0.0.7       kub-worker-1   <none>           <none>
linkerd          pod/linkerd-identity-7dd5f596f7-jbch6          2/2     Running     0               6d21h   10.0.0.191     kub-worker-1   <none>           <none>
linkerd          pod/linkerd-proxy-injector-54fcfd8567-sw5gb    2/2     Running     1 (6d21h ago)   6d21h   10.0.0.211     kub-worker-1   <none>           <none>
linkerd          pod/linkerd-sp-validator-75c666bd8c-spqjj      2/2     Running     1 (6d21h ago)   6d21h   10.0.0.68      kub-worker-1   <none>           <none>

NAMESPACE        NAME                                         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                        AGE     SELECTOR
default          service/kubernetes                           ClusterIP   10.96.0.1        <none>        443/TCP                        6d22h   <none>
emojivoto        service/emoji-svc                            ClusterIP   10.110.37.119    <none>        8080/TCP,8801/TCP              6d19h   app=emoji-svc
emojivoto        service/voting-svc                           ClusterIP   10.103.158.91    <none>        8080/TCP,8801/TCP              6d19h   app=voting-svc
emojivoto        service/web-svc                              ClusterIP   10.109.231.168   <none>        80/TCP                         6d19h   app=web-svc
ingress-nginx    service/ingress-nginx-controller             NodePort    10.110.109.139   <none>        80:30261/TCP,443:32461/TCP     35m     app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
ingress-nginx    service/ingress-nginx-controller-admission   ClusterIP   10.100.17.100    <none>        443/TCP                        35m     app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
kube-system      service/hubble-relay                         ClusterIP   10.101.85.5      <none>        80/TCP                         6d18h   k8s-app=hubble-relay
kube-system      service/hubble-ui                            ClusterIP   10.108.24.27     <none>        80/TCP                         6d18h   k8s-app=hubble-ui
kube-system      service/kube-dns                             ClusterIP   10.96.0.10       <none>        53/UDP,53/TCP,9153/TCP         6d22h   k8s-app=kube-dns
linkerd-jaeger   service/collector                            ClusterIP   10.111.30.86     <none>        55678/TCP,9411/TCP,14268/TCP   6d21h   component=collector
linkerd-jaeger   service/jaeger                               ClusterIP   10.96.151.140    <none>        14268/TCP,16686/TCP            6d21h   component=jaeger
linkerd-jaeger   service/jaeger-injector                      ClusterIP   10.99.109.135    <none>        443/TCP                        6d21h   component=jaeger-injector,linkerd.io/extension=jaeger
linkerd-viz      service/grafana                              ClusterIP   10.100.141.172   <none>        3000/TCP                       6d21h   component=grafana,linkerd.io/extension=viz
linkerd-viz      service/metrics-api                          ClusterIP   10.97.122.71     <none>        8085/TCP                       6d21h   component=metrics-api,linkerd.io/extension=viz
linkerd-viz      service/prometheus                           ClusterIP   10.109.112.12    <none>        9090/TCP                       6d21h   component=prometheus,linkerd.io/extension=viz
linkerd-viz      service/tap                                  ClusterIP   10.100.188.238   <none>        8088/TCP,443/TCP               6d21h   component=tap,linkerd.io/extension=viz
linkerd-viz      service/tap-injector                         ClusterIP   10.102.180.241   <none>        443/TCP                        6d21h   component=tap-injector,linkerd.io/extension=viz
linkerd-viz      service/web                                  ClusterIP   10.100.182.244   <none>        8084/TCP,9994/TCP              6d21h   component=web,linkerd.io/extension=viz
linkerd          service/linkerd-controller-api               ClusterIP   10.108.155.195   <none>        8085/TCP                       6d21h   linkerd.io/control-plane-component=controller
linkerd          service/linkerd-dst                          ClusterIP   10.105.248.125   <none>        8086/TCP                       6d21h   linkerd.io/control-plane-component=destination
linkerd          service/linkerd-dst-headless                 ClusterIP   None             <none>        8086/TCP                       6d21h   linkerd.io/control-plane-component=destination
linkerd          service/linkerd-identity                     ClusterIP   10.103.224.28    <none>        8080/TCP                       6d21h   linkerd.io/control-plane-component=identity
linkerd          service/linkerd-identity-headless            ClusterIP   None             <none>        8080/TCP                       6d21h   linkerd.io/control-plane-component=identity
linkerd          service/linkerd-proxy-injector               ClusterIP   10.96.124.140    <none>        443/TCP                        6d21h   linkerd.io/control-plane-component=proxy-injector
linkerd          service/linkerd-sp-validator                 ClusterIP   10.98.50.98      <none>        443/TCP                        6d21h   linkerd.io/control-plane-component=sp-validator

NAMESPACE     NAME                    DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE     CONTAINERS     IMAGES                         SELECTOR
kube-system   daemonset.apps/cilium   2         2         2       2            2           <none>          6d18h   cilium-agent   quay.io/cilium/cilium:v1.9.9   k8s-app=cilium

NAMESPACE        NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS                     IMAGES                                                                                                               SELECTOR
emojivoto        deployment.apps/emoji                      1/1     1            1           6d19h   emoji-svc                      docker.l5d.io/buoyantio/emojivoto-emoji-svc:v11                                                                      app=emoji-svc,version=v11
emojivoto        deployment.apps/vote-bot                   1/1     1            1           6d19h   vote-bot                       docker.l5d.io/buoyantio/emojivoto-web:v11                                                                            app=vote-bot,version=v11
emojivoto        deployment.apps/voting                     1/1     1            1           6d19h   voting-svc                     docker.l5d.io/buoyantio/emojivoto-voting-svc:v11                                                                     app=voting-svc,version=v11
emojivoto        deployment.apps/web                        1/1     1            1           6d19h   web-svc                        docker.l5d.io/buoyantio/emojivoto-web:v11                                                                            app=web-svc,version=v11
ingress-nginx    deployment.apps/ingress-nginx-controller   1/1     1            1           35m     controller                     k8s.gcr.io/ingress-nginx/controller:v1.0.0@sha256:0851b34f69f69352bf168e6ccf30e1e20714a264ab1ecd1933e4d8c0fc3215c6   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
kube-system      deployment.apps/cilium-operator            2/2     2            2           6d18h   cilium-operator                quay.io/cilium/operator-generic:v1.9.9                                                                               io.cilium/app=operator,name=cilium-operator
kube-system      deployment.apps/coredns                    2/2     2            2           6d22h   coredns                        k8s.gcr.io/coredns/coredns:v1.8.4                                                                                    k8s-app=kube-dns
kube-system      deployment.apps/hubble-relay               1/1     1            1           6d18h   hubble-relay                   quay.io/cilium/hubble-relay:v1.9.9                                                                                   k8s-app=hubble-relay
kube-system      deployment.apps/hubble-ui                  1/1     1            1           6d18h   frontend,backend,proxy         quay.io/cilium/hubble-ui:v0.7.3,quay.io/cilium/hubble-ui-backend:v0.7.3,docker.io/envoyproxy/envoy:v1.14.5           k8s-app=hubble-ui
linkerd-jaeger   deployment.apps/collector                  1/1     1            1           6d21h   oc-collector                   omnition/opencensus-collector:0.1.11                                                                                 component=collector
linkerd-jaeger   deployment.apps/jaeger                     1/1     1            1           6d21h   jaeger                         jaegertracing/all-in-one:1.19.2                                                                                      component=jaeger
linkerd-jaeger   deployment.apps/jaeger-injector            1/1     1            1           6d21h   jaeger-injector                cr.l5d.io/linkerd/jaeger-webhook:stable-2.10.2                                                                       component=jaeger-injector,linkerd.io/extension=jaeger
linkerd-viz      deployment.apps/grafana                    1/1     1            1           6d21h   grafana                        cr.l5d.io/linkerd/grafana:stable-2.10.2                                                                              component=grafana,linkerd.io/extension=viz,namespace=linkerd-viz
linkerd-viz      deployment.apps/metrics-api                1/1     1            1           6d21h   metrics-api                    cr.l5d.io/linkerd/metrics-api:stable-2.10.2                                                                          component=metrics-api,linkerd.io/extension=viz
linkerd-viz      deployment.apps/prometheus                 1/1     1            1           6d21h   prometheus                     prom/prometheus:v2.19.3                                                                                              component=prometheus,linkerd.io/extension=viz,namespace=linkerd-viz
linkerd-viz      deployment.apps/tap                        1/1     1            1           6d21h   tap                            cr.l5d.io/linkerd/tap:stable-2.10.2                                                                                  component=tap,linkerd.io/extension=viz,namespace=linkerd-viz
linkerd-viz      deployment.apps/tap-injector               1/1     1            1           6d21h   tap-injector                   cr.l5d.io/linkerd/tap:stable-2.10.2                                                                                  component=tap-injector
linkerd-viz      deployment.apps/web                        1/1     1            1           6d21h   web                            cr.l5d.io/linkerd/web:stable-2.10.2                                                                                  component=web,linkerd.io/extension=viz,namespace=linkerd-viz
linkerd          deployment.apps/linkerd-controller         1/1     1            1           6d21h   public-api,linkerd-proxy       cr.l5d.io/linkerd/controller:stable-2.10.2,cr.l5d.io/linkerd/proxy:stable-2.10.2                                     linkerd.io/control-plane-component=controller,linkerd.io/control-plane-ns=linkerd,linkerd.io/proxy-deployment=linkerd-controller
linkerd          deployment.apps/linkerd-destination        1/1     1            1           6d21h   destination,linkerd-proxy      cr.l5d.io/linkerd/controller:stable-2.10.2,cr.l5d.io/linkerd/proxy:stable-2.10.2                                     linkerd.io/control-plane-component=destination,linkerd.io/control-plane-ns=linkerd,linkerd.io/proxy-deployment=linkerd-destination
linkerd          deployment.apps/linkerd-identity           1/1     1            1           6d21h   identity,linkerd-proxy         cr.l5d.io/linkerd/controller:stable-2.10.2,cr.l5d.io/linkerd/proxy:stable-2.10.2                                     linkerd.io/control-plane-component=identity,linkerd.io/control-plane-ns=linkerd,linkerd.io/proxy-deployment=linkerd-identity
linkerd          deployment.apps/linkerd-proxy-injector     1/1     1            1           6d21h   proxy-injector,linkerd-proxy   cr.l5d.io/linkerd/controller:stable-2.10.2,cr.l5d.io/linkerd/proxy:stable-2.10.2                                     linkerd.io/control-plane-component=proxy-injector
linkerd          deployment.apps/linkerd-sp-validator       1/1     1            1           6d21h   sp-validator,linkerd-proxy     cr.l5d.io/linkerd/controller:stable-2.10.2,cr.l5d.io/linkerd/proxy:stable-2.10.2                                     linkerd.io/control-plane-component=sp-validator

NAMESPACE        NAME                                                 DESIRED   CURRENT   READY   AGE     CONTAINERS                     IMAGES                                                                                                               SELECTOR
emojivoto        replicaset.apps/emoji-66ccdb4d86                     1         1         1       6d19h   emoji-svc                      docker.l5d.io/buoyantio/emojivoto-emoji-svc:v11                                                                      app=emoji-svc,pod-template-hash=66ccdb4d86,version=v11
emojivoto        replicaset.apps/vote-bot-69754c864f                  1         1         1       6d19h   vote-bot                       docker.l5d.io/buoyantio/emojivoto-web:v11                                                                            app=vote-bot,pod-template-hash=69754c864f,version=v11
emojivoto        replicaset.apps/voting-f999bd4d7                     1         1         1       6d19h   voting-svc                     docker.l5d.io/buoyantio/emojivoto-voting-svc:v11                                                                     app=voting-svc,pod-template-hash=f999bd4d7,version=v11
emojivoto        replicaset.apps/web-79469b946f                       1         1         1       6d19h   web-svc                        docker.l5d.io/buoyantio/emojivoto-web:v11                                                                            app=web-svc,pod-template-hash=79469b946f,version=v11
ingress-nginx    replicaset.apps/ingress-nginx-controller-8cf5559f8   1         1         1       35m     controller                     k8s.gcr.io/ingress-nginx/controller:v1.0.0@sha256:0851b34f69f69352bf168e6ccf30e1e20714a264ab1ecd1933e4d8c0fc3215c6   app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx,pod-template-hash=8cf5559f8
kube-system      replicaset.apps/cilium-operator-65d464c5f6           2         2         2       6d18h   cilium-operator                quay.io/cilium/operator-generic:v1.9.9                                                                               io.cilium/app=operator,name=cilium-operator,pod-template-hash=65d464c5f6
kube-system      replicaset.apps/coredns-78fcd69978                   2         2         2       6d22h   coredns                        k8s.gcr.io/coredns/coredns:v1.8.4                                                                                    k8s-app=kube-dns,pod-template-hash=78fcd69978
kube-system      replicaset.apps/hubble-relay-6fbb7b8d7c              1         1         1       6d18h   hubble-relay                   quay.io/cilium/hubble-relay:v1.9.9                                                                                   k8s-app=hubble-relay,pod-template-hash=6fbb7b8d7c
kube-system      replicaset.apps/hubble-ui-5df5fb587d                 1         1         1       6d18h   frontend,backend,proxy         quay.io/cilium/hubble-ui:v0.7.3,quay.io/cilium/hubble-ui-backend:v0.7.3,docker.io/envoyproxy/envoy:v1.14.5           k8s-app=hubble-ui,pod-template-hash=5df5fb587d
linkerd-jaeger   replicaset.apps/collector-7b5b896747                 1         1         1       6d21h   oc-collector                   omnition/opencensus-collector:0.1.11                                                                                 component=collector,pod-template-hash=7b5b896747
linkerd-jaeger   replicaset.apps/jaeger-6fdd6746c8                    1         1         1       6d21h   jaeger                         jaegertracing/all-in-one:1.19.2                                                                                      component=jaeger,pod-template-hash=6fdd6746c8
linkerd-jaeger   replicaset.apps/jaeger-injector-696b84c4cf           1         1         1       6d21h   jaeger-injector                cr.l5d.io/linkerd/jaeger-webhook:stable-2.10.2                                                                       component=jaeger-injector,linkerd.io/extension=jaeger,pod-template-hash=696b84c4cf
linkerd-viz      replicaset.apps/grafana-5c47556496                   1         1         1       6d21h   grafana                        cr.l5d.io/linkerd/grafana:stable-2.10.2                                                                              component=grafana,linkerd.io/extension=viz,namespace=linkerd-viz,pod-template-hash=5c47556496
linkerd-viz      replicaset.apps/metrics-api-86b88fddc                1         1         1       6d21h   metrics-api                    cr.l5d.io/linkerd/metrics-api:stable-2.10.2                                                                          component=metrics-api,linkerd.io/extension=viz,pod-template-hash=86b88fddc
linkerd-viz      replicaset.apps/prometheus-554c6cc89f                1         1         1       6d21h   prometheus                     prom/prometheus:v2.19.3                                                                                              component=prometheus,linkerd.io/extension=viz,namespace=linkerd-viz,pod-template-hash=554c6cc89f
linkerd-viz      replicaset.apps/tap-b6cb479d                         1         1         1       6d21h   tap                            cr.l5d.io/linkerd/tap:stable-2.10.2                                                                                  component=tap,linkerd.io/extension=viz,namespace=linkerd-viz,pod-template-hash=b6cb479d
linkerd-viz      replicaset.apps/tap-injector-86cf45cf7f              1         1         1       6d21h   tap-injector                   cr.l5d.io/linkerd/tap:stable-2.10.2                                                                                  component=tap-injector,pod-template-hash=86cf45cf7f
linkerd-viz      replicaset.apps/web-6b8894468b                       1         1         1       6d21h   web                            cr.l5d.io/linkerd/web:stable-2.10.2                                                                                  component=web,linkerd.io/extension=viz,namespace=linkerd-viz,pod-template-hash=6b8894468b
linkerd          replicaset.apps/linkerd-controller-7c54c8f4db        1         1         1       6d21h   public-api,linkerd-proxy       cr.l5d.io/linkerd/controller:stable-2.10.2,cr.l5d.io/linkerd/proxy:stable-2.10.2                                     linkerd.io/control-plane-component=controller,linkerd.io/control-plane-ns=linkerd,linkerd.io/proxy-deployment=linkerd-controller,pod-template-hash=7c54c8f4db
linkerd          replicaset.apps/linkerd-destination-6d6bdb88c8       1         1         1       6d21h   destination,linkerd-proxy      cr.l5d.io/linkerd/controller:stable-2.10.2,cr.l5d.io/linkerd/proxy:stable-2.10.2                                     linkerd.io/control-plane-component=destination,linkerd.io/control-plane-ns=linkerd,linkerd.io/proxy-deployment=linkerd-destination,pod-template-hash=6d6bdb88c8
linkerd          replicaset.apps/linkerd-identity-7dd5f596f7          1         1         1       6d21h   identity,linkerd-proxy         cr.l5d.io/linkerd/controller:stable-2.10.2,cr.l5d.io/linkerd/proxy:stable-2.10.2                                     linkerd.io/control-plane-component=identity,linkerd.io/control-plane-ns=linkerd,linkerd.io/proxy-deployment=linkerd-identity,pod-template-hash=7dd5f596f7
linkerd          replicaset.apps/linkerd-proxy-injector-54fcfd8567    1         1         1       6d21h   proxy-injector,linkerd-proxy   cr.l5d.io/linkerd/controller:stable-2.10.2,cr.l5d.io/linkerd/proxy:stable-2.10.2                                     linkerd.io/control-plane-component=proxy-injector,pod-template-hash=54fcfd8567
linkerd          replicaset.apps/linkerd-sp-validator-75c666bd8c      1         1         1       6d21h   sp-validator,linkerd-proxy     cr.l5d.io/linkerd/controller:stable-2.10.2,cr.l5d.io/linkerd/proxy:stable-2.10.2                                     linkerd.io/control-plane-component=sp-validator,pod-template-hash=75c666bd8c

NAMESPACE   NAME                              SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE     CONTAINERS   IMAGES                                       SELECTOR
linkerd     cronjob.batch/linkerd-heartbeat   53 10 * * *   False     0        21h             6d21h   heartbeat    cr.l5d.io/linkerd/controller:stable-2.10.2   <none>

NAMESPACE       NAME                                       COMPLETIONS   DURATION   AGE   CONTAINERS   IMAGES                                                                                                                       SELECTOR
ingress-nginx   job.batch/ingress-nginx-admission-create   1/1           4s         35m   create       k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068   controller-uid=91ce4ff4-1f00-4928-b792-6a9268fca2bc
ingress-nginx   job.batch/ingress-nginx-admission-patch    1/1           3s         35m   patch        k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0@sha256:f3b6b39a6062328c095337b4cadcefd1612348fdd5190b1dcbcb9b9e90bd8068   controller-uid=4ce7d608-4601-4766-9a29-8c3322d8c8f5

Name:         ingress-nginx-controller-8cf5559f8-2fl97
Namespace:    ingress-nginx
Priority:     0
Node:         kub-worker-1/192.168.2.86
Start Time:   Thu, 26 Aug 2021 08:03:51 +0000
Labels:       app.kubernetes.io/component=controller
              app.kubernetes.io/instance=ingress-nginx
              app.kubernetes.io/name=ingress-nginx
              pod-template-hash=8cf5559f8
Annotations:  <none>
Status:       Running
IP:           10.0.0.95
IPs:
  IP:           10.0.0.95
Controlled By:  ReplicaSet/ingress-nginx-controller-8cf5559f8
Containers:
  controller:
    Container ID:  docker://94660ceb65143005ad1d5d65f7b2280dd0c41278108dce75d181033ef92c59e0
    Image:         k8s.gcr.io/ingress-nginx/controller:v1.0.0@sha256:0851b34f69f69352bf168e6ccf30e1e20714a264ab1ecd1933e4d8c0fc3215c6
    Image ID:      docker-pullable://k8s.gcr.io/ingress-nginx/controller@sha256:0851b34f69f69352bf168e6ccf30e1e20714a264ab1ecd1933e4d8c0fc3215c6
    Ports:         80/TCP, 443/TCP, 8443/TCP
    Host Ports:    0/TCP, 0/TCP, 0/TCP
    Args:
      /nginx-ingress-controller
      --election-id=ingress-controller-leader
      --controller-class=k8s.io/ingress-nginx
      --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
      --validating-webhook=:8443
      --validating-webhook-certificate=/usr/local/certificates/cert
      --validating-webhook-key=/usr/local/certificates/key
    State:          Running
      Started:      Thu, 26 Aug 2021 08:03:53 +0000
    Ready:          True
    Restart Count:  0
    Requests:
      cpu:      100m
      memory:   90Mi
    Liveness:   http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=5
    Readiness:  http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=3
    Environment:
      POD_NAME:       ingress-nginx-controller-8cf5559f8-2fl97 (v1:metadata.name)
      POD_NAMESPACE:  ingress-nginx (v1:metadata.namespace)
      LD_PRELOAD:     /usr/local/lib/libmimalloc.so
    Mounts:
      /usr/local/certificates/ from webhook-cert (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-h4b99 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  webhook-cert:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  ingress-nginx-admission
    Optional:    false
  kube-api-access-h4b99:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              kubernetes.io/os=linux
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From                      Message
  ----    ------     ----  ----                      -------
  Normal  Scheduled  37m   default-scheduler         Successfully assigned ingress-nginx/ingress-nginx-controller-8cf5559f8-2fl97 to kub-worker-1
  Normal  Pulled     37m   kubelet                   Container image "k8s.gcr.io/ingress-nginx/controller:v1.0.0@sha256:0851b34f69f69352bf168e6ccf30e1e20714a264ab1ecd1933e4d8c0fc3215c6" already present on machine
  Normal  Created    37m   kubelet                   Created container controller
  Normal  Started    37m   kubelet                   Started container controller
  Normal  RELOAD     37m   nginx-ingress-controller  NGINX reload triggered due to a change in configuration 

Name:                     ingress-nginx-controller
Namespace:                ingress-nginx
Labels:                   app.kubernetes.io/component=controller
                          app.kubernetes.io/instance=ingress-nginx
                          app.kubernetes.io/managed-by=Helm
                          app.kubernetes.io/name=ingress-nginx
                          app.kubernetes.io/version=1.0.0
                          helm.sh/chart=ingress-nginx-4.0.1
Annotations:              <none>
Selector:                 app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
Type:                     NodePort
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.110.109.139
IPs:                      10.110.109.139
Port:                     http  80/TCP
TargetPort:               http/TCP
NodePort:                 http  30261/TCP
Endpoints:                10.0.0.95:80
Port:                     https  443/TCP
TargetPort:               https/TCP
NodePort:                 https  32461/TCP
Endpoints:                10.0.0.95:443
Session Affinity:         None
External Traffic Policy:  Cluster
Events:
  Type    Reason  Age   From                Message
  ----    ------  ----  ----                -------
  Normal  Type    11m   service-controller  NodePort -> LoadBalancer
  Normal  Type    10m   service-controller  LoadBalancer -> NodePort
  • Current state of ingress object, if applicable:
NAME                  CLASS    HOSTS                      ADDRESS        PORTS   AGE
ingress-linkerd-web   <none>   linkerd.internal.damn.li   192.168.2.86   80      6d18h

Name:             ingress-linkerd-web
Namespace:        linkerd-viz
Address:          192.168.2.86
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host                      Path  Backends
  ----                      ----  --------
  linkerd.internal.damn.li  
                            /   web:8084 (10.0.0.251:8084)
Annotations:                kubernetes.io/ingress.class: nginx
                            nginx.ingress.kubernetes.io/configuration-snippet:
                              proxy_set_header Origin "";
                              proxy_hide_header l5d-remote-ip;
                              proxy_hide_header l5d-server-id; 
                            nginx.ingress.kubernetes.io/upstream-vhost: $service_name.$namespace.svc.cluster.local:8084
Events:
  Type    Reason  Age                From                      Message
  ----    ------  ----               ----                      -------
  Normal  Sync    48m                nginx-ingress-controller  Scheduled for sync
  Normal  Sync    44m (x2 over 45m)  nginx-ingress-controller  Scheduled for sync
  Normal  Sync    40m (x2 over 41m)  nginx-ingress-controller  Scheduled for sync

From host:

*   Trying 192.168.2.52:80...
* TCP_NODELAY set
* connect to 192.168.2.52 port 80 failed: Connection refused
* Failed to connect to linkerd.internal.damn.li port 80: Connection refused
* Closing connection 0
curl: (7) Failed to connect to linkerd.internal.damn.li port 80: Connection refused

From within ingress controller pod:

*   Trying 127.0.0.1:80...
* Connected to localhost (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: linkerd.internal.damn.li
> User-Agent: curl/7.78.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Thu, 26 Aug 2021 08:47:40 GMT
< Content-Type: text/html
< Content-Length: 835
< Connection: keep-alive
< x-content-type-options: nosniff
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< 

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width">
    <title>Linkerd</title>
    <meta name="description" content="Linkerd">
    <meta name="keywords" content="Linkerd">
    <link rel="icon" type="image/png" href="/dist/img/favicon.png">
    <link href="https://fonts.googleapis.com/css?family=Lato:300,400,700,900" rel="stylesheet">
    
  
    <script type="text/javascript" src="/dist/index_bundle.js" async></script>
  

  </head>
  <body>
    
  <div class="main" id="main"
    data-release-version="stable-2.10.2"
    data-go-version="go1.16.2"
    data-controller-namespace="linkerd"
    data-uuid="0766d708-1cdd-4225-a4b5-c587b503c3e6"
    data-grafana="grafana.linkerd-viz.svc.cluster.local:3000"
    data-jaeger="">
    
  </div>

  </body>
</html>
* Connection #0 to host localhost left intact

What happened: Resources are created but port is not exposed on host:

systemd-r     899 systemd-resolve   13u  IPv4    21166      0t0  TCP 127.0.0.53:53 (LISTEN)
sshd        18827            root    3u  IPv4    52153      0t0  TCP *:22 (LISTEN)
sshd        18827            root    4u  IPv6    52155      0t0  TCP *:22 (LISTEN)
container   20083            root   13u  IPv4    60776      0t0  TCP 127.0.0.1:37531 (LISTEN)
kube-cont   34531            root    7u  IPv4   107006      0t0  TCP 127.0.0.1:10257 (LISTEN)
etcd        34589            root    7u  IPv4   106830      0t0  TCP 192.168.2.52:2380 (LISTEN)
etcd        34589            root    8u  IPv4   106834      0t0  TCP 127.0.0.1:2379 (LISTEN)
etcd        34589            root    9u  IPv4   106835      0t0  TCP 192.168.2.52:2379 (LISTEN)
etcd        34589            root   13u  IPv4   107720      0t0  TCP 127.0.0.1:2381 (LISTEN)
kube-sche   34603            root    7u  IPv4   107783      0t0  TCP 127.0.0.1:10259 (LISTEN)
kube-apis   34618            root    7u  IPv6   107757      0t0  TCP *:6443 (LISTEN)
kubelet     37098            root   13u  IPv4   114512      0t0  TCP 127.0.0.1:33915 (LISTEN)
kubelet     37098            root   31u  IPv6   114560      0t0  TCP *:10250 (LISTEN)
kubelet     37098            root   34u  IPv4   114566      0t0  TCP 127.0.0.1:10248 (LISTEN)
cilium-op  143227            root    7u  IPv4   563370      0t0  TCP 127.0.0.1:9891 (LISTEN)
cilium-op  143227            root    8u  IPv4   563375      0t0  TCP 127.0.0.1:9234 (LISTEN)
cilium-ag 3889033            root    7u  IPv4 15314874      0t0  TCP 127.0.0.1:9890 (LISTEN)
cilium-ag 3889033            root   23u  IPv6 15316092      0t0  TCP *:42707 (LISTEN)
cilium-ag 3889033            root   51u  IPv4 15318371      0t0  TCP 127.0.0.1:9876 (LISTEN)
cilium-ag 3889033            root   63u  IPv6 15319057      0t0  TCP *:4244 (LISTEN)
cilium-ag 3889033            root   67u  IPv6 15319066      0t0  TCP *:4240 (LISTEN)

What you expected to happen: Port is exposed on host.

How to reproduce it: Have a Ubuntu 20.04 machine and install ingress nginx for bare metal.
@darox darox added the kind/bug Categorizes issue or PR as related to a bug. label Aug 26, 2021
@k8s-ci-robot
Copy link
Contributor

@darox: This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority labels Aug 26, 2021
@kundan2707
Copy link
Contributor

/assign

@longwuyuan
Copy link
Contributor

longwuyuan commented Aug 26, 2021 •
edited
Loading

Thank you for the detailed information. It helps a lot to make progress.
You need to send your request to port number 30261 because your ingress-controller service is of type NodePort ;

ingress-nginx service/ingress-nginx-controller NodePort 10.110.109.139 80:30261/TCP,443:32461/TCP 35m app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx

Also, I suggest please close this issue and come discuss at kubernetes.slack.com in the ingress-nginx-users channel. You can get many developers and engineers looking at the problem report.

/remove-kind bug
/kind support

@k8s-ci-robot k8s-ci-robot added kind/support Categorizes issue or PR as a support question. and removed kind/bug Categorizes issue or PR as related to a bug. labels Aug 26, 2021
@darox darox closed this as completed Aug 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kundan2707 kundan2707

Labels
kind/support Categorizes issue or PR as a support question. needs-priority needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@longwuyuan @darox @k8s-ci-robot @kundan2707