Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

close over the ingress variable or the last assignment will be used #433

Merged
merged 1 commit into from
Mar 14, 2017

Conversation

ibawt
Copy link
Contributor

@ibawt ibawt commented Mar 14, 2017

When you have more than one ingress per namespace and they are "mixed", the scoping on the ing variable will end up pointing to the last one assigned inside the go function.

This will cause the ingress to keep trying to update the ip and fight the GCE load balancer.

Without fix:

I0313 23:32:21.765558       7 launch.go:94] &{NGINX 0.9.0-beta.2 git-7013a52 git@github.com:ixdy/kubernetes-ingress.git}
I0313 23:32:21.765693       7 launch.go:97] Watching for ingress class: nginx
I0313 23:32:21.765635       7 nginx.go:112] starting NGINX process...
I0313 23:32:21.766427       7 launch.go:223] Creating API server client for https://....
I0313 23:32:21.799352       7 launch.go:113] validated kube-system/default-http-backend as the default backend
I0313 23:32:21.807605       7 controller.go:1032] starting Ingress controller
I0313 23:32:21.811920       7 leaderelection.go:247] lock is held by nginxinc-ingress-1580632739-cjsh1 and has not yet expired
I0313 23:32:21.812374       7 controller.go:169] ignoring add for ingress glbc based on annotation kubernetes.io/ingress.class
I0313 23:32:21.812547       7 event.go:217] Event(api.ObjectReference{Kind:"Ingress", Namespace:"tweety-production", Name:"nginx", UID:"9bde60b4-083a-11e7-af15-42010af000e0", APIVersion:"extensions", ResourceVersion:"3812359", FieldPath:""}): type: 'Normal' reason: 'CREATE' Ingress tweety-production/nginx
I0313 23:32:21.812620       7 event.go:217] Event(api.ObjectReference{Kind:"Ingress", Namespace:"kube-shell-production", Name:"web", UID:"227bcdbf-ff18-11e6-a328-42010af000e0", APIVersion:"extensions", ResourceVersion:"3812357", FieldPath:""}): type: 'Normal' reason: 'CREATE' Ingress kube-shell-production/web
I0313 23:32:21.812644       7 event.go:217] Event(api.ObjectReference{Kind:"Ingress", Namespace:"groundcontrol-production", Name:"web", UID:"03524766-ff14-11e6-a328-42010af000e0", APIVersion:"extensions", ResourceVersion:"3812358", FieldPath:""}): type: 'Normal' reason: 'CREATE' Ingress groundcontrol-production/web
W0313 23:32:22.812750       7 queue.go:87] requeuing groundcontrol-production/web, err deferring sync till endpoints controller has synced
I0313 23:32:31.862011       7 controller.go:423] ingress backend successfully reloaded...
I0313 23:32:34.757502       7 leaderelection.go:247] lock is held by nginxinc-ingress-1580632739-cjsh1 and has not yet expired
I0313 23:32:43.669990       7 leaderelection.go:247] lock is held by nginxinc-ingress-1580632739-cjsh1 and has not yet expired
I0313 23:32:52.046489       7 leaderelection.go:247] lock is held by nginxinc-ingress-1580632739-cjsh1 and has not yet expired
I0313 23:33:02.258168       7 leaderelection.go:247] lock is held by nginxinc-ingress-1580632739-cjsh1 and has not yet expired
I0313 23:33:14.399864       7 leaderelection.go:247] lock is held by nginxinc-ingress-1580632739-cjsh1 and has not yet expired
I0313 23:33:29.226732       7 leaderelection.go:247] lock is held by nginxinc-ingress-1580632739-cjsh1 and has not yet expired
I0313 23:33:38.658590       7 leaderelection.go:247] lock is held by nginxinc-ingress-1580632739-cjsh1 and has not yet expired
I0313 23:33:49.587678       7 leaderelection.go:247] lock is held by nginxinc-ingress-1580632739-cjsh1 and has not yet expired
I0313 23:33:59.958809       7 leaderelection.go:188] sucessfully acquired lease default/ingress-controller-leader
I0313 23:34:21.821281       7 status.go:261] updating Ingress tweety-production/nginx status to [{104.196.12.96 }]
I0313 23:34:21.822229       7 status.go:261] updating Ingress tweety-production/glbc status to [{104.196.12.96 }]
I0313 23:34:21.822803       7 status.go:261] updating Ingress kube-shell-production/web status to [{104.196.12.96 }]
I0313 23:34:21.823072       7 status.go:261] updating Ingress groundcontrol-production/web status to [{104.196.12.96 }]
I0313 23:34:21.837162       7 event.go:217] Event(api.ObjectReference{Kind:"Ingress", Namespace:"tweety-production", Name:"nginx", UID:"9bde60b4-083a-11e7-af15-42010af000e0", APIVersion:"extensions", ResourceVersion:"3812538", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress tweety-production/nginx
I0313 23:34:21.842893       7 event.go:217] Event(api.ObjectReference{Kind:"Ingress", Namespace:"groundcontrol-production", Name:"web", UID:"03524766-ff14-11e6-a328-42010af000e0", APIVersion:"extensions", ResourceVersion:"3812539", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress groundcontrol-production/web
I0313 23:34:21.846625       7 event.go:217] Event(api.ObjectReference{Kind:"Ingress", Namespace:"kube-shell-production", Name:"web", UID:"227bcdbf-ff18-11e6-a328-42010af000e0", APIVersion:"extensions", ResourceVersion:"3812540", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress kube-shell-production/web
I0313 23:34:51.819860       7 status.go:261] updating Ingress tweety-production/glbc status to [{XXX.XXX.XXX.XXX }]
I0313 23:35:21.818149       7 status.go:261] updating Ingress tweety-production/glbc status to [{XXX.XXX.XXX.XXX}]

with the fix

I0313 23:26:48.496510       8 launch.go:96] &{NGINX 0.9.0-beta.2 git-0fa2a32 git@github.com:ibawt/ingress}
I0313 23:26:48.496538       8 launch.go:99] Watching for ingress class: nginx
I0313 23:26:48.496756       8 launch.go:230] Creating API server client for https://...
I0313 23:26:48.524251       8 launch.go:115] validated kube-system/default-http-backend as the default backend
I0313 23:26:48.531273       8 controller.go:1071] starting Ingress controller
I0313 23:26:48.533452       8 leaderelection.go:247] lock is held by nginxinc-ingress-1394641404-vd9cc and has not yet expired
I0313 23:26:48.537702       8 controller.go:168] ignoring add for ingress glbc based on annotation kubernetes.io/ingress.class
I0313 23:26:48.537873       8 event.go:217] Event(api.ObjectReference{Kind:"Ingress", Namespace:"tweety-production", Name:"nginx", UID:"9bde60b4-083a-11e7-af15-42010af000e0", APIVersion:"extensions", ResourceVersion:"3811844", FieldPath:""}): type: 'Normal' reason: 'CREATE' Ingress tweety-production/nginx
I0313 23:26:48.537974       8 event.go:217] Event(api.ObjectReference{Kind:"Ingress", Namespace:"kube-shell-production", Name:"web", UID:"227bcdbf-ff18-11e6-a328-42010af000e0", APIVersion:"extensions", ResourceVersion:"3811845", FieldPath:""}): type: 'Normal' reason: 'CREATE' Ingress kube-shell-production/web
I0313 23:26:48.538005       8 event.go:217] Event(api.ObjectReference{Kind:"Ingress", Namespace:"groundcontrol-production", Name:"web", UID:"03524766-ff14-11e6-a328-42010af000e0", APIVersion:"extensions", ResourceVersion:"3811846", FieldPath:""}): type: 'Normal' reason: 'CREATE' Ingress groundcontrol-production/web
W0313 23:26:49.537187       8 queue.go:87] requeuing default/fluentd-agg-1, err deferring sync till endpoints controller has synced
I0313 23:26:58.792818       8 metrics.go:34] changing prometheus collector from  to default
I0313 23:26:58.849618       8 controller.go:439] ingress backend successfully reloaded...
I0313 23:27:01.479030       8 leaderelection.go:247] lock is held by nginxinc-ingress-1394641404-vd9cc and has not yet expired
I0313 23:27:10.390731       8 leaderelection.go:247] lock is held by nginxinc-ingress-1394641404-vd9cc and has not yet expired
I0313 23:27:18.770166       8 leaderelection.go:188] sucessfully acquired lease default/ingress-controller-leader
I0313 23:27:48.541974       8 status.go:271] updating Ingress tweety-production/nginx status to [{35.185.66.209 }]
I0313 23:27:48.542135       8 status.go:271] updating Ingress groundcontrol-production/web status to [{35.185.66.209 }]
I0313 23:27:48.542243       8 status.go:271] updating Ingress kube-shell-production/web status to [{35.185.66.209 }]
I0313 23:27:48.545968       8 event.go:217] Event(api.ObjectReference{Kind:"Ingress", Namespace:"groundcontrol-production", Name:"web", UID:"03524766-ff14-11e6-a328-42010af000e0", APIVersion:"extensions", ResourceVersion:"3811944", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress groundcontrol-production/web
I0313 23:27:48.552779       8 event.go:217] Event(api.ObjectReference{Kind:"Ingress", Namespace:"kube-shell-production", Name:"web", UID:"227bcdbf-ff18-11e6-a328-42010af000e0", APIVersion:"extensions", ResourceVersion:"3811946", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress kube-shell-production/web
I0313 23:27:48.552813       8 event.go:217] Event(api.ObjectReference{Kind:"Ingress", Namespace:"tweety-production", Name:"nginx", UID:"9bde60b4-083a-11e7-af15-42010af000e0", APIVersion:"extensions", ResourceVersion:"3811945", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress tweety-production/nginx

You can see it starting to fight the GCE one fairly quickly on the unpatched, and the delightful lack of ip updating in the patched one.

@k8s-ci-robot
Copy link
Contributor

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://github.com/kubernetes/kubernetes/wiki/CLA-FAQ to sign the CLA.

Once you've signed, please reply here (e.g. "I signed it!") and we'll verify. Thanks.


Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Mar 14, 2017
@k8s-reviewable
Copy link

This change is Reviewable

@ibawt
Copy link
Contributor Author

ibawt commented Mar 14, 2017

I signed it!

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Mar 14, 2017
@aledbf aledbf self-assigned this Mar 14, 2017
@aledbf
Copy link
Member

aledbf commented Mar 14, 2017

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 14, 2017
@coveralls
Copy link

Coverage Status

Coverage increased (+0.03%) to 43.066% when pulling 0fa2a32 on ibawt:fix-scoping into 448a42a on kubernetes:master.

@aledbf
Copy link
Member

aledbf commented Mar 14, 2017

@ibawt thanks!

@aledbf aledbf merged commit 7c17dc3 into kubernetes:master Mar 14, 2017
@ibawt
Copy link
Contributor Author

ibawt commented Mar 14, 2017

np!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants