Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix kubeadm upgrade regression #70893

Merged
merged 1 commit into from
Nov 13, 2018

Conversation

fabriziopandini
Copy link
Member

@fabriziopandini fabriziopandini commented Nov 9, 2018

What type of PR is this?
/kind bug

What this PR does / why we need it:
To fix kubeadm v1.13 error when upgrading clusters created with kubeadm v1.12.

Which issue(s) this PR fixes:
Fixes #kubernetes/kubeadm#1224

Special notes for your reviewer:
The error happens when connecting to etcd, and this regression was introduced by #69486

kubeadm v1.12 cluster are deployed with etcd listening on localhost only, while kubeadm v1.13 assumes etcd is listening both on localhost and on the advertising address.
This PR makes kubadm v1.13 support both cases.

Does this PR introduce a user-facing change?:

NONE

/sig cluster-lifecycle
/priority critical-urgent

/cc @timothysc
/cc @rdodev
/cc @neolit123
@kubernetes/sig-cluster-lifecycle-pr-reviews

@k8s-ci-robot k8s-ci-robot added sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Nov 9, 2018
@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. labels Nov 9, 2018
@k8s-ci-robot
Copy link
Contributor

@fabriziopandini: GitHub didn't allow me to request PR reviews from the following users: rdodev.

Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

What type of PR is this?
/kind bug

What this PR does / why we need it:
To fix kubeadm v1.13 error when upgrading clusters created with kubeadm v1.12.

Which issue(s) this PR fixes:
Fixes #kubernetes/kubeadm#1224

Special notes for your reviewer:
The error happens when connecting to etcd, and this regression was introduced by #69486

kubeadm v1.12 cluster are deployed with etcd listening on localhost only, while kubeadm v1.13 assumes etcd is listening both on localhost and on the advertising address.
This PR makes kubadm v1.13 support both cases.

Does this PR introduce a user-facing change?:
NONE

/sig cluster-lifecycle
/priority critical-urgent

/cc @timothysc
/cc @rdodev
/cc @neolit123
@kubernetes/sig-cluster-lifecycle-pr-reviews

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. area/kubeadm labels Nov 9, 2018
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fabriziopandini

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Nov 9, 2018
@fabriziopandini
Copy link
Member Author

@rdodev fyi

@rdodev
Copy link
Contributor

rdodev commented Nov 9, 2018

/LGTM

@k8s-ci-robot
Copy link
Contributor

@rdodev: changing LGTM is restricted to assignees, and only kubernetes/kubernetes repo collaborators may be assigned issues.

In response to this:

/LGTM

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

}
etcdContainer := etcdPod.Spec.Containers[0]
for _, arg := range etcdContainer.Command {
if arg == "--listen-client-urls=https://127.0.0.1:2379" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i guess there is no other way to determine if these pod manifests were created by kubeadm 1.12?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a fan of this solution, but considering it should go aways next cycle I think this can be accepted (the same hack is used elsewhere in kubeadm itself).
Nevertheless, if there are any better idea ...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we probably should just merge on monday unless there is further feedback.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for me.
In the future it may be handy to add a comment at the beginning of generated YAML files, stating that these files are autogenerated and the kubeadm version with which it was done. This will not only provide a start point for issues like this, but make users less tempted to tamper the generated YAMLs themselves, rather than edit the kubeadm config and rerun it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very good point @rosti, please file an issue for it!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@luxas @rosti what about adding an annotation too? comments are fine, but cannot be used by app

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for 1.12 -> 1.13 there is no way.

for later versions an annotation seem OK as long as we need it and/or want to maintain it..
annotating which pods are created with what kubeadm version seems like a nice to have to me.

kind: Pod
metadata:
  annotations:
    kubeadmVersion: <version-from-client>

version-from-client should be the same as kubeadm version -o=short
and be parsed using: https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/apimachinery/pkg/util/version/version.go#L111

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue created:
kubernetes/kubeadm#1231

@luxas luxas added this to the v1.13 milestone Nov 12, 2018
@neolit123
Copy link
Member

/lgtm
/hold

@k8s-ci-robot k8s-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Nov 12, 2018
@neolit123
Copy link
Member

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 12, 2018
@dims
Copy link
Member

dims commented Nov 13, 2018

@fabriziopandini @neolit123 - this PR has glog which fails in the CI job. Can you please s/glog/klog/?

@dims
Copy link
Member

dims commented Nov 13, 2018

/hold

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 13, 2018
@dims
Copy link
Member

dims commented Nov 13, 2018

/lgtm cancel

@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 13, 2018
@neolit123
Copy link
Member

this PR has glog which fails in the CI job

we obviously missed that. but the odd part is that i didn't see CI failures.

@fabriziopandini
Copy link
Member Author

@dims @neolit123 Thanks for the hint!
Fixed! If ok, please remove hold

@dims
Copy link
Member

dims commented Nov 13, 2018

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 13, 2018
@dims
Copy link
Member

dims commented Nov 13, 2018

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 13, 2018
@k8s-ci-robot k8s-ci-robot merged commit da33d8a into kubernetes:master Nov 13, 2018
@fabriziopandini fabriziopandini deleted the fix-stacked-etcd branch December 15, 2018 08:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/kubeadm cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. release-note-none Denotes a PR that doesn't merit a release note. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants