Bump vendored code to match that of kubernetes-v1.10.0-alpha.3 #183

shashidharatd · 2018-01-08T14:57:24Z

Add versions in glide.yaml file to match the corresponding version from kubernetes-1.10.0-alpha.3 ~~1.9.2-beta.0~~
Updated vendor code to match kubernetes-1.10.0-alpha.3 ~~1.9.2-beta.0~~ and fix the usage in federation code

/cc @kubernetes/sig-multicluster-pr-reviews @irfanurrehman @marun

shashidharatd · 2018-01-09T09:42:48Z

@marun, @irfanurrehman, All known issues have been fixed and tests are green. PTAL. Thanks !

irfanurrehman · 2018-01-09T14:03:28Z

glide.yaml

+- package: k8s.io/metrics
+  version: kubernetes-1.9.2-beta.0
+- package: k8s.io/utils
+  version: aedf551cdb8b0119df3a19c65fde413a13b34997


Is this autofilled by glide? I see that some k8s.io repos still don't have tags mapping to a particular k8s release tag (perhaps not required). But I do not clearly understand how the compatibility mapping is maintained for these as of now. Does it make sense to have any discussion on this?

The version is not auto-filled by glide. It is manually added by looking into Godeps.Json from kubernetes repo for the specific version we are bumping. Most of the repos under kubernetes are tagged now for the release. Going forward we can write a simple tool to pick up the versions and populate the glide.yaml file.

Out of curiosity, why are you looking to manually control the version of dependencies from k8s rather than let glide pick them up from the godeps.json file? Glide walks the tree and when it finds a dependency it uses the information picked up along the way on dependency versions. If k8s lists a dependency revision in the godeps.json file before others list it in their metadata files it will use that version.

Imagine the case where you manually manage the version and k8s changes. Then you'll be out of sync with k8s on the version to use.

There are plenty of reasons to manually manage a dependency. I'm curious of the reason here.

@mattfarina, federation also imports some third party packages which might be imported by k8s version and if glide resorts to using latest version of the code for such packages. it might cause some incompatibility. ( I should admit i don't know how glide handles this scenario). To ensure that we are vendoring the exact same version of the package which kubernetes version uses, i resorted to explicitly specifying version in the package. kubernetes code makes up of the bulk of vendored code what we use. and we always rely on a particular kubernetes version (which is stable enough).
IMHO, In our case, Its just a bit of overhead to change all the version of the package which we vendor when we bump.
I would also do some experiments in glide and confirm, if we can do away not mentioning version for every package.

If you have kubernetes first in your glide.yaml than the dependency versions listed in the Kubernetes Godeps.json file will be used if no version is specified in the glide.yaml file. This is because Glide will start at the first one and iterate over the dependencies first to last in the glide.yaml file. When it picks up a version the first one wins (unless there are some semver ranges it can munge together which isn't the case with Godeps).

The ordering in Glide was an intentional decision to give developers some control.

Thanks for this helpful tip @mattfarina. i will update accordingly.

irfanurrehman · 2018-01-09T14:04:35Z

glide.yaml

  - pkg/watch
 - package: k8s.io/apiserver
+  version: kubernetes-1.9.2-beta.0


Its amazing that we now are at a point where a mapping tag is available for staging repos!

yes, most of the repos in kubernetes are tagged with release version.

Release versions have always been available, but my understanding is that federation development needs to be able to chase master to ensure compatibility over a given release cycle. Has that thinking changed?

i did not get why federation development needs to chase kubernetes master. we should ideally be jumping from one stable version to another stable version.

Stable release versions of federation should absolutely target stable release versions of kubernetes. I'm not sure that would be ideal for master, though. The risk of compatibility problems being introduced between federation and kubernetes during development of a release is greater if federation development only targets stable release versions vs updating a pin against master on a regular basis.

Partially agree with your concern, but we need to consider that we are consumers of the kubernetes code and it would make sense for us to vendor in stable code as opposed to under development code. In worse cases if something breaks because of compatibility issue we can resort to bumping to the particular commit which can fix the issue. So, IMHO, the bumping strategy should be from one stable release to another stable release.
Also if you think practically, we didn't face any such issues in past couple of months, given that our federated clusters in ci jobs ran with latest master k8s code.

There were many problems discovered by chasing trunk in previous releases, so I'm not sure a lack of recent evidence is justification for not doing so going forward. The danger in not discovering problems as soon as possible is that the code introducing breakage will be 'baked in' and more difficult to back out or change. The strategy for minimizing disruption to the federation development effort would be to only update to a new version of k/k and its dependencies once CI had validated its stability. Only periodic jobs that validate against k/k HEAD would ever be broken, minimizing disruption for federation developers.

That said, I'm not responsible for supporting the current codebase (v1) so do what you think is best. For v2 (which should not depend on k/k), I will advocate strongly for keeping vendored staging repos as close to master as possible.

irfanurrehman · 2018-01-09T14:10:33Z

glide.yaml

@@ -1,15 +1,19 @@
 package: k8s.io/federation
 import:
 - package: cloud.google.com/go
+  version: 3b1ae45394a234c385be014e9a488f2bb6eef821


Any specific reason that we need to add the specific hash for this and many other repos? Probably @marun can comment with his experience of using glide earlier.

just added the version to be extra careful to use the same version of the package used by corresponding kubernetes version. if we don't use version and not pinned to the same version used by kubernetes, there are possibilities that we might use the latest version of the package which might have syntactical or semantic changes which may cause problems for our build.

I'm assuming glide should be able to specify the versions automatically? I hope that it isn't necessary to hand-curate the versions of vendored dependencies, that seems onerous and error-prone.

i think, glide does not fill the version automatically and always picks the latest commits in each package. It would be foolproof, if we pin the dependent package version to that of what is used in kubernetes version. There will be no compatibility issues and also kubernetes undergoes lot more testing than federation code. I agree that it is error-prone to hand-curate the versions, but its a one time activity and some 10-20 packages that we need to fill it by looking in Godeps.json file from corresponding kubernetes version.

irfanurrehman · 2018-01-09T14:21:18Z

test/integration/framework/api.go

 		return true, nil
 	})
 	if err != nil {
 		return fmt.Errorf("Timed out waiting for the %s: %v", apiNoun, err)
 	}
+
+	runOptions.InsecureServing.BindPort = port
+	runOptions.SecureServing.ServerCert.CertDirectory = "/tmp"


Will this be created if /tmp does not exist or not mounted? os packages TmpDir() can be used or taken as an example.

no "/tmp" will not be created but only used here. This is kind of test fixture, /tmp will be available on most root file systems and is meant for temporary usage which is world writable. I just did the simplest thing necessary, to fix the issue.

I don't think it's desirable to use /tmp here, since tests run in parallel may step on one another. A new temp directory as Irfan suggests would be preferable, and that directory should be cleaned up on teardown.

okie, updated.

irfanurrehman · 2018-01-09T14:24:28Z

Thanks @shashidharatd for this. I have few minor comments, otherwise looks good to me. @marun please have a look at this, given your first hand experience at initial vendoring.

irfanurrehman · 2018-01-12T18:21:54Z

@marun, will u be able to have a quick look at this ?

k8s-ci-robot · 2018-02-09T16:22:42Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: shashidharatd

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these OWNERS Files:

~~OWNERS~~ [shashidharatd]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

shashidharatd · 2018-02-13T08:51:12Z

/test pull-federation-e2e-gce

shashidharatd · 2018-02-20T05:23:34Z

Closing this in favour of #237

k8s-ci-robot added sig/multicluster Categorizes an issue or PR as relevant to sig-multicluster. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jan 8, 2018

shashidharatd force-pushed the bump-k8s-v1.9.2-beta.0 branch 2 times, most recently from f9c8354 to 9606128 Compare January 8, 2018 17:15

shashidharatd changed the title ~~Bump vendored code to match that of kubernetes-v1.9.2-beta.0~~ [WIP] Bump vendored code to match that of kubernetes-v1.9.2-beta.0 Jan 9, 2018

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 9, 2018

shashidharatd force-pushed the bump-k8s-v1.9.2-beta.0 branch 3 times, most recently from 94dcf26 to effbdfb Compare January 9, 2018 08:19

shashidharatd changed the title ~~[WIP] Bump vendored code to match that of kubernetes-v1.9.2-beta.0~~ Bump vendored code to match that of kubernetes-v1.9.2-beta.0 Jan 9, 2018

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 9, 2018

irfanurrehman reviewed Jan 9, 2018

View reviewed changes

shashidharatd force-pushed the bump-k8s-v1.9.2-beta.0 branch from effbdfb to b3d3f7c Compare January 17, 2018 05:45

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 17, 2018

shashidharatd changed the title ~~Bump vendored code to match that of kubernetes-v1.9.2-beta.0~~ Bump vendored code to match that of kubernetes-v1.9.2 Jan 22, 2018

shashidharatd force-pushed the bump-k8s-v1.9.2-beta.0 branch 2 times, most recently from 35f157d to 55a4bc7 Compare January 22, 2018 12:02

marun mentioned this pull request Jan 23, 2018

Workaround gazelle moving to a new repo #172

Merged

irfanurrehman mentioned this pull request Feb 6, 2018

Federation Provider for Azuredns #219

Merged

shashidharatd added 3 commits February 9, 2018 21:43

Fix an issue hack/update-bazel.sh

51c4632

Install gazelle from bazelbuild/bazel-gazelle instead of rules_go

cef50b9

Fix an issue in integration test framework

720cc4a

shashidharatd force-pushed the bump-k8s-v1.9.2-beta.0 branch from 55a4bc7 to 33ead9c Compare February 9, 2018 16:22

shashidharatd force-pushed the bump-k8s-v1.9.2-beta.0 branch from 33ead9c to 9359675 Compare February 12, 2018 05:21

shashidharatd added 10 commits February 12, 2018 11:13

Adapt to code changes in kubernetes-1.10.0-alpha.3

62bf3c6

Admission controller related changes

0b13cd8

Correct import path for coredns package

ddc3bf6

Regenerate and fix glide.yaml to use k8s-1.10.0-alpha.3

a29c58d

Remove git clean from hack/prune-deps.sh

7549d4b

Update vendor, prune deps and regenerate bazel build files

1d95cc8

Auto generated bazel build files on non-vendored code

3d0bfd1

Changes from running hack/update-all.sh

fe2c773

Update bazel build files by refering to k8s repo

e49d16b

Update build dir by refering to k8s repo

4caf45b

shashidharatd force-pushed the bump-k8s-v1.9.2-beta.0 branch from 9359675 to 4caf45b Compare February 12, 2018 06:46

shashidharatd changed the title ~~Bump vendored code to match that of kubernetes-v1.9.2~~ Bump vendored code to match that of kubernetes-v1.10.0-alpha.3 Feb 12, 2018

shashidharatd closed this Feb 20, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bump vendored code to match that of kubernetes-v1.10.0-alpha.3 #183

Bump vendored code to match that of kubernetes-v1.10.0-alpha.3 #183

shashidharatd commented Jan 8, 2018 •

edited

Loading

shashidharatd commented Jan 9, 2018

irfanurrehman Jan 9, 2018

shashidharatd Jan 10, 2018

mattfarina Jan 17, 2018

shashidharatd Jan 18, 2018

mattfarina Jan 18, 2018

shashidharatd Jan 19, 2018

irfanurrehman Jan 9, 2018

shashidharatd Jan 10, 2018

marun Jan 16, 2018

shashidharatd Jan 17, 2018

marun Jan 17, 2018

shashidharatd Jan 18, 2018

marun Jan 18, 2018

irfanurrehman Jan 9, 2018

shashidharatd Jan 10, 2018

marun Jan 16, 2018

shashidharatd Jan 17, 2018

irfanurrehman Jan 9, 2018 •

edited

Loading

shashidharatd Jan 10, 2018

marun Jan 16, 2018

shashidharatd Jan 17, 2018

irfanurrehman commented Jan 9, 2018

irfanurrehman commented Jan 12, 2018

k8s-ci-robot commented Feb 9, 2018

shashidharatd commented Feb 13, 2018

shashidharatd commented Feb 20, 2018

Bump vendored code to match that of kubernetes-v1.10.0-alpha.3 #183

Bump vendored code to match that of kubernetes-v1.10.0-alpha.3 #183

Conversation

shashidharatd commented Jan 8, 2018 • edited Loading

shashidharatd commented Jan 9, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

irfanurrehman Jan 9, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

irfanurrehman commented Jan 9, 2018

irfanurrehman commented Jan 12, 2018

k8s-ci-robot commented Feb 9, 2018

shashidharatd commented Feb 13, 2018

shashidharatd commented Feb 20, 2018

shashidharatd commented Jan 8, 2018 •

edited

Loading

irfanurrehman Jan 9, 2018 •

edited

Loading