Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support new registry option #45

Merged
merged 3 commits into from
Jul 12, 2018
Merged

support new registry option #45

merged 3 commits into from
Jul 12, 2018

Conversation

kwmonroe
Copy link
Member

Let users specify the docker registry they wish to use when deploying
addons. This facilitates offline deployments.

Let users specify the docker registry they wish to use when deploying
addons. This facilitates offline deployments.
@kwmonroe kwmonroe requested a review from Cynerva July 11, 2018 20:31

content = content.replace("cdkbot", "{{ registry|default('cdkbot') }}")
content = content.replace("k8s.gcr.io", "{{ registry|default('k8s.gcr.io') }}")
content = content.replace("nvidia", "{{ registry|default('nvidia') }}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The content-wide replacement of nvidia is scary. Consider nvidia-device-plugin.yml - replacing nvidia in here will butcher the template pretty bad.

Not that the existing code in here is a shining example of anything. The same concern applies to content-wide replacements of amd64, etc. But the nvidia case seems particularly bad - I don't think we can ship this.

Doing a replace on something like image: nvidia/ might be more acceptable, but still dirty. Probably more ideal would be to yaml.load the data, walk through and find the image fields, and only modify those.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DOH! great catch @Cynerva. I put 'nvidia' in there as an afterthought (not testing on gpu-enabled hardware). i'll play around with the 'image: ' munging.

@kwmonroe
Copy link
Member Author

kwmonroe commented Jul 11, 2018

@Cynerva yaml.load_all proved difficult to walk templates with multi docs (like heapster-controller) because pyyaml gets confused on templated items like {{ foo }}.

I pushed a version that does the replacement on image: foo/, and to be less dirty, i did it with re.sub so we catch instances where the templates have wacky spacing between image: and foo. How does this sit with you?

Edit: i know it's inefficient to keep doing re.sub, but I didn't feel that these 3 subs that only happen at snap build time warranted a new method with a more complicated re compilation (iow, i got tired of reading stack overflow).

@kwmonroe
Copy link
Member Author

kwmonroe commented Jul 11, 2018

FYI, testing looks like this. Deploy cdk with k8s-master using a cdk-addons snap with this PR built-in, check the addon images:

root@ip-172-31-9-89:~/snap/cdk-addons/x1/addons# grep image *
heapster-controller.yaml:        image: k8s.gcr.io/heapster-amd64:v1.5.3
heapster-controller.yaml:        image: k8s.gcr.io/heapster-amd64:v1.5.3
heapster-controller.yaml:        image: cdkbot/addon-resizer-amd64:1.8.1
heapster-controller.yaml:        image: cdkbot/addon-resizer-amd64:1.8.1
influxdb-grafana-controller.yaml:      - image: k8s.gcr.io/heapster-influxdb-amd64:v1.3.3
influxdb-grafana-controller.yaml:        image: k8s.gcr.io/heapster-grafana-amd64:v4.4.3
kube-dns.yaml:        image: k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.10
kube-dns.yaml:        image: k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.10
kube-dns.yaml:        image: k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.10
kubernetes-dashboard.yaml:        image: k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3
metrics-server-deployment.yaml:        image: k8s.gcr.io/metrics-server-amd64:v0.2.1
metrics-server-deployment.yaml:        image: cdkbot/addon-resizer-amd64:1.8.1

Set the cdk-addons registry with sudo snap set cdk-addons registry='wat:5000', re-run cdk-addons.apply and check again:

root@ip-172-31-9-89:~/snap/cdk-addons/x1/addons# grep image *
heapster-controller.yaml:        image: wat:5000/heapster-amd64:v1.5.3
heapster-controller.yaml:        image: wat:5000/heapster-amd64:v1.5.3
heapster-controller.yaml:        image: wat:5000/addon-resizer-amd64:1.8.1
heapster-controller.yaml:        image: wat:5000/addon-resizer-amd64:1.8.1
influxdb-grafana-controller.yaml:      - image: wat:5000/heapster-influxdb-amd64:v1.3.3
influxdb-grafana-controller.yaml:        image: wat:5000/heapster-grafana-amd64:v4.4.3
kube-dns.yaml:        image: wat:5000/k8s-dns-kube-dns-amd64:1.14.10
kube-dns.yaml:        image: wat:5000/k8s-dns-dnsmasq-nanny-amd64:1.14.10
kube-dns.yaml:        image: wat:5000/k8s-dns-sidecar-amd64:1.14.10
kubernetes-dashboard.yaml:        image: wat:5000/kubernetes-dashboard-amd64:v1.8.3
metrics-server-deployment.yaml:        image: wat:5000/metrics-server-amd64:v0.2.1
metrics-server-deployment.yaml:        image: wat:5000/addon-resizer-amd64:1.8.1

@hyperbolic2346
Copy link
Contributor

LGTM

@ktsakalozos
Copy link
Member

LGTM +1

@ktsakalozos
Copy link
Member

We might want this pushed to 1.11 branch of cdk addons

@Cynerva
Copy link
Contributor

Cynerva commented Jul 12, 2018

LGTM

We might want this pushed to 1.11 branch of cdk addons

Good catch, thanks. Merging this to master will only affect CI builds for 1.12 and onward. We probably want to cherry-pick this into release-1.11, and possibly release-1.10 and release-1.9 as well. @kwmonroe leaving you in charge of making that happen.

@Cynerva Cynerva merged commit 6ae66d3 into master Jul 12, 2018
k8s-github-robot pushed a commit to kubernetes/kubernetes that referenced this pull request Jul 12, 2018
Automatic merge from submit-queue (batch tested with PRs 66095, 66092). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

make the addons docker registry configurable

**What this PR does / why we need it**:
Allow users to configure the docker registry used when applying cdk-addons templates.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
Depends on charmed-kubernetes/cdk-addons#45

**Release note**:

```release-note
Expose docker registry config for addons used in Juju deployments
```
@kwmonroe kwmonroe deleted the feature/registry branch July 12, 2018 21:22
kwmonroe added a commit that referenced this pull request Jul 12, 2018
* support new registry option

Let users specify the docker registry they wish to use when deploying
addons. This facilitates offline deployments.

* dont be so loose with content replacement

* gah! forgot the content!
kwmonroe added a commit that referenced this pull request Jul 12, 2018
* support new registry option

Let users specify the docker registry they wish to use when deploying
addons. This facilitates offline deployments.

* dont be so loose with content replacement

* gah! forgot the content!
kwmonroe added a commit that referenced this pull request Jul 12, 2018
* support new registry option

Let users specify the docker registry they wish to use when deploying
addons. This facilitates offline deployments.

* dont be so loose with content replacement

* gah! forgot the content!
ktsakalozos added a commit that referenced this pull request Jul 13, 2018
ktsakalozos added a commit that referenced this pull request Jul 13, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
4 participants