AWS: run k8s master in different account or different provider #39996

scheeles · 2017-01-17T07:52:39Z

Currently the master and the nodes must run in the same account. With this change the master can run in a different AWS account, on a different cloud provider or on premise.

Release Notes

AWS cloud provider: allow to run the master with a different AWS account or even on a different cloud provider than the nodes.

k8s-ci-robot · 2017-01-17T07:52:40Z

Hi @scheeles. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with @k8s-bot ok to test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

k8s-ci-robot · 2017-01-17T07:52:41Z

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://github.com/kubernetes/kubernetes/wiki/CLA-FAQ to sign the CLA.

Once you've signed, please reply here (e.g. "I signed it!") and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

k8s-ci-robot · 2017-01-17T07:52:41Z

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://github.com/kubernetes/kubernetes/wiki/CLA-FAQ to sign the CLA.

Once you've signed, please reply here (e.g. "I signed it!") and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

k8s-reviewable · 2017-01-17T07:52:45Z

This change is

scheeles · 2017-01-17T08:01:57Z

I signed it! Von: k8s-ci-robot <notifications@github.com> Antworten an: kubernetes/kubernetes <reply@reply.github.com> Datum: Montag, 16. Januar 2017 um 23:53 An: kubernetes/kubernetes <kubernetes@noreply.github.com> Cc: Sebastian <scheele.s@web.de>, Mention <mention@noreply.github.com> Betreff: Re: [kubernetes/kubernetes] AWS: run k8s master in different account or on a provider (#39996) Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). 📝 Please follow instructions at https://github.com/kubernetes/kubernetes/wiki/CLA-FAQ to sign the CLA. Once you've signed, please reply here (e.g. "I signed it!") and we'll verify. Thanks.

…

________________________________ * If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits<https://help.github.com/articles/setting-your-email-in-git/>. * If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized. Instructions for interacting with me using PR comments are available here<https://github.com/kubernetes/community/blob/master/contributors/devel/pull-request-commands.md>. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra<https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:> repository. I understand the commands that are listed here<https://github.com/kubernetes/test-infra/blob/master/prow/commands.md>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#39996 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AJlO0PMpvue1nsUXYHnZvZP_DIcv3jhvks5rTHOCgaJpZM4LlU7b>.

sttts · 2017-01-17T10:23:19Z

pkg/cloudprovider/providers/aws/aws.go

-		// Maybe if we're not running on AWS, e.g. bootstrap; for now it is not very useful
+		// This flag enables the possibility to run the master components on a different
+		// aws account, on a different cloud provider or on premise.
+		// If the flag is set also Zone, VPCID and KubernetesClusterTag must be provided


nit: If the flag is set

sttts · 2017-01-17T10:25:18Z

pkg/cloudprovider/providers/aws/aws.go

+	} else if cfg.Global.ExternalMaster == true && (cfg.Global.VPCID == "" || cfg.Global.Zone == "" || cfg.Global.KubernetesClusterTag == "") {
+		// For the external master the Zone, VPCID and KubernetesClusterTag must be set.
+		// It is not possible to detect it
+		return nil, fmt.Errorf("Run with eternal Master but Zone: %s or VPCID: %s or KubernetesClusterTag: %s is not set", cfg.Global.Zone, cfg.Global.VPCID, cfg.Global.KubernetesClusterTag)


justinsb

I think we want to remove the AZ specification

justinsb · 2017-01-18T05:07:32Z

pkg/cloudprovider/providers/aws/aws.go

+	metadata         EC2Metadata
+	cfg              *CloudConfig
+	region           string
+	availabilityZone string


This is odd, because we can be multi-AZ

justinsb · 2017-01-18T05:20:19Z

pkg/cloudprovider/providers/aws/aws.go

@@ -1090,7 +1112,7 @@ func (c *Cloud) getAllZones() (sets.String, error) {
 // GetZone implements Zones.GetZone
 func (c *Cloud) GetZone() (cloudprovider.Zone, error) {
 	return cloudprovider.Zone{
-		FailureDomain: c.selfAWSInstance.availabilityZone,
+		FailureDomain: c.availabilityZone,


I don't think this is what we want. It appears to be used in two places:

pkg/controller/service/servicecontroller.go: zone, err := zones.GetZone() pkg/kubelet/kubelet_node_status.go: zone, err := zones.GetZone()

For kubelet node status, we want to continue to label with the node's zone, not the master's zone.

For servicecontroller, zone is actually unused since be9ce30 I believe. I have proposed it for removal in #40060

Hence I think we want to keep it returning selfAWSInstance.

sttts · 2017-01-22T14:43:52Z

pkg/cloudprovider/providers/aws/aws.go

@@ -389,6 +390,11 @@ type CloudConfig struct {
 		// Maybe if we're not running on AWS, e.g. bootstrap; for now it is not very useful
 		Zone string

+		// The aws VPC flag enables the possibility to run the master components


k8s-ci-robot · 2017-03-01T17:03:55Z

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://github.com/kubernetes/kubernetes/wiki/CLA-FAQ to sign the CLA.

Once you've signed, please reply here (e.g. "I signed it!") and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

scheeles · 2017-03-01T17:04:49Z

@zmerlynn @justinsb Sorry fixed it now

justinsb · 2017-03-01T17:07:14Z

/lgtm

k8s-github-robot · 2017-03-01T17:52:34Z

[APPROVALNOTIFIER] This PR is APPROVED

The following people have approved this PR: justinsb, k8s-merge-robot, scheeles, sttts

Needs approval from an approver in each of these OWNERS Files:

~~pkg/cloudprovider/providers/aws/OWNERS~~ [justinsb]

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

sttts · 2017-03-01T18:22:51Z

@k8s-bot gci gce e2e test this

justinsb · 2017-03-01T18:59:22Z

@k8s-bot gci gce e2e test this
@k8s-bot cvm gce e2e test this

k8s-github-robot · 2017-03-01T23:51:05Z

Automatic merge from submit-queue

k8s-ci-robot · 2017-03-02T00:24:48Z

@scheeles: The following test(s) failed:

Test name	Commit	Details	Rerun command
Jenkins GCI GCE e2e	`0be5e60`	link	`@k8s-bot gci gce e2e test this`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Automatic merge from submit-queue (batch tested with PRs 43925, 42512) AWS: add KubernetesClusterID as additional option when VPC is set This is a small enhancement after the PRs #41695 and #39996 ## Release Notes ```release-note AWS cloud provider: allow to set KubernetesClusterID or KubernetesClusterTag in combination with VPC. ```

…bnetid-routetableid Automatic merge from submit-queue Specify subnetid and routetableid via cloud provider config **What this PR does / why we need it**: This is a fix for #39996 which is needed since 1.6 Changes introduced from 1.6 broke partially(LoadBalancer) the support for running the master components in a different environment (different aws account/on premise). This PR will add support for specifying the Subnet & RouteTable to use via the cloud provider config. **Release note**: ```release-note AWS cloud provider: fix support running the master with a different AWS account or even on a different cloud provider than the nodes. ```

Automatic merge from submit-queue (batch tested with PRs 40060, 44860, 44865, 44825, 44162) servicecontroller: remove unused zone field The zone field was unused, and this complicated e.g. #39996 ```release-note NONE ```

The zone field was unused, and this complicated e.g. kubernetes#39996

Automatic merge from submit-queue (batch tested with PRs 43925, 42512) AWS: add KubernetesClusterID as additional option when VPC is set This is a small enhancement after the PRs kubernetes/kubernetes#41695 and kubernetes/kubernetes#39996 ## Release Notes ```release-note AWS cloud provider: allow to set KubernetesClusterID or KubernetesClusterTag in combination with VPC. ```

k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Jan 17, 2017

k8s-github-robot assigned justinsb Jan 17, 2017

k8s-github-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. release-note-label-needed labels Jan 17, 2017

sttts reviewed Jan 17, 2017

View reviewed changes

justinsb mentioned this pull request Jan 18, 2017

servicecontroller: remove unused zone field #40060

Merged

justinsb suggested changes Jan 18, 2017

View reviewed changes

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Jan 22, 2017

sttts reviewed Jan 22, 2017

View reviewed changes

k8s-github-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 22, 2017

k8s-github-robot assigned zmerlynn and unassigned justinsb Jan 30, 2017

zmerlynn assigned justinsb and unassigned zmerlynn Jan 30, 2017

k8s-github-robot assigned zmerlynn and unassigned justinsb Jan 30, 2017

zmerlynn assigned justinsb and unassigned zmerlynn Jan 30, 2017

k8s-github-robot assigned zmerlynn and unassigned justinsb Jan 30, 2017

zmerlynn removed their assignment Jan 30, 2017

k8s-ci-robot removed the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Mar 1, 2017

k8s-ci-robot added cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Mar 1, 2017

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 1, 2017

k8s-github-robot merged commit 5c168e2 into kubernetes:master Mar 1, 2017

scheeles deleted the aws branch March 2, 2017 05:32

scheeles mentioned this pull request Mar 4, 2017

AWS: add KubernetesClusterID as additional option when VPC is set #42512

Merged

This was referenced Apr 7, 2017

Feature/specify aws subnetid routetableid #44210

Closed

Specify subnetid and routetableid via cloud provider config #44211

Closed

Specify subnetid and routetableid via cloud provider config #44235

Merged

jayunit100 pushed a commit to jayunit100/kubernetes that referenced this pull request Apr 25, 2017

servicecontroller: remove unused zone field

26ee4a5

The zone field was unused, and this complicated e.g. kubernetes#39996

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AWS: run k8s master in different account or different provider #39996

AWS: run k8s master in different account or different provider #39996

scheeles commented Jan 17, 2017 •

edited by sttts

Loading

k8s-ci-robot commented Jan 17, 2017

k8s-ci-robot commented Jan 17, 2017

k8s-ci-robot commented Jan 17, 2017

k8s-reviewable commented Jan 17, 2017

scheeles commented Jan 17, 2017 via email •

edited

Loading

sttts Jan 17, 2017

sttts Jan 17, 2017

justinsb left a comment

justinsb Jan 18, 2017

justinsb Jan 18, 2017

sttts Jan 22, 2017

k8s-ci-robot commented Mar 1, 2017

scheeles commented Mar 1, 2017

justinsb commented Mar 1, 2017

k8s-github-robot commented Mar 1, 2017

sttts commented Mar 1, 2017

justinsb commented Mar 1, 2017

k8s-github-robot commented Mar 1, 2017

k8s-ci-robot commented Mar 2, 2017

AWS: run k8s master in different account or different provider #39996

AWS: run k8s master in different account or different provider #39996

Conversation

scheeles commented Jan 17, 2017 • edited by sttts Loading

Release Notes

k8s-ci-robot commented Jan 17, 2017

k8s-ci-robot commented Jan 17, 2017

k8s-ci-robot commented Jan 17, 2017

k8s-reviewable commented Jan 17, 2017

scheeles commented Jan 17, 2017 via email • edited Loading

sttts Jan 17, 2017

Choose a reason for hiding this comment

sttts Jan 17, 2017

Choose a reason for hiding this comment

justinsb left a comment

Choose a reason for hiding this comment

justinsb Jan 18, 2017

Choose a reason for hiding this comment

justinsb Jan 18, 2017

Choose a reason for hiding this comment

sttts Jan 22, 2017

Choose a reason for hiding this comment

k8s-ci-robot commented Mar 1, 2017

scheeles commented Mar 1, 2017

justinsb commented Mar 1, 2017

k8s-github-robot commented Mar 1, 2017

sttts commented Mar 1, 2017

justinsb commented Mar 1, 2017

k8s-github-robot commented Mar 1, 2017

k8s-ci-robot commented Mar 2, 2017

scheeles commented Jan 17, 2017 •

edited by sttts

Loading

scheeles commented Jan 17, 2017 via email •

edited

Loading