Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear env var so Azure e2e generates a local keypair #19498

Merged
merged 1 commit into from
Oct 9, 2020

Conversation

mboersma
Copy link
Contributor

@mboersma mboersma commented Oct 8, 2020

To implement ClusterLogCollector for CAPZ using an SSH mechanism, Azure e2e tests need access to the private key as well. By leaving AZURE_SSH_PUBLIC_KEY_FILE empty, testing will fall back to generating a local keypair just for that run.

This works well as demonstrated in kubernetes-sigs/cluster-api-provider-azure#976.

Since the key materials aren't accessible outside of the prow CI environment, I don't think this introduces a security risk. But please let me know if you disagree or have any other feedback. cc: @nader-ziada @CecileRobertMichon @devigned

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 8, 2020
@k8s-ci-robot
Copy link
Contributor

Hi @mboersma. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Oct 8, 2020
@k8s-ci-robot k8s-ci-robot added area/config Issues or PRs related to code in /config area/jobs area/provider/azure Issues or PRs related to azure provider sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/testing Categorizes an issue or PR as relevant to SIG Testing. labels Oct 8, 2020
Copy link
Contributor

@devigned devigned left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

The key is short lived. I feel good about this change.

@k8s-ci-robot k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Oct 8, 2020
@CecileRobertMichon
Copy link
Member

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 8, 2020
@devigned
Copy link
Contributor

devigned commented Oct 8, 2020

/lgtm cancel

Looks like there is more work to be done.

@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 8, 2020
@devigned
Copy link
Contributor

devigned commented Oct 8, 2020

/approve cancel

@k8s-ci-robot k8s-ci-robot removed the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 8, 2020
@mboersma
Copy link
Contributor Author

mboersma commented Oct 8, 2020

I thought overriding the env var was the least disruptive way, but apparently that won't work.

I think either we could remove the preset-azure-cred label from the e2e task, or remove usage of AZURE_SSH_PUBLIC_KEY_FILE entirely. I don't see how either approach would break things, but let me look deeper.

@CecileRobertMichon
Copy link
Member

Pretty sure we need preset-azure-cred for the service principal creds

@mboersma mboersma force-pushed the remove-azure-pubkey branch from 34765e2 to 2b68cc0 Compare October 8, 2020 19:17
@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Oct 8, 2020
@mboersma
Copy link
Contributor Author

mboersma commented Oct 8, 2020

I made a new config label that retains everything e2e appears to need, but removes the AZURE_SSH_PUBLIC_KEY_FILE env var.

@devigned
Copy link
Contributor

devigned commented Oct 8, 2020

@mboersma what do you think about reusing preset-azure-cred-only? Just remove the ssh pub key from it. I don't think it's needed there.

@mboersma
Copy link
Contributor Author

mboersma commented Oct 8, 2020

what do you think about reusing preset-azure-cred-only?

Sure, but it appears we need REGISTRY as well (and may be expecting KUBE_VERBOSE=0). I can add those to preset-azure-cred-only instead if that's ok.

@devigned
Copy link
Contributor

devigned commented Oct 8, 2020

I think we should be ok with the default registry via https://github.com/kubernetes-sigs/cluster-api-provider-azure/blob/2e100d19787016a78130fd5b9440dedbad754c85/scripts/ci-e2e.sh#L53-L64.

We run other ci-e2e jobs with just preset-azure-cred-only.

@mboersma mboersma force-pushed the remove-azure-pubkey branch from 2b68cc0 to e2a8713 Compare October 8, 2020 20:00
Copy link
Contributor

@devigned devigned left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 8, 2020
@@ -38,7 +38,7 @@ presubmits:
labels:
preset-dind-enabled: "true"
preset-kind-volume-mounts: "true"
preset-azure-cred: "true"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are there any other jobs (eg. periodic CAPI e2e) that we're expecting to get logs from that should also be updated?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, looks like we should do the same for pull-cluster-api-provider-azure-capi-e2e--good thinking. Those are the two places that ./scripts/ci-e2e.sh is called.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Derp, nevermind--we are already there: that was why preset-azure-cred-only was added in the first place. So I think that covers it.

@CecileRobertMichon
Copy link
Member

/lgtm
/assign @chewong @dims

@dims
Copy link
Member

dims commented Oct 9, 2020

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: devigned, dims, mboersma

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 9, 2020
@k8s-ci-robot k8s-ci-robot merged commit 3cb0db1 into kubernetes:master Oct 9, 2020
@k8s-ci-robot k8s-ci-robot added this to the v1.20 milestone Oct 9, 2020
@k8s-ci-robot
Copy link
Contributor

@mboersma: Updated the following 2 configmaps:

  • job-config configmap in namespace default at cluster test-infra-trusted using the following files:
    • key cluster-api-provider-azure-presubmits.yaml using file config/jobs/kubernetes-sigs/cluster-api-provider-azure/cluster-api-provider-azure-presubmits.yaml
  • config configmap in namespace default at cluster test-infra-trusted using the following files:
    • key config.yaml using file config/prow/config.yaml

In response to this:

To implement ClusterLogCollector for CAPZ using an SSH mechanism, Azure e2e tests need access to the private key as well. By leaving AZURE_SSH_PUBLIC_KEY_FILE empty, testing will fall back to generating a local keypair just for that run.

This works well as demonstrated in kubernetes-sigs/cluster-api-provider-azure#976.

Since the key materials aren't accessible outside of the prow CI environment, I don't think this introduces a security risk. But please let me know if you disagree or have any other feedback. cc: @nader-ziada @CecileRobertMichon @devigned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@mboersma mboersma deleted the remove-azure-pubkey branch October 9, 2020 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/config Issues or PRs related to code in /config area/jobs area/provider/azure Issues or PRs related to azure provider cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants