Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

infra/gcp/main: add k8s-project-metrics special-case #2442

Merged
merged 1 commit into from
Aug 2, 2021

Conversation

spiffxp
Copy link
Member

@spiffxp spiffxp commented Aug 2, 2021

Related:

k8s-metrics is a low-traffic GCS bucket for the project, setup permissions to trial moving it from one org to the other. The steps for this process are going to look something like:

  • allow google.com prow to write to the new bucket
  • allow humans to own the new bucket
  • sync contents from old bucket to new bucket
  • setup canary job on k8s-infra-prow-build-trusted that writes to the new
    bucket to confirm permissions are correct for:
    • executing bigquery queries that use the k8s-gubernator:builds dataset
    • writing to the new bucket
  • remove canary job / move old job to k8s-infra-prow-build-trusted
  • delete old bucket
  • rename new bucket to old bucket

If the rename fails, we'll use the new bucket name and look into a redirect from old files.

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: spiffxp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 2, 2021
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Aug 2, 2021
@spiffxp
Copy link
Member Author

spiffxp commented Aug 2, 2021

/cc @ameukam @dims
/cc @MushuEE @BenTheElder

k8s-metrics is a low-traffic GCS bucket for the project, setup
permissions to trial moving it from one org to the other. The steps for
this process are going to look something like:

- allow google.com prow to write to the new bucket
- allow k8s-infra-prow-build-trusted to write to the new bucket
- allow humans to own the new bucket
- sync contents from old bucket to new bucket
- setup canary job on k8s-infra-prow-build-trusted that writes to the new
  bucket to confirm permissions are correct for executing bigquery
  queries that use the k8s-gubernator:builds dataset and writing to the
  new bucket
- remove canary job / move old job to k8s-infra-prow-build-trusted
- delete old bucket
- rename new bucket to old bucket
@k8s-ci-robot k8s-ci-robot added area/terraform Terraform modules, testing them, writing more of them, code in infra/gcp/clusters/ area/prow Setting up or working with prow in general, prow.k8s.io, prow build clusters sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Aug 2, 2021
@ameukam
Copy link
Member

ameukam commented Aug 2, 2021

/lgtm
/hold
Remove hold when ready.

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 2, 2021
@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 2, 2021
@dims
Copy link
Member

dims commented Aug 2, 2021

/lgtm

@spiffxp spiffxp changed the title infra/gcp/main: add k8s-metrics-canary special-case infra/gcp/main: add k8s-project-metrics special-case Aug 2, 2021
@spiffxp
Copy link
Member Author

spiffxp commented Aug 2, 2021

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 2, 2021
@k8s-ci-robot k8s-ci-robot merged commit a075ae2 into kubernetes:main Aug 2, 2021
@k8s-ci-robot k8s-ci-robot added this to the v1.22 milestone Aug 2, 2021
@spiffxp
Copy link
Member Author

spiffxp commented Aug 2, 2021

Ensuring prow special cases for: kubernetes-public
  Special case: ensuring k8s-infra-ci-robot-github-token accessible by k8s-infra-prow-build-trusted
  Special case: ensuring gs://k8s-metrics-canary exists for gs://k8s-metrics migration
    Creating gs://k8s-project-metrics/...
    Enabling Bucket Policy Only for gs://k8s-project-metrics...
    @@ -1,3 +1,5 @@
    +- member: allUsers
    +  role: roles/storage.objectViewer
     - member: projectEditor:kubernetes-public
       role: roles/storage.legacyBucketOwner
     - member: projectOwner:kubernetes-public
    Setting lifecycle configuration on gs://k8s-project-metrics/...
    @@ -1 +1 @@
    -gs://k8s-project-metrics/ has no lifecycle configuration.
    +{"rule": [{"action": {"type": "Delete"}, "condition": {"age": 365}}]}
    Updated IAM policy for project [kubernetes-public].
    @@ -8,6 +8,8 @@
       role: roles/logging.privateLogViewer
     - member: group:gke-security-groups@kubernetes.io
       role: roles/monitoring.viewer
    +- member: group:k8s-infra-artifact-admins@kubernetes.io
    +  role: roles/viewer
     - member: group:k8s-infra-aws-admins@kubernetes.io
       role: roles/viewer
     - member: group:k8s-infra-bigquery-admins@kubernetes.io
    @@ -1,5 +1,7 @@
     - member: allUsers
       role: roles/storage.objectViewer
    +- member: group:k8s-infra-artifact-admins@kubernetes.io
    +  role: roles/storage.objectAdmin
     - member: projectEditor:kubernetes-public
       role: roles/storage.legacyBucketOwner
     - member: projectOwner:kubernetes-public
    @@ -1,5 +1,7 @@
     - member: allUsers
       role: roles/storage.objectViewer
    +- member: group:k8s-infra-artifact-admins@kubernetes.io
    +  role: roles/storage.legacyBucketOwner
     - member: group:k8s-infra-artifact-admins@kubernetes.io
       role: roles/storage.objectAdmin
     - member: projectEditor:kubernetes-public
    @@ -4,6 +4,8 @@
       role: roles/storage.legacyBucketOwner
     - member: group:k8s-infra-artifact-admins@kubernetes.io
       role: roles/storage.objectAdmin
    +- member: group:k8s-infra-prow-oncall@kubernetes.io
    +  role: roles/storage.objectAdmin
     - member: projectEditor:kubernetes-public
       role: roles/storage.legacyBucketOwner
     - member: projectOwner:kubernetes-public
    @@ -4,6 +4,8 @@
       role: roles/storage.legacyBucketOwner
     - member: group:k8s-infra-artifact-admins@kubernetes.io
       role: roles/storage.objectAdmin
    +- member: group:k8s-infra-prow-oncall@kubernetes.io
    +  role: roles/storage.legacyBucketOwner
     - member: group:k8s-infra-prow-oncall@kubernetes.io
       role: roles/storage.objectAdmin
     - member: projectEditor:kubernetes-public
    @@ -14,3 +14,5 @@
       role: roles/storage.legacyBucketOwner
     - member: projectViewer:kubernetes-public
       role: roles/storage.legacyBucketReader
    +- member: serviceAccount:triage@k8s-gubernator.iam.gserviceaccount.com
    +  role: roles/storage.objectAdmin
    @@ -14,5 +14,7 @@
       role: roles/storage.legacyBucketOwner
     - member: projectViewer:kubernetes-public
       role: roles/storage.legacyBucketReader
    +- member: serviceAccount:triage@k8s-gubernator.iam.gserviceaccount.com
    +  role: roles/storage.legacyBucketWriter@@ -14,6 +14,8 @@
       role: roles/storage.legacyBucketOwner
     - member: projectViewer:kubernetes-public
       role: roles/storage.legacyBucketReader
    +- member: serviceAccount:k8s-metrics@k8s-infra-prow-build-trusted.iam.gserviceaccount.com
    +  role: roles/storage.objectAdmin
     - member: serviceAccount:triage@k8s-gubernator.iam.gserviceaccount.com
       role: roles/storage.legacyBucketWriter
     - member: serviceAccount:triage@k8s-gubernator.iam.gserviceaccount.com
    @@ -14,6 +14,8 @@
       role: roles/storage.legacyBucketOwner
     - member: projectViewer:kubernetes-public
       role: roles/storage.legacyBucketReader
    +- member: serviceAccount:k8s-metrics@k8s-infra-prow-build-trusted.iam.gserviceaccount.com
    +  role: roles/storage.legacyBucketWriter
     - member: serviceAccount:k8s-metrics@k8s-infra-prow-build-trusted.iam.gserviceaccount.com
       role: roles/storage.objectAdmin
     - member: serviceAccount:triage@k8s-gubernator.iam.gserviceaccount.com     - member: serviceAccount:triage@k8s-gubernator.iam.gserviceaccount.com
       role: roles/storage.objectAdmin

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/prow Setting up or working with prow in general, prow.k8s.io, prow build clusters area/terraform Terraform modules, testing them, writing more of them, code in infra/gcp/clusters/ cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants