Redirect dl.k8s.io traffic to a community-owned GCS bucket instead of kubernetes-release

[spiffxp]

Part of umbrella issues:

• to deprecate and migrate away from gs://kubernetes-release: Deprecate and migrate away from gs://kubernetes-release #2396
• to migrate away from GCP project google-containers: Migrate away from google.com GCP project google-containers #1571

Traffic gets to gs://kubernetes-release in one of two ways:

• uri's that have the bucket name hardcoded (e.g. https://cs.k8s.io/?q=kubernetes-release%5B%5E-%5D&i=nope&files=&repos=)
• dl.k8s.io uri's that redirect to the bucket (too many matches for cs.k8s.io to display, but e.g. used in kubernetes/kubernetes CHANGELOGs)

We discussed the idea of cutting over dl.k8s.io traffic first:

• perhaps we could end up cutting over a large chunk of traffic sooner vs. later
• this will help us better plan how to migrate traffic that has the bucket name hardcoded

The sketch is something like:

• decide which project/bucket to use?
• sync from old bucket to new bucket
• change dl.k8s.io to use new bucket instead of old bucket
• see what the delta in traffic is
• decide how to keep buckets in sync?

Which project/bucket to use:

• we have k8s-release hosting gs://k8s-release
• we could choose to use a bucket hosted in k8s-artifacts-prod (simplifies billing)
• we could choose to block until we have similar staging/promotion process for GCS artifacts

How to sync:

• https://cloud.google.com/storage-transfer is pretty simple/fast to setup, I would at least use this for initial sync
  • There is currently no gcloud command, but there is an API and some bindings (ref: https://cloud.google.com/storage-transfer/docs/create-manage-transfer-program#cloud-to-cloud)
  • It supports recurring jobs, but no more frequent than hourly (ref: https://cloud.google.com/storage-transfer/docs/reference/rest/v1/transferJobs#TransferJob)
  • It supports one-time jobs, with a limit of 1K per day per project (ref: https://cloud.google.com/storage-transfer/quotas)
• For ongoing sync we could
  • rely on hourly sync
  • periodically create one-time sync jobs more frequently
  • use gcs pubsub notifications to create sync jobs
  • alter release tooling to publish to multiple buckets
  • alter release tooling to create sync jobs

Seeing the delta in traffic:

• we'll be able to see an influx of traffic in our billing report
• @thockin and I need to decide if/how to share the existing traffic gs://kubernetes-release is getting

My preferences would be:

• delete/recreate the gs://k8s-release bucket in k8s-artifacts-prod
• use sts, rely on hourly sync job first, then prowjob that periodically creates one-time syncs

/assign @thockin
since you were part of the discussion
/assign @justaugustus @hasheddan
For @kubernetes/release-engineering input

/wg k8s-infra
/sig release
/sig testing
/area artifacts
/milestone v1.21

[justaugustus]

(Opened #1857 for dl.k8s.io/ci.)

[spiffxp]

I manually initiated a one-time transfer from gs://kubernetes-release/release to gs://k8s-release/release following instructions at https://cloud.google.com/storage-transfer/docs/create-manage-transfer-console to get a sense of size/cost of transfer

[spiffxp]

delete/recreate the gs://k8s-release bucket in k8s-artifacts-prod

@justaugustus I would still like us to do this before we consider flipping over dl.k8s.io for real

I could be convinced to flip over temporarily to get an estimate of the amount of traffic this flips, but we should ultimately be using a similar approach as we do for GCR, with promotion from a staging bucket etc.

/assign @saschagrunert @puerco @cpanato @jeremyrickard
other SIG Release leads for input

[ameukam]

Postponed to v1.28

/milestone v1.28

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[ameukam]

/remove-lifecycle stale

[BenTheElder]

We're still not always redirecting away from the bucket and we have most traffic going to the bucket directly (some of which is just skipping the redirect because users have come to depend on the GCS bucket).

We need to:
a) ensure dl.k8s.io never redirects to the bucket
b) switch publishing to a new bucket, restrict access to fastly only, and send users through dl.k8s.io => cdn.dl.k8s.io

[ameukam]

/milestone v1.29

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[ameukam]

/remove-lifecycle stale

[ameukam]

/milestone v1.31

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[ameukam]

/remove-lifecycle stale

[ameukam]

/milestone v1.32