Unable to create a Cloud Run service on GKE

[IkechukwuAKalu]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

Google Cloud Run has two service platforms; a fully-managed platform and another running on GKE. Using Terraform, the fully-managed platform works well but that for GKE does not work well yet. This is because Terraform does not currently provide a way to specify that a Cloud Run service should be run on a GKE cluster. We would really appreciate it if this feature is added.

Affected Resource

• google_cloud_run_service

Potential Terraform Configuration

resource "google_cloud_run_service" "default" {
  name = "test-cloud-run-srvc"

  platform = "GKE"
  cluster  = google_container_cluster.primary.name
  
  ...
}

References

• Cloud Run for Anthos #5054
• GCP setup documentation

Initially asked question

Please is there a way to create a Cloud Run service to run on GKE because I can't seem to find how to do it.

Also, the issue here didn't seem to state a resolution.

If you have got this working or know how to go about it, kindly point me in the right direction. Thanks

[venkykuberan]

Currently terraform provider don't support deploying Cloud Run service on to a GKE Cluster. Please raise a feature request for it, Team will triage it and work on it . You can add a little more details and change this an enhancement ticket.

[IkechukwuAKalu]

@venkykuberan thank you for your quick response. I have modified the issue to an enhancement ticket.

[upodroid]

@mbrancato

This can be implemented in Terraform but it is going to be messy.

https://cloud.google.com/run/docs/reference/rest/v1/namespaces.services

You need to supply ca cert of the cluster, namespace, cluster endpoint(needs to be reachable from where Terraform is running), sufficient IAM roles(Kubernetes Engine Developer or more) on the cluster.

# Anthos API Call
REDACTED  MCW0CDP3YY  ~  $  http GET https://34.90.175.88/apis/serving.knative.dev/v1/namespaces/default/services "Authorization: Bearer $(gcloud auth print-access-token)" --verify no
HTTP/1.1 200 OK
Audit-Id: 6d80a7ce-990e-4706-935f-9a6f050b78ca
Cache-Control: no-cache, private
Content-Length: 1034
Content-Type: application/json
Date: Wed, 30 Dec 2020 22:05:05 GMT

{
    "apiVersion": "serving.knative.dev/v1",
    "items": [
        {
            "apiVersion": "serving.knative.dev/v1",
            "kind": "Service",
            "metadata": {
                "annotations": {
                    "client.knative.dev/user-image": "us-docker.pkg.dev/cloudrun/container/hello",
                    "run.googleapis.com/client-name": "cloud-console"
                },
                "creationTimestamp": "2020-12-30T21:47:58Z",
                "generation": 1,
                "labels": {
                    "serving.knative.dev/visibility": "cluster-local"
                },
                "name": "dev",
                "namespace": "default",
                "resourceVersion": "28914563",
                "selfLink": "/apis/serving.knative.dev/v1/namespaces/default/services/dev",
                "uid": "8b8277d6-355c-4b28-b6bc-5560ddcd9b1c"
            },
            "spec": {
                "template": {
                    "metadata": {
                        "annotations": {
                            "run.googleapis.com/client-name": "cloud-console"
                        },
                        "name": "dev-00001-mey"
                    },
                    "spec": {
                        "containerConcurrency": 80,
                        "containers": [
                            {
                                "image": "us-docker.pkg.dev/cloudrun/container/hello",
                                "ports": [
                                    {
                                        "containerPort": 8080
                                    }
                                ],
                                "resources": {
                                    "limits": {
                                        "memory": "256Mi"
                                    }
                                }
                            }
                        ],
                        "timeoutSeconds": 300
                    }
                }
            }
        }
    ],
    "kind": "ServiceList",
    "metadata": {
        "continue": "",
        "resourceVersion": "28920766",
        "selfLink": "/apis/serving.knative.dev/v1/namespaces/default/services"
    }
}

# Cloud Run Managed
 REDACTED  MCW0CDP3YY  ~  $  http GET https://europe-west4-run.googleapis.com/apis/serving.knative.dev/v1/namespaces/REDACTED/services "Authorization: Bearer $(gcloud auth print-access-token)"
HTTP/1.1 200 OK
Alt-Svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Encoding: gzip
Content-Type: application/json; charset=UTF-8
Date: Wed, 30 Dec 2020 22:14:21 GMT
Server: ESF
Transfer-Encoding: chunked
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 0

{
    "apiVersion": "serving.knative.dev/v1",
    "items": [
        {
            "apiVersion": "serving.knative.dev/v1",
            "kind": "Service",
            "metadata": {
                "annotations": {
                    "client.knative.dev/user-image": "gcr.io/cloudrun/hello",
                    "run.googleapis.com/client-name": "cloud-console",
                    "serving.knative.dev/creator": "REDACTED",
                    "serving.knative.dev/lastModifier": "REDACTED"
                },
                "creationTimestamp": "2020-10-07T11:01:50.200821Z",
                "generation": 1,
                "labels": {
                    "cloud.googleapis.com/location": "europe-west4"
                },
                "name": "hello",
                "namespace": "678632180571",
                "resourceVersion": "AAWxEqb4Da0",
                "selfLink": "/apis/serving.knative.dev/v1/namespaces/678632180571/services/hello",
                "uid": "39cbc1dc-4ef4-4ebf-90ff-a77f7bd63231"
            },
            "spec": {
                "template": {
                    "metadata": {
                        "annotations": {
                            "autoscaling.knative.dev/maxScale": "1000",
                            "run.googleapis.com/client-name": "cloud-console"
                        },
                        "name": "hello-00001-qux"
                    },
                    "spec": {
                        "containerConcurrency": 80,
                        "containers": [
                            {
                                "image": "gcr.io/cloudrun/hello",
                                "ports": [
                                    {
                                        "containerPort": 8080
                                    }
                                ],
                                "resources": {
                                    "limits": {
                                        "cpu": "1000m",
                                        "memory": "256Mi"
                                    }
                                }
                            }
                        ],
                        "serviceAccountName": "678632180571-compute@developer.gserviceaccount.com",
                        "timeoutSeconds": 300
                    }
                },
                "traffic": [
                    {
                        "latestRevision": true,
                        "percent": 100
                    }
                ]
            },
            "status": {
                "address": {
                    "url": "https://hello-4txm7cz3ca-ez.a.run.app"
                },
                "conditions": [
                    {
                        "lastTransitionTime": "2020-10-07T11:02:32.355245Z",
                        "status": "True",
                        "type": "Ready"
                    },
                    {
                        "lastTransitionTime": "2020-10-07T11:02:31.956106Z",
                        "status": "True",
                        "type": "ConfigurationsReady"
                    },
                    {
                        "lastTransitionTime": "2020-10-07T11:02:32.355245Z",
                        "status": "True",
                        "type": "RoutesReady"
                    }
                ],
                "latestCreatedRevisionName": "hello-00001-qux",
                "latestReadyRevisionName": "hello-00001-qux",
                "observedGeneration": 1,
                "traffic": [
                    {
                        "latestRevision": true,
                        "percent": 100,
                        "revisionName": "hello-00001-qux"
                    }
                ],
                "url": "https://hello-4txm7cz3ca-ez.a.run.app"
            }
        }
    ],
    "kind": "ServiceList",
    "metadata": {
        "resourceVersion": "1602068552355245",
        "selfLink": "/apis/serving.knative.dev/v1/namespaces/678632180571/services"
    }
}

gcloud.

 REDACTED  MCW0CDP3YY  ~  $  gcloud run services list --platform gke --log-http --cluster anthos --cluster-location europe-west4-b --project REDACTED
=======================
==== request start ====
uri: https://container.googleapis.com/v1/projects/REDACTED/locations/europe-west4-b/clusters/anthos?alt=json
method: GET
== headers start ==
b'accept': b'application/json'
b'accept-encoding': b'gzip, deflate'
b'authorization': --- Token Redacted ---
b'content-length': b'0'
b'user-agent': b'google-cloud-sdk gcloud/321.0.0 command/gcloud.run.services.list invocation-id/924fa6876d2e4d03bb72bf9c5be5308f environment/None environment-version/None interactive/True from-script/False python/3.8.5 term/xterm-256color (Macintosh; Intel Mac OS X 19.6.0)'
== headers end ==
== body start ==

== body end ==
==== request end ====
---- response start ----
status: 200
-- headers start --
-content-encoding: gzip
alt-svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
cache-control: private
content-length: 4536
content-location: https://container.googleapis.com/v1/projects/REDACTED/locations/europe-west4-b/clusters/anthos?alt=json
content-type: application/json; charset=UTF-8
date: Wed, 30 Dec 2020 21:49:42 GMT
server: ESF
transfer-encoding: chunked
vary: Origin, X-Origin, Referer
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-xss-protection: 0
-- headers end --
-- body start --
{
  "name": "anthos",
  "masterAuth": {
    "clusterCaCertificate": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURLakNDQWhLZ0F3SUJBZ0lRSkNqOVRGSXBNYVFad0VaUU83bzBkekFOQmdrcWhraUc5dzBCQVFzRkFEQXYKTVMwd0t3WURWUVFERXlRMU9ETm1ObUUxTUMwek1EY3hMVFJpTTJZdFlXRmlNUzA0T0RRMk1EVTFZVEExWVRFdwpIaGNOTWpBeE1URXdNVEl6TlRNMldoY05NalV4TVRBNU1UTXpOVE0yV2pBdk1TMHdLd1lEVlFRREV5UTFPRE5tCk5tRTFNQzB6TURjeExUUmlNMll0WVdGaU1TMDRPRFEyTURVMVlUQTFZVEV3Z2dFaU1BMEdDU3FHU0liM0RRRUIKQVFVQUE0SUJEd0F3Z2dFS0FvSUJBUUN3Qzc3a2pHbUs1UUVYQzR1L1dUUjZuNEI0Zi8zZUlnVThxR0xUeEY1ZwpTeGdrUmJpSE5JdTUrR3FDOWRSb2Y0UVFQbXVqRzM0YjNqY2ZFVVZHVWRvd0xUU2RDVHloYkxxMXQwOHlGWWErCjgySzVvaDhvRVVpSWJXUkRnM3A2cmIvcGFQQW5VSUcrWWVVUjdleVBQNFNTamZHNFVoa1hCbDVuenVCRmlHTmcKVlBQazVOMnd4TGRacGs2QlBld3BXNW1XalViRWM1VEJUMFRGelVZb1FYc0lUVlprWjgvVmxCSnVLcHRiZ1lCRwpEdTVOY1V4RjhvSk1raW1jS1FOaDZ4Unl3MTdCRmorLy9odUR6M0wxZXdXU2tOUmtxNXNVTUkvUkRiUXhrM3FkClU5V1phbUNGc3BsRVFtN3dFRm54NEZNTzl2bjJKREs3MTNpcWIzdTZNckp6QWdNQkFBR2pRakJBTUE0R0ExVWQKRHdFQi93UUVBd0lDQkRBUEJnTlZIUk1CQWY4RUJUQURBUUgvTUIwR0ExVWREZ1FXQkJRTmEyV09SUXU2UnBVcgpaNExlNy9YcmJwQUZuREFOQmdrcWhraUc5dzBCQVFzRkFBT0NBUUVBanRDTC8vcHVVT2Y2OGFRMC9PNXl1Yi9yClk2eEtlN2l2cU54QVN1UklMZzRLUGJzaTJBUDQzL2wwSERUcDZHUjBnMEpXVHNlU0gxVFJrZWY2M05ZWHgyZVYKZmtCVURZQ2FwOW4wdENub21Vakx2a0RxWlUwb2NUalBEeG1reDlHc1BjRTRjVmV5OUMxbnpnbnhaZFYwSzZOZwpaMHZNdytGNkFZZFJacHhUZGN6cHhLeEh0Z2VJL3hMVTMxTGN4UzBEbGx6d0hxemE4eGxialA5enUwSzFNRjFYCmg1aWt2RktXNHQwT1pQNXBxaXVXcjY5SElBZjJhYi9zNHpKUWV6QmtBS2xNTWxab2ViZHpZS3ZMdEh5bHBIQWEKb1JVV3FkZHU0dnJDTmtqSFJqNUxqdGVacG1mLzNVd1U2cCtnaTFzcUdqdWlDMjlsS2Q0alpZT2M1M29HSUE9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg=="
  },
  "loggingService": "logging.googleapis.com/kubernetes",
  "monitoringService": "monitoring.googleapis.com/kubernetes",
  "network": "dev-net",
  "clusterIpv4Cidr": "10.201.64.0/18",
  "addonsConfig": {
    "httpLoadBalancing": {},
    "horizontalPodAutoscaling": {},
    "kubernetesDashboard": {
      "disabled": true
    },
    "networkPolicyConfig": {
      "disabled": true
    },
    "cloudRunConfig": {
      "loadBalancerType": "LOAD_BALANCER_TYPE_EXTERNAL"
    },
    "dnsCacheConfig": {},
    "configConnectorConfig": {},
    "gcePersistentDiskCsiDriverConfig": {
      "enabled": true
    }
  },
  "subnetwork": "dev-net-nl",
  "locations": [
    "europe-west4-b"
  ],
  "resourceLabels": {
    "asmv": "1-7-3-asm-6",
    "mesh_id": "proj-678632180571"
  },
  "labelFingerprint": "46325326",
  "legacyAbac": {},
  "networkPolicy": {},
  "ipAllocationPolicy": {
    "useIpAliases": true,
    "clusterIpv4Cidr": "10.201.64.0/18",
    "servicesIpv4Cidr": "10.201.0.0/18",
    "clusterSecondaryRangeName": "pods-1",
    "servicesSecondaryRangeName": "services-1",
    "clusterIpv4CidrBlock": "10.201.64.0/18",
    "servicesIpv4CidrBlock": "10.201.0.0/18"
  },
  "masterAuthorizedNetworksConfig": {
    "enabled": true,
    "cidrBlocks": [
      {
        "displayName": "external",
        "cidrBlock": "0.0.0.0/0"
      }
    ]
  },
  "maintenancePolicy": {
    "resourceVersion": "e3b0c442"
  },
  "autoscaling": {},
  "networkConfig": {
    "network": "projects/REDACTED/global/networks/dev-net",
    "subnetwork": "projects/REDACTED/regions/europe-west4/subnetworks/dev-net-nl"
  },
  "defaultMaxPodsConstraint": {
    "maxPodsPerNode": "110"
  },
  "resourceUsageExportConfig": {
    "bigqueryDestination": {
      "datasetId": "gke"
    },
    "enableNetworkEgressMetering": true,
    "consumptionMeteringConfig": {
      "enabled": true
    }
  },
  "authenticatorGroupsConfig": {},
  "privateClusterConfig": {
    "enablePrivateNodes": true,
    "masterIpv4CidrBlock": "10.247.0.0/28",
    "privateEndpoint": "10.247.0.2",
    "publicEndpoint": "34.90.175.88",
    "peeringName": "gke-nad8dd9719347663c115-92f1-b82c-peer",
    "masterGlobalAccessConfig": {
      "enabled": true
    }
  },
  "databaseEncryption": {
    "state": "DECRYPTED"
  },
  "shieldedNodes": {
    "enabled": true
  },
  "releaseChannel": {
    "channel": "REGULAR"
  },
  "workloadIdentityConfig": {
    "workloadPool": "REDACTED.svc.id.goog"
  },
  "selfLink": "https://container.googleapis.com/v1/projects/REDACTED/zones/europe-west4-b/clusters/anthos",
  "zone": "europe-west4-b",
  "endpoint": "34.90.175.88",
  "initialClusterVersion": "1.17.12-gke.1504",
  "currentMasterVersion": "1.17.13-gke.2001",
  "currentNodeVersion": "1.17.12-gke.1504",
  "createTime": "2020-11-10T13:35:36+00:00",
  "status": "RUNNING",
  "servicesIpv4Cidr": "10.201.0.0/18",
  "location": "europe-west4-b"
}

-- body end --
total round trip time (request+response): 0.241 secs
---- response end ----
----------------------
For cluster [anthos] in [europe-west4-b]:
=======================
==== request start ====
uri: https://34.90.175.88/apis/serving.knative.dev/v1/namespaces/default/services?alt=json
method: GET
== headers start ==
b'X-Goog-User-Project': b'REDACTED'
b'accept': b'application/json'
b'accept-encoding': b'gzip, deflate'
b'authorization': --- Token Redacted ---
b'content-length': b'0'
b'user-agent': b'google-cloud-sdk gcloud/321.0.0 command/gcloud.run.services.list invocation-id/a80836bb448b43fd8693f5791812a6c3 environment/None environment-version/None interactive/True from-script/False python/3.8.5 term/xterm-256color (Macintosh; Intel Mac OS X 19.6.0)'
== headers end ==
== body start ==

== body end ==
==== request end ====
---- response start ----
status: 200
-- headers start --
audit-id: cd8beeae-8fa1-4470-a192-aab4ab7d31ba
cache-control: no-cache, private
content-length: 1034
content-location: https://34.90.175.88/apis/serving.knative.dev/v1/namespaces/default/services?alt=json
content-type: application/json
date: Wed, 30 Dec 2020 21:49:43 GMT
-- headers end --
-- body start --
{"apiVersion":"serving.knative.dev/v1","items":[{"apiVersion":"serving.knative.dev/v1","kind":"Service","metadata":{"annotations":{"client.knative.dev/user-image":"us-docker.pkg.dev/cloudrun/container/hello","run.googleapis.com/client-name":"cloud-console"},"creationTimestamp":"2020-12-30T21:47:58Z","generation":1,"labels":{"serving.knative.dev/visibility":"cluster-local"},"name":"dev","namespace":"default","resourceVersion":"28914563","selfLink":"/apis/serving.knative.dev/v1/namespaces/default/services/dev","uid":"8b8277d6-355c-4b28-b6bc-5560ddcd9b1c"},"spec":{"template":{"metadata":{"annotations":{"run.googleapis.com/client-name":"cloud-console"},"name":"dev-00001-mey"},"spec":{"containerConcurrency":80,"containers":[{"image":"us-docker.pkg.dev/cloudrun/container/hello","ports":[{"containerPort":8080}],"resources":{"limits":{"memory":"256Mi"}}}],"timeoutSeconds":300}}}}],"kind":"ServiceList","metadata":{"continue":"","resourceVersion":"28915194","selfLink":"/apis/serving.knative.dev/v1/namespaces/default/services"}}

-- body end --
total round trip time (request+response): 0.131 secs
---- response end ----
----------------------
   SERVICE  NAMESPACE  URL  LAST DEPLOYED BY  LAST DEPLOYED AT
…  dev      default

[rileykarson]

We'll likely detail how to use the kubernetes_manifest resource to provision this instead: #8127

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!