Broken testEmptyProjection in BigQuery connector

[ebyhr]

Error:  io.trino.plugin.bigquery.TestBigQueryAvroConnectorTest.testEmptyProjection[SNAPSHOT, CREATE SNAPSHOT TABLE %s CLONE tpch.region, DROP SNAPSHOT TABLE %s](5)  Time elapsed: 0.456 s  <<< FAILURE!
com.google.cloud.bigquery.BigQueryException: Cannot have more than 1000 active clones and snapshots of a table. Please use a deep copy, or remove some active clones or snapshots.
	at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:115)
	at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.queryRpc(HttpBigQueryRpc.java:720)
	at com.google.cloud.bigquery.BigQueryImpl$35.call(BigQueryImpl.java:1322)
	at com.google.cloud.bigquery.BigQueryImpl$35.call(BigQueryImpl.java:1319)
	at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103)
	at com.google.cloud.bigquery.BigQueryRetryHelper.run(BigQueryRetryHelper.java:86)
	at com.google.cloud.bigquery.BigQueryRetryHelper.runWithRetries(BigQueryRetryHelper.java:49)
	at com.google.cloud.bigquery.BigQueryImpl.queryRpc(BigQueryImpl.java:1318)
	at com.google.cloud.bigquery.BigQueryImpl.query(BigQueryImpl.java:1306)
	at io.trino.plugin.bigquery.BigQueryQueryRunner$BigQuerySqlExecutor.executeQuery(BigQueryQueryRunner.java:149)
	at io.trino.plugin.bigquery.BigQueryQueryRunner$BigQuerySqlExecutor.execute(BigQueryQueryRunner.java:143)
	at io.trino.plugin.bigquery.BaseBigQueryConnectorTest.onBigQuery(BaseBigQueryConnectorTest.java:995)
	at io.trino.plugin.bigquery.BaseBigQueryConnectorTest.testEmptyProjection(BaseBigQueryConnectorTest.java:227)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:851)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1177)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
POST https://www.googleapis.com/bigquery/v2/projects/sep-bq-cicd/queries
{
  "code": 400,
  "errors": [
    {
      "domain": "global",
      "message": "Cannot have more than 1000 active clones and snapshots of a table. Please use a deep copy, or remove some active clones or snapshots.",
      "reason": "invalid"
    }
  ],
  "message": "Cannot have more than 1000 active clones and snapshots of a table. Please use a deep copy, or remove some active clones or snapshots.",
  "status": "INVALID_ARGUMENT"
}
	at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
	at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
	at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$3.interceptResponse(AbstractGoogleClientRequest.java:466)
	at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:552)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:493)
	at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:603)
	at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.queryRpc(HttpBigQueryRpc.java:718)
	... 24 more

https://github.com/trinodb/trino/actions/runs/5525099575/jobs/10080465642?pr=17667

[ebyhr]

cc: @wendigo @hashhar

[wendigo]

It's not flaky, it's broken entirely

[ebyhr]

The test passed recently https://github.com/trinodb/trino/actions/runs/5550808994/jobs/10136759388

[ebyhr]

https://github.com/trinodb/trino/actions/runs/5603944874/jobs/10251456079

[hashhar]

I'll rewrite the test to clone a newly created random table instead. It seems to be a BigQuery issue:

Doing a CREATE SNAPSHOT TABLE test.snapshot CLONE tpch.region in BigQuery console fails with same error. But cloning some other table works.
Also listing any existing clones doesn't show any: SELECT * FROM test.INFORMATION_SCHEMA.TABLES WHERE table_type = 'SNAPSHOT TABLE'.

Additionally performing a snapshot using the BigQuery console UI (there's a SNAPSHOT button in the details page for each table) seems to succeed.