CI: Fix Flakey GBQ Tests

[alimcmaster1]

-ref #30478 (comment)
We see the below in the logs:

google.api_core.exceptions.Conflict: 409 POST https://bigquery.googleapis.com/bigquery/v2/projects/pandas-travis/datasets: Already Exists: Dataset pandas-travis:pydata_pandas_bq_testing_py31

https://travis-ci.org/pandas-dev/pandas/jobs/631599036

Despite attempting to delete the dataset in the previous line.
self.client.delete_dataset(self.dataset, delete_contents=True)

Since we run with dist=loadfile these tests are run sequentially by pytest. But they could potentially clash across builds?

We now create a unique dataset name per test function and teardown when complete

GBQ Tests will run against my fork will post results on here.

cc. @jreback, @tswast

[WillAyd]

Hmm wouldn't this keep the dataset lingering around? Perhaps an alternate is to just split this into two fixtures and scope the dataset one to say the module level or even higher

[alimcmaster1]

Hmm wouldn't this keep the dataset lingering around? Perhaps an alternate is to just split this into two fixtures and scope the dataset one to say the module level or even higher

Sure if we don't want to keep the dataset - then an easy way is to give the dataset a random name and create/delete in this fixture. I've pushed the update.

[jreback]

hmm, are we supposed to clean up these datasets? @tswast

[alimcmaster1]

We now just do this here:

pandas/pandas/tests/io/test_gbq.py

Line 173 in a91b8a3

self.client.delete_dataset(self.dataset, delete_contents=True)

[jreback]

lgtm. very minor comment. ping on green.

[alimcmaster1]

nitpick can you make this a module level function

Sure done.

Link to my travis fork where these tests have passed: (since they only run on our master branch)

[jreback]

great thanks @alimcmaster1

[alimcmaster1]

Let me know if you see any more issues - thanks!

[jreback]

thanks @alimcmaster1