Try using SQL for create_schema

[jtcohen6]

resolves #182

I'd like to avoid duplicating code from SQLAdapter if we can help it, but I'm not sure of the right way to plumb through its methods without turning BigQueryAdapter into an unholy hybrid of BaseAdapter + SQLAdapter.

Checklist

• I have signed the CLA
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• I have updated the CHANGELOG.md and added information about my change to the "dbt-bigquery next" section.

[jtcohen6]

• Unit tests for create_schema + drop_schema fail, which isn't too surprising
• All our legacy integration tests (tests/integration) are failing as well. Just out of curiosity, I'm going to reorder to run the new functional tests (tests/functional) first, since that was working for me locally

[jtcohen6]

Cool! Per latest GHA run, this change seems to be working with our new testing framework, just not the legacy one. I'm going to close this PR for now, and consider it blocked by our ongoing test conversion work.

[jtcohen6]

The main value here is around converting create_schema, since that's what dbt runs at start-up. The only cases where dbt actually uses its own drop_schema method/macro are:

• clean up after integration/functional test
• clean up PR schemas in dbt Cloud

I'm running into a very confusing permissions issue at the intersection of the old testing framework and SQL drop schema. So, we could unblock this work for the meantime by moving ahead with just create schema, and opening a new tech debt ticket to track the unfinished work around drop schema. I believe that work will be unblocked once we've totally put to rest the old testing framework.

[McKnight-42]

This is very interesting, do you think we will eventually move completely away from using BaseAdapter for BigQuery?

[jtcohen6]

This is very interesting, do you think we will eventually move completely away from using BaseAdapter for BigQuery?

@McKnight-42 Very cool question! I think we're still a ways away from getting there, but it does seem like the direction we're heading in, slowly.

That said, work like #136 should convince us, if we weren't convinced already, that BigQuery's Python API might always be half a step ahead of its SQL API, in terms of performance / cost / reliability.

So, I could see us adding bits and pieces of the SQLAdapter here, until at some point we realize that it would be more expedient to inherit from the SQLAdapter instead, and reimplement the pieces that still need to be in Python. For today, I think the gaps are still pretty wide. In the meantime, I'd also say it's good, for our own edification, that our 5 maintained adapters offer a representative cross-section of the oddities that are out there.

[github-christophe-oudar]

I would add that it's indeed pretty frustrating that we can't rely only on SQL on BigQuery. #167 is yet another example on how BigQuery lacks consistency on that topic and makes it harder for its users to rely only on SQL (and that we need some smart custom dbt logic to make it easier to work with for dbt users until Google figures it out).

[McKnight-42]

@jtcohen6 do we plan on leaving the test commented out or is it okay to remove?

[jtcohen6]

Good catch - probably okay to remove. I don't think we can have a dedicated unit test like this (mocking a Python method) if dbt is running real SQL instead. I also don't feel like we need a dedicated test for create_schema, since it's a required "setup" step in every single functional test.

[McKnight-42]

This Looks fine to me!