Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use the relation object passed into get_column_values, instead of making our own #440

Merged
merged 3 commits into from
Nov 11, 2021
Merged

Use the relation object passed into get_column_values, instead of making our own #440

merged 3 commits into from
Nov 11, 2021

Conversation

joellabes
Copy link
Contributor

@joellabes joellabes commented Nov 9, 2021 •
edited
Loading

Resolves #424 - Mixed-quoting styles on Snowflake created an invalid Relation using the adapter

This is a:

  • bug fix PR with no breaking changes — please ensure the base branch is master

Checklist

  • I have verified that these changes work locally on the following warehouses (Note: it's okay if you do not have access to all warehouses, this helps us understand what has been covered)
    • BigQuery
    • Postgres
    • Redshift
    • Snowflake
  • I followed guidelines to ensure that my changes will work on "non-core" adapters by:
    • dispatching any new macro(s) so non-core adapters can also use them (e.g. the star() source)
    • using the limit_zero() macro in place of the literal string: limit 0
    • using dbt_utils.type_* macros instead of explicit datatypes (e.g. dbt_utils.type_timestamp() instead of TIMESTAMP
  • I have updated the README.md (if applicable)
  • I have added tests & descriptions to my models (and macros if applicable)
  • I have added an entry to CHANGELOG.md

@joellabes joellabes requested a review from jasnonaz November 9, 2021 21:29
@joellabes joellabes linked an issue Nov 9, 2021 that may be closed by this pull request
dbt_utils.get_column_values should use the Relation object it's given instead of making its own #424
Closed
5 tasks
jasnonaz
jasnonaz reviewed Nov 11, 2021
View reviewed changes
CHANGELOG.md
🚨 This is a compatibility release in preparation for `dbt-core` v1.0.0 (🎉). Projects using dbt-utils 0.7.4 with dbt-core v1.0.0 can expect to see a deprecation warning. This will be resolved in dbt_utils v0.8.0.

## Fixes
- `get_column_values()` now works correctly with mixed-quoting styles on Snowflake ([#424](https://github.com/dbt-labs/dbt-utils/issues/424), [#440](https://github.com/dbt-labs/dbt-utils/pull/440))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is nitpicky but should we say what we did / changed to make it work with mixed-quoting styles on Snowflake?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think release notes can be the what; people can click through to the PR for the how

jasnonaz
jasnonaz approved these changes Nov 11, 2021
View reviewed changes
Copy link
Contributor

@jasnonaz jasnonaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for sleuthing out this fix!

@joellabes joellabes merged commit 3a5d30e into next/patch Nov 11, 2021
@joellabes joellabes deleted the bugfix/424-relation-object branch November 11, 2021 01:44
joellabes added a commit that referenced this pull request Nov 11, 2021
@joellabes @TimoKruth
@fivetran-joemarkiewicz @dataders @sean-rose
dbt 0.7.4 release (#441)
b42e6f9 
* Update require-dbt-version to be 1.0

* Fix SQL 42000 on Exasol (#420)

" SQL-Error [42000]: syntax error, unexpected '*' "
If you specify the * in the unioned with their respectiv names <name>.* you do not receive the SQL Error posted above. This should not inflict any further problems since it is redundant for most DBs.

* Minor readme link fixes (#431)

* minor readme link fixes

* changelog addition

Co-authored-by: Joel Labes <joel.labes@dbtlabs.com>

* 0.7.4 changelog (#432)

* Update CHANGELOG.md

* Note branch name change

* use `limit_zero` macro instead of `limit 0` (#437)

* Utils 0.7.4b1  (#433)

* Update require-dbt-version to be 1.0

* Fix SQL 42000 on Exasol (#420)

" SQL-Error [42000]: syntax error, unexpected '*' "
If you specify the * in the unioned with their respectiv names <name>.* you do not receive the SQL Error posted above. This should not inflict any further problems since it is redundant for most DBs.

* Minor readme link fixes (#431)

* minor readme link fixes

* changelog addition

Co-authored-by: Joel Labes <joel.labes@dbtlabs.com>

* 0.7.4 changelog (#432)

* Update CHANGELOG.md

* Note branch name change

Co-authored-by: Timo Kruth <timo_kruth@gmx.de>
Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com>

* standard convention

* Update integration_tests/tests/jinja_helpers/test_slugify.sql

Taking the liberty of committing on your behalf so that the CI job starts again

* Change limit_zero to be a macro

Co-authored-by: Joel Labes <joel.labes@dbtlabs.com>
Co-authored-by: Timo Kruth <timo_kruth@gmx.de>
Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com>

* Add col_name alias to else state too (#437)

* Remove extra semicolon in `insert_by_period` materialization (#439)

* Remove extra semicolon in `insert_by_period` materialization.

`create_table_as()` generates a SQL statement that already ends with a semicolon, so the extra semicolon after a `create_table_as()` call in the `insert_by_period` materialization ends up being an empty SQL statement, and at least when using Snowflake this causes the dbt run to fail with a "cannot unpack non-iterable NoneType object" error.

* Update changelog for PR 439.

* Use the relation object passed into get_column_values, instead of making our own (#440)

* Use the relation object passed into get_column_values, instead of making our own

* Rename variables in get column value test to be clearer

* Update CHANGELOG.md

* Update CHANGELOG.md

Co-authored-by: Timo Kruth <timo_kruth@gmx.de>
Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com>
Co-authored-by: Anders <swanson.anders@gmail.com>
Co-authored-by: Sean Rose <sean.s.rose@gmail.com>
@joellabes joellabes mentioned this pull request Nov 18, 2021
Closed
1 task
@joellabes joellabes mentioned this pull request Nov 26, 2021
Merged
15 tasks
joellabes added a commit that referenced this pull request Dec 2, 2021
@joellabes @TimoKruth
@fivetran-joemarkiewicz @dataders @sean-rose
dbt v1 compat (#453)
baf4053 
* dbt 0.7.4 release (#441)

* Update require-dbt-version to be 1.0

* Fix SQL 42000 on Exasol (#420)

" SQL-Error [42000]: syntax error, unexpected '*' "
If you specify the * in the unioned with their respectiv names <name>.* you do not receive the SQL Error posted above. This should not inflict any further problems since it is redundant for most DBs.

* Minor readme link fixes (#431)

* minor readme link fixes

* changelog addition

Co-authored-by: Joel Labes <joel.labes@dbtlabs.com>

* 0.7.4 changelog (#432)

* Update CHANGELOG.md

* Note branch name change

* use `limit_zero` macro instead of `limit 0` (#437)

* Utils 0.7.4b1  (#433)

* Update require-dbt-version to be 1.0

* Fix SQL 42000 on Exasol (#420)

" SQL-Error [42000]: syntax error, unexpected '*' "
If you specify the * in the unioned with their respectiv names <name>.* you do not receive the SQL Error posted above. This should not inflict any further problems since it is redundant for most DBs.

* Minor readme link fixes (#431)

* minor readme link fixes

* changelog addition

Co-authored-by: Joel Labes <joel.labes@dbtlabs.com>

* 0.7.4 changelog (#432)

* Update CHANGELOG.md

* Note branch name change

Co-authored-by: Timo Kruth <timo_kruth@gmx.de>
Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com>

* standard convention

* Update integration_tests/tests/jinja_helpers/test_slugify.sql

Taking the liberty of committing on your behalf so that the CI job starts again

* Change limit_zero to be a macro

Co-authored-by: Joel Labes <joel.labes@dbtlabs.com>
Co-authored-by: Timo Kruth <timo_kruth@gmx.de>
Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com>

* Add col_name alias to else state too (#437)

* Remove extra semicolon in `insert_by_period` materialization (#439)

* Remove extra semicolon in `insert_by_period` materialization.

`create_table_as()` generates a SQL statement that already ends with a semicolon, so the extra semicolon after a `create_table_as()` call in the `insert_by_period` materialization ends up being an empty SQL statement, and at least when using Snowflake this causes the dbt run to fail with a "cannot unpack non-iterable NoneType object" error.

* Update changelog for PR 439.

* Use the relation object passed into get_column_values, instead of making our own (#440)

* Use the relation object passed into get_column_values, instead of making our own

* Rename variables in get column value test to be clearer

* Update CHANGELOG.md

* Update CHANGELOG.md

Co-authored-by: Timo Kruth <timo_kruth@gmx.de>
Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com>
Co-authored-by: Anders <swanson.anders@gmail.com>
Co-authored-by: Sean Rose <sean.s.rose@gmail.com>

* Regression: Correctly handle missing relations in get_column_values (#448)

* Create integration test for a dropped relation

* Update get_column_values.sql

* Swap out adapter call for a good old fashioned drop table

* Add missing curlies

* what person wrote this code :/ (it was me)

* wrap values in quotes

* GOOD

* bigquery compat (they don't like except)

* Backport android url changes from #426 (#452)

* Update CHANGELOG.md

* Change require-dbt-version, update dbt_project.yml for integration tests proj

* Upgrade python version in CI, improve drop relation integration test

* Clarify version pinning

* Drop support for release candidates of 1.0.0

Co-authored-by: Timo Kruth <timo_kruth@gmx.de>
Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com>
Co-authored-by: Anders <swanson.anders@gmail.com>
Co-authored-by: Sean Rose <sean.s.rose@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jasnonaz jasnonaz jasnonaz approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

dbt_utils.get_column_values should use the Relation object it's given instead of making its own
2 participants
@joellabes @jasnonaz