Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove some deprecated macros #268

Merged
merged 4 commits into from
Aug 24, 2020
+29 −47
Merged

Remove some deprecated macros #268

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
818d793
Remove get_tables_by_prefix and union_tables (deprecated in v0.2.4 / …
clrcrl Aug 20, 2020
1323176
Remove get_tables_by_pattern (erroneously introduced in v0.5.0)
clrcrl Aug 22, 2020
7ae385d
Update changelog
clrcrl Aug 22, 2020
7aa8038
Remove crufty Jinja
clrcrl Aug 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,31 @@
# dbt-utils v0.6.0 (unreleased)

## Breaking changes
- :rotating_light: dbt v0.18.0 or greater is required for this release. If you are not ready to upgrade, consider using a previous release of this package
- :rotating_light: The `get_tables_by_prefix`, `union_tables` and `get_tables_by_pattern` macros have been removed

## Migration instructions
- Upgrade your dbt project to v0.18.0 using [these instructions](https://discourse.getdbt.com/t/prerelease-v0-18-0-marian-anderson/1545).
- Upgrade your `dbt_project.yml` file to use version `0.7.0`. Run `dbt clean` and `dbt deps`.
- If your project uses the `get_tables_by_prefix` macro, replace it with `get_relations_by_prefix`. All arguments have retained the same name.
- If your project uses the `union_tables` macro, replace it with `union_relations`. While the order of arguments has stayed consistent, the `tables` argument has been renamed to `relations`. Further, the default value for the `source_column_name` argument has changed from `'_dbt_source_table'` to `'_dbt_source_relation'` — you may want to explicitly define this argument to avoid breaking changes.

```
-- before:
{{ dbt_utils.union_tables(
tables=[ref('my_model'), source('my_source', 'my_table')],
exclude=["_loaded_at"]
) }}

-- after:
{{ dbt_utils.union_relations(
relations=[ref('my_model'), source('my_source', 'my_table')],
exclude=["_loaded_at"],
source_column_name='_dbt_source_table'
) }}
```
- If your project uses the `get_tables_by_pattern` macro, replace it with `get_tables_by_pattern_sql` — all arguments are consistent.

## Fixes

## Features
Expand All @@ -11,6 +37,8 @@ specific to their database (#267)
database adapters that use different prefixes (#267)

## Quality of life
* Remove deprecated macros `get_tables_by_prefix` and `union_tables` (#268)
* Remove `get_tables_by_pattern` macro, which is equivalent to the `get_tables_by_pattern_sql` macro (the latter has a more logical name)

# dbt-utils v0.5.1

Expand Down
5 changes: 0 additions & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -442,9 +442,6 @@ Usage:
...
```
#### get_relations_by_prefix
> This replaces the `get_tables_by_prefix` macro. Note that the `get_tables_by_prefix` macro will
be deprecated in a future release of this package.

Returns a list of [Relations](https://docs.getdbt.com/docs/writing-code-in-dbt/class-reference/#relation)
that match a given prefix, with an optional exclusion pattern. It's particularly
handy paired with `union_relations`.
Expand Down Expand Up @@ -516,8 +513,6 @@ from {{ref('my_model')}}
```

#### union_relations ([source](macros/sql/union.sql))
> This replaces the `union_tables` macro. Note that the `union_tables` macro will
be deprecated in a future release of this package.

This macro unions together an array of [Relations](https://docs.getdbt.com/docs/writing-code-in-dbt/class-reference/#relation),
even when columns have differing orders in each Relation, and/or some columns are
Expand Down
4 changes: 0 additions & 4 deletions integration_tests/models/sql/test_get_tables_by_prefix_and_union.sql
View file
Open in desktop

This file was deleted.

12 changes: 0 additions & 12 deletions macros/sql/get_relations_by_pattern.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,3 @@
{%- endif -%}

{% endmacro %}

{% macro get_tables_by_pattern(schema_pattern, table_pattern, exclude='', database=target.database) %}
{%- set error_message = '
Warning: the `get_tables_by_pattern` macro is no longer supported and will be deprecated in a future release of dbt-utils. \
Use the `get_relations_by_prefix` macro instead. \
The {}.{} model triggered this warning. \
'.format(model.package_name, model.name) -%}
{%- do exceptions.warn(error_message) -%}

{{ return(dbt_utils.get_relations_by_pattern(schema_pattern, table_pattern, exclude='', database=target.database)) }}

{% endmacro %}
12 changes: 0 additions & 12 deletions macros/sql/get_relations_by_prefix.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,3 @@
{%- endif -%}

{% endmacro %}

{% macro get_tables_by_prefix(schema, prefix, exclude='', database=target.database) %}
{%- set error_message = '
Warning: the `get_tables_by_prefix` macro is no longer supported and will be deprecated in a future release of dbt-utils. \
Use the `get_relations_by_prefix` macro instead. \
The {}.{} model triggered this warning. \
'.format(model.package_name, model.name) -%}
{%- do exceptions.warn(error_message) -%}

{{ return(dbt_utils.get_relations_by_prefix(schema, prefix, exclude, database)) }}

{% endmacro %}
15 changes: 1 addition & 14 deletions macros/sql/union.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{%- macro union_relations(relations, column_override=none, include=[], exclude=[], source_column_name=none) -%}
{%- macro union_relations(relations, column_override=none, include=[], exclude=[], source_column_name='_dbt_source_relation') -%}

{%- if exclude and include -%}
{{ exceptions.raise_compiler_error("Both an exclude and include list were provided to the `union` macro. Only one is allowed") }}
Expand All @@ -10,7 +10,6 @@
{% endif -%}

{%- set column_override = column_override if column_override is not none else {} -%}
{%- set source_column_name = source_column_name if source_column_name is not none else '_dbt_source_relation' -%}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jtcohen6 — can you just sense check this for me (related to line #1). I looked at it a number of times and can't figure out why it's like this.

I guess similarly, why do we have column_override=none instead of column_override={}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is because of the python convention to set default arguments to none, always, out of fear of mutable default arguments.

It's less of an issue with strings and dicts than with arrays, so I think it's probably fine to make this change.


{%- set relation_columns = {} -%}
{%- set column_superset = {} -%}
Expand Down Expand Up @@ -83,15 +82,3 @@
{%- endfor -%}

{%- endmacro -%}

{%- macro union_tables(tables, column_override=none, include=[], exclude=[], source_column_name='_dbt_source_table') -%}
{%- set error_message = '
Warning: the `union_tables` macro is no longer supported and will be deprecated in a future release of dbt-utils. \
Use the `union_relations` macro instead. \
The {}.{} model triggered this warning. \
'.format(model.package_name, model.name) -%}
{%- do exceptions.warn(error_message) -%}

{{ return(dbt_utils.union_relations(tables, column_override, include, exclude, source_column_name)) }}

{%- endmacro -%}