Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-2338] [Bug] {{this}} evaluates incorrectly with Snowflake when used in a macro #7237

Closed
2 tasks done
rory-donaldson opened this issue Mar 29, 2023 · 2 comments
Closed
2 tasks done
Labels
bug Something isn't working duplicate This issue or pull request already exists

Comments

@rory-donaldson
Copy link

rory-donaldson commented Mar 29, 2023

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

{{ this }} evaluates to the correct full path of the table in Snowflake when used directly in a model's pre-hook/post-hook.
When it is passed from a model as an argument to a macro, only the table name information is retained and the Snowflake schema and database information is lost.

Expected Behavior

When {{ this }} is passed from a model to the macro, the schema and database info is not lost.

Steps To Reproduce

When using {{ this }} directly in a post_hook, it correctly evaluates to the full path of the table in Snowflake.

{{ config(
    post_hook=[
        "grant select on {{this}} to role DW_GL"
    ]
)}}

In the above example, {{this}} would evaluate to PRESENTATION.B2B.MY_DBT_MODEL_NAME, which is as intended.
Now I want to take this logic and put it into a macro.

I created a macro as below:

{% macro grant_select(table_name, roles) %}
    -- Macro to grant select on table to one or more roles.

    {% for role in roles %}

        grant select on {{ table_name }} to role {{role}}
        
    {% endfor %}
{% endmacro %}

In the model I call the macro as below:

{{ config(
    post_hook = grant_select(this, ['DW_GL'])
)}}

Now when the macro runs, this is passed to the macro as the table_name variable.
However it now evaluates to RAW.PUBLIC.MY_DBT_MODEL_NAME.

RAW.PUBLIC is my default schema in Snowflake. The actual database and schema info is lost when passing through to the macro.

Relevant log output

08:00:50  Database Error in model vw_bfn_d365dimretailstore (models/presentation/bfn_presentation/vw_bfn_d365dimretailstore.sql)
08:00:50    002003 (42S02): SQL compilation error:
08:00:50    Table 'RAW.PUBLIC.VW_BFN_D365DIMRETAILSTORE' does not exist or not authorized.


Note: Does not exist because table is created at `PRESENTATION96447.BFN_PRESENTATION.vw_bfn_d365dimretailstore`

Environment

- OS:Mac OS 12.6.3
- Python: 3.10.4
- dbt: 1.4.5
- dbt-snowflake: 1.4.5

Which database adapter are you using with dbt?

snowflake

Additional Context

N/A

@rory-donaldson rory-donaldson added bug Something isn't working triage labels Mar 29, 2023
@github-actions github-actions bot changed the title [Bug] {{this}} evaluates incorrectly with Snowflake when used in a macro [CT-2338] [Bug] {{this}} evaluates incorrectly with Snowflake when used in a macro Mar 29, 2023
@jtcohen6 jtcohen6 removed the triage label Mar 31, 2023
@jtcohen6
Copy link
Contributor

@rory-donaldson Unfortunately, you need to "nest your curlies" in this case:

{{ config(
    post_hook = "{{ grant_select(this, ['DW_GL']) }}"
)}}

That way, the grant_select macro will be re-rendered at runtime, when the full configuration for {{ this }} is available.

I know this is a bit ugly & a bit odd. It is documented behavior:

https://docs.getdbt.com/docs/building-a-dbt-project/dont-nest-your-curlies#an-exception

(though we could do a better job of documenting it: dbt-labs/docs.getdbt.com#2818)


Also - did you know that, since v1.2, dbt has had built-in support for grants?

https://docs.getdbt.com/reference/resource-configs/grants

{{ config(grants = {'select': ['DW_GL']}) }}

@jtcohen6 jtcohen6 closed this as not planned Won't fix, can't repro, duplicate, stale Mar 31, 2023
@jtcohen6 jtcohen6 added the duplicate This issue or pull request already exists label Mar 31, 2023
@rory-donaldson
Copy link
Author

Thanks, nesting the curlies fixed it! And yeah an update to the docs to make this more clear would be awesome

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working duplicate This issue or pull request already exists
Projects
None yet
Development

No branches or pull requests

2 participants