Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADAP-505] [Regression] v1.5 breaks models with descriptions of BigQuery struct columns #691

Closed
2 tasks done
bendiktv2 opened this issue May 2, 2023 · 5 comments
Closed
2 tasks done
Labels
type:bug Something isn't working type:regression

Comments

@bendiktv2
Copy link

bendiktv2 commented May 2, 2023

Is this a new bug in dbt-bigquery?

  • I believe this is a new bug in dbt-bigquery
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

I am not able to add descriptions to STRUCT-columns in DBT v1.5.

Expected Behavior

I should be able to add descriptions to STRUCT-columns and their nested columns, as I was able to do in versions prior to 1.5.

Steps To Reproduce

# models/bugged.sql
{{ config(materialized='table') }}

SELECT
  STRUCT( "complete" AS status ) AS nested
# models/bugged.yml
version: 2

models:
  - name: bugged
    columns:
      - name: nested
        description: "Describe the nested struct"

Run dbt run --full-refresh --select bugged

Relevant log output

08:35:55  Running with dbt=1.5.0
08:35:56  Unable to do partial parsing because saved manifest not found. Starting full parse.
08:36:01  Found 1 models, 0 tests, 0 snapshots, 0 analyses, 0 macros, 0 operations, 0 seed files, 0 sources, 0 exposures, 0 metrics, 0 groups
08:36:01  
08:36:03  Concurrency: 16 threads (target='dev')
08:36:03  
08:36:03  1 of 1 START sql table model my_dataset.bugged ............................ [RUN]
08:36:07  Unhandled error while executing target/run/my_model/models/bugged.sql
400 PATCH https://bigquery.googleapis.com/bigquery/v2/projects/my-project/datasets/my_dataset/tables/bugged?prettyPrint=false: Policy tags can only be attached to a leaf field.
08:36:07  1 of 1 ERROR creating sql table model my_dataset.bugged ................... [ERROR in 3.55s]
08:36:07  
08:36:07  Finished running 1 table model in 0 hours 0 minutes and 5.80 seconds (5.80s).
08:36:07  
08:36:07  Completed with 1 error and 0 warnings:
08:36:07  
08:36:07  400 PATCH https://bigquery.googleapis.com/bigquery/v2/projects/my-project/datasets/my_dataset/tables/bugged?prettyPrint=false: Policy tags can only be attached to a leaf field.
08:36:07  
08:36:07  Done. PASS=0 WARN=0 ERROR=1 SKIP=0 TOTAL=1

Environment

- OS: MacOS Ventura 13.3.1 (22E261)
- Python: Python 3.11.3
- dbt:

Core:
  - installed: 1.5.0
  - latest:    1.5.0 - Up to date!

Plugins:
  - bigquery: 1.5.0 - Up to date!

Additional Context

Probably related to #673, but that is a feature request for data_type and contract enforcment, this bug-issue is a breaking change.

@bendiktv2 bendiktv2 added type:bug Something isn't working triage:product labels May 2, 2023
@github-actions github-actions bot changed the title [Bug] v1.5 breaks models with descriptions of BigQuery struct columns [ADAP-505] [Bug] v1.5 breaks models with descriptions of BigQuery struct columns May 2, 2023
@bendiktv2
Copy link
Author

Markdown removed the link to the possibly related issue, so adding it here for links: #673

@bendiktv2
Copy link
Author

It appears this config is also required to reproduce the error:

# dbt_project.yml
models:
    my_model:
      +persist_docs:
        columns: true

@StoryFabian
Copy link

Just found out we have the same issue.
This is a huge breaking change as we have a lot of nested columns documented.

@StoryFabian
Copy link

#687

This one is related

@dbeatty10 dbeatty10 changed the title [ADAP-505] [Bug] v1.5 breaks models with descriptions of BigQuery struct columns [ADAP-505] [Regression] v1.5 breaks models with descriptions of BigQuery struct columns May 2, 2023
@dbeatty10
Copy link
Contributor

Thanks for reaching out @bendiktv2 and @StoryFabian !

This is the same as #687, so I'm going to close this in favor of that one.

There is a PR to fix this that you can track along with: #688

@dbeatty10 dbeatty10 closed this as not planned Won't fix, can't repro, duplicate, stale May 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:bug Something isn't working type:regression
Projects
None yet
Development

No branches or pull requests

3 participants