Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] persist_docs not working for seeds #562

Open
2 tasks done
dpguthrie opened this issue Apr 30, 2024 · 3 comments
Open
2 tasks done

[Bug] persist_docs not working for seeds #562

dpguthrie opened this issue Apr 30, 2024 · 3 comments
Labels
feature:docs Issues related to dbt docs command pkg:dbt-bigquery Issue affects dbt-bigquery type:bug Something isn't working as documented

Comments

@dpguthrie
Copy link

Is this a new bug in dbt-bigquery?

  • I believe this is a new bug in dbt-bigquery
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

The descriptions for the seed and columns are not being persisted to BQ when using the appropriate config.

seeds:
  [<resource-path>](https://docs.getdbt.com/reference/resource-configs/resource-path):
    +persist_docs:
      relation: true
      columns: true

Expected Behavior

The descriptions are populated at the object and column level for the seed in my dbt project.

Steps To Reproduce

  1. Use the yaml below to persist_docs for seeds
seeds:
  [<resource-path>](https://docs.getdbt.com/reference/resource-configs/resource-path):
    +persist_docs:
      relation: true
      columns: true
  1. Run dbt seed
  2. View descriptions in BQ for that seed

This may have something to do with the size of the seed though. From @mikealfare "It might also have something to do with the fact that tables are sharded in BQ, and we only pull the column descriptions from the most recent shard (because they can change)."

Relevant log output

No response

Environment

- OS:
- Python:
- dbt-core:
- dbt-bigquery:

Additional Context

No response

@dpguthrie dpguthrie added type:bug Something isn't working as documented triage:product In Product's queue labels Apr 30, 2024
@jtcohen6
Copy link
Contributor

It looks like we persist top-level descriptions for seeds, but not column-level descriptions:

https://github.com/dbt-labs/dbt-bigquery/blob/b7bb0bc0818644617b8c10eb0723560ce98147a5/dbt/include/bigquery/macros/materializations/seed.sql#L20-L23

I'm not sure why that is - this implementation goes way back:

Screenshot 2024-05-14 at 13 46 57

@dpguthrie
Copy link
Author

@jtcohen6 that's interesting, I was able to persist my column descriptions:

image

@amychen1776 amychen1776 removed the triage:product In Product's queue label Jul 25, 2024
@dpguthrie
Copy link
Author

@jtcohen6 @amychen1776 My customer is asking if we're able to prioritize this at all?

@amychen1776 amychen1776 added the feature:docs Issues related to dbt docs command label Jan 8, 2025
@mikealfare mikealfare added the pkg:dbt-bigquery Issue affects dbt-bigquery label Jan 14, 2025
@mikealfare mikealfare transferred this issue from dbt-labs/dbt-bigquery Jan 14, 2025
mikealfare added a commit that referenced this issue Jan 20, 2025
* Update flake8 requirement from ~=6.0 to ~=6.1

Updates the requirements on [flake8](https://github.com/pycqa/flake8) to permit the latest version.
- [Commits](PyCQA/flake8@6.0.0...6.1.0)

---
updated-dependencies:
- dependency-name: flake8
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

* remove extra line at eof in the changelog

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
Co-authored-by: Mike Alfare <mike.alfare@dbtlabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature:docs Issues related to dbt docs command pkg:dbt-bigquery Issue affects dbt-bigquery type:bug Something isn't working as documented
Projects
None yet
Development

No branches or pull requests

4 participants