Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update for v0.19: Result, response objects #504

Merged
merged 8 commits into from
Jan 13, 2021

Conversation

jtcohen6
Copy link
Collaborator

@jtcohen6 jtcohen6 commented Jan 6, 2021

Description & motivation

Thanks to @clrcrl, I realized this evening (dbt-labs/dbt-utils#319 + dbt-labs/dbt-utils#320) that the call statement object has very subtly changed in dbt v0.19.0.

Instead of returning a status, the call statement block now returns an adapter-specific response which includes the same basic info in a structured format, while still rendering to the same thing status used to, via its _message attribute. I'd say this is a big improvement overall!

{
    'response': BigQueryAdapterResponse(_message='OK', code=None, rows_affected=None, bytes_processed=None),
    'data': [(Decimal('1'),)],
    'table': <agate.table.Table object at 0x111ceafd0>
}

It's also a potentially breaking change for macros, materializations, packages (e.g. dbt_utils.insert_by_period, dbt_external_tables.stage_external_sources) that hook right into the statement['status'] field. It should be as simple as switching this from statement['status'] to statement['response']. Or, to be compatible both ways:

{% set my_msg = statement['response'] if 'response' in statement.keys() else statement['status'] %}

While I was there, I realized that I'd failed to update the "dbt Classes" Result object docs. I... had also kind of forgotten these docs existed. They were still linking to a JSON Schema that went away in v0.15.0. I think the whole "dbt Classes" docs could use some love.

To-do before merge

Pre-release docs

Is this change related to an unreleased version of dbt?

  • Yes: please
    • update the base branch to next
    • add Changelog components: <Changelog>[New/Changed] in v0.x.0</Changelog>
    • add links to the "New and changed documentation" section of the latest Migration Guide
  • No: please ensure the base branch is current
  • Unsure: we'll let you know!

Co-authored-by: Claire Carroll <claire@fishtownanalytics.com>
@jtcohen6 jtcohen6 marked this pull request as ready for review January 11, 2021 12:07
Copy link
Contributor

@clrcrl clrcrl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small pieces of feedback, otherwise LGTM

- [dbt-artifacts](dbt-artifacts): The JSON artifacts produced by dbt—manifest, catalog, run results, and sources—are simpler to consume and more clearly documented.
- [snapshots](snapshots) ([invalidate_hard_deletes](invalidate_hard_deletes)): If the config `invalidate_hard_deletes` is enabled, `dbt snapshot` will update records whose unique key no longer exist in the snapshot query. Should those uniquely identified records "revive," `dbt snapshot` will re-add them.
- [dbt Artifacts](dbt-artifacts): The JSON artifacts produced by dbt—manifest, catalog, run results, and sources—are simpler to consume and more clearly documented.
- [dbt Classes](dbt-classes#result-objects), [on-run-end Context](on-run-end-context#results): The `Result` object has a new schema,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hanging comma, should this be a period instead?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ty for catching!

@@ -20,16 +20,17 @@ title: "Upgrading to 0.19.0"

Please be aware of the following changes in v0.19.0:

- dbt artifacts have a new schema. From here on, artifact schemas are officially versioned at **schemas.getdbt.com**. Future breaking changes will be limited to minor version releases.
- Defer, a beta feature introduced in v0.18.0, has subtly changed to better support the "Slim CI" use case.
1. dbt artifacts have a new schema. From here on, artifact schemas are officially versioned at **schemas.getdbt.com**. Future breaking changes will be limited to minor version releases. Some dbt classes, such as the `Result` object, have associated breaking changes.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is a PR for next, but schemas.getdbt.com doesn't have any content — is that expected for now?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup, expected for now — it'll have something there by the time of final release (:

@jtcohen6 jtcohen6 merged commit 0f3fc27 into next Jan 13, 2021
@jtcohen6 jtcohen6 deleted the fix/result-response-objects branch January 13, 2021 10:54
nghi-ly pushed a commit that referenced this pull request May 2, 2024
REPO SYNC - Public to Private
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants