Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] freshness tests on external tables should fail to parse loaded_at_field #9979

Closed
3 tasks done
Tracked by #9425
dataders opened this issue Apr 18, 2024 · 3 comments
Closed
3 tasks done
Tracked by #9425
Labels
enhancement New feature or request freshness related to the dbt source freshness command stale Issues that have gone stale user docs [docs.getdbt.com] Needs better documentation
Milestone

Comments

@dataders
Copy link
Contributor

Is this your first time submitting a feature request?

  • I have read the expectations for open source contributors
  • I have searched the existing issues, and I could not find an existing issue for this feature
  • I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

see dbt-labs/dbt-redshift#757

metadata-generated freshness tests (ie those defined without a loaded_at_field) are not supported for External Tables. In addition to documenting this, dbt should error in this scenario and tell the user something more meaningful than current state.

Current behavior

Got an invalid result from "get_relation_last_modified" macro: []

Proposed Behavior

external table source XXX.XXX has metadata-based freshness test defined which is not supported. please add loaded_at_field or remove the freshness test

Describe alternatives you've considered

Perhaps some data platforms do support metadata-based freshness checks for external tables?

If so, perhaps this justifies an expansion of the Capability.TableLastModifiedMetadata entry of the Capability matrix (e.g. SnowflakeAdapter's
)

Who will this benefit?

dbt users with external tables defined

Are you interested in contributing this feature?

No response

Anything else?

No response

@dataders dataders added enhancement New feature or request user docs [docs.getdbt.com] Needs better documentation triage labels Apr 18, 2024
@dbeatty10 dbeatty10 added the freshness related to the dbt source freshness command label Apr 18, 2024
@graciegoheen graciegoheen added this to the v1.9 milestone Apr 23, 2024
@dataders
Copy link
Contributor Author

somewhat related to #9320?

Copy link
Contributor

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

@github-actions github-actions bot added the stale Issues that have gone stale label Oct 28, 2024
Copy link
Contributor

github-actions bot commented Nov 5, 2024

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Nov 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request freshness related to the dbt source freshness command stale Issues that have gone stale user docs [docs.getdbt.com] Needs better documentation
Projects
None yet
Development

No branches or pull requests

3 participants