Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Warn if timestamp updated_at field uses incompatible timestamp #10352

Merged
merged 14 commits into from
Sep 4, 2024

Conversation

gshank
Copy link
Contributor

@gshank gshank commented Jun 21, 2024

resolves #10234

Problem

Snapshot tables use the 'get_snapshot_time' macro, which may be incompatible with the data type of the field specified as the 'updated_at' field, causing unexpected results.

Solution

Warn if the two sources of timestamp columns have different data types.

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX
  • This PR includes type annotations for new and modified functions

@gshank gshank requested a review from a team as a code owner June 21, 2024 20:46
@cla-bot cla-bot bot added the cla:yes label Jun 21, 2024
Copy link

codecov bot commented Jun 21, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.88%. Comparing base (b56d96d) to head (1d1152b).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #10352      +/-   ##
==========================================
- Coverage   88.94%   88.88%   -0.07%     
==========================================
  Files         180      180              
  Lines       22787    22798      +11     
==========================================
- Hits        20267    20263       -4     
- Misses       2520     2535      +15     
Flag Coverage Δ
integration 86.12% <100.00%> (-0.13%) ⬇️
unit 62.35% <87.50%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
Unit Tests 62.35% <87.50%> (-0.01%) ⬇️
Integration Tests 86.12% <100.00%> (-0.13%) ⬇️

@@ -1,7 +1,7 @@
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler. DO NOT EDIT!
# source: core_types.proto
# Protobuf Python Version: 5.26.1
# Protobuf Python Version: 4.25.3
Copy link
Contributor

@MichelleArk MichelleArk Sep 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we'd have CI for this, but for now let's ensure we're using 5.26.1 if possible to generate the protos: #10456

Copy link
Contributor

@MichelleArk MichelleArk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small note on core/dbt/events/core_types_pb2.py, otherwise looks good!

@MichelleArk
Copy link
Contributor

Added a tech debt ticket re: protoc versions; #10658

@MichelleArk MichelleArk closed this Sep 3, 2024
@MichelleArk MichelleArk reopened this Sep 3, 2024
@gshank gshank merged commit c28cb92 into main Sep 4, 2024
65 of 66 checks passed
@gshank gshank deleted the snapshot_timestamp_data_types branch September 4, 2024 18:42
Copy link
Contributor

github-actions bot commented Sep 4, 2024

The backport to 1.8.latest failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.8.latest 1.8.latest
# Navigate to the new working tree
cd .worktrees/backport-1.8.latest
# Create a new branch
git switch --create backport-10352-to-1.8.latest
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c28cb92af51d7f2cb27618aeb43705ba951aa3ef
# Push it to GitHub
git push --set-upstream origin backport-10352-to-1.8.latest
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.8.latest

Then, create a pull request where the base branch is 1.8.latest and the compare/head branch is backport-10352-to-1.8.latest.

gshank added a commit that referenced this pull request Sep 4, 2024
Co-authored-by: Michelle Ark <michelle.ark@dbtlabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Warn if updated_at field for snapshot is not same datatype as what's returned in snapshot_get_time()
2 participants