Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(presto/trino): Add TIME/TIMESTAMP WITH TIME ZONE #19263

Merged

Conversation

john-bodley
Copy link
Member

SUMMARY

Presto (and thus Trino) have specific TIME and TIMESTAMP types which support time zones. If the underlying column type is TIMESTAMP WITH TIME ZONE then the PrestoEngineSpec.convert_dttm method would incorrectly format the Python datatime as a SQL string literal as opposed to casting it to a TIMESTAMP.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

Added unit tests and conformed in Presto:

  1. That the from_iso8601_timestamp UDF supported time zones, i.e., from_iso8601_timestamp('2022-01-01T01:23:46.600+00:00')
  2. That Presto supported a comparison between a time zone aware and time zone naive (defaulting to the time zone of the database engine) timestamp, i.e., TIMESTAMP SELECT TIMESTAMP '2001-08-22 03:04:05.321 America/Los_Angeles' > TIMESTAMP '2001-08-22 03:04:05.321' returned true for a Presto server using a UTC-localized time zone.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@john-bodley john-bodley force-pushed the john-bodley--extend-convert-dttm branch from e3fcaaf to 54b5528 Compare March 19, 2022 01:07
@pull-request-size pull-request-size bot added size/L and removed size/M labels Mar 19, 2022
@john-bodley john-bodley force-pushed the john-bodley--extend-convert-dttm branch from 54b5528 to 49a1db9 Compare March 19, 2022 01:50
@codecov
Copy link

codecov bot commented Mar 19, 2022

Codecov Report

Merging #19263 (a813bad) into master (d645579) will increase coverage by 0.00%.
The diff coverage is 100.00%.

❗ Current head a813bad differs from pull request most recent head 299f6b0. Consider uploading reports for the commit 299f6b0 to get more accurate results

@@           Coverage Diff           @@
##           master   #19263   +/-   ##
=======================================
  Coverage   66.53%   66.54%           
=======================================
  Files        1667     1667           
  Lines       64360    64362    +2     
  Branches     6493     6493           
=======================================
+ Hits        42824    42827    +3     
+ Misses      19854    19853    -1     
  Partials     1682     1682           
Flag Coverage Δ
hive 52.65% <50.00%> (+<0.01%) ⬆️
mysql 81.49% <100.00%> (+<0.01%) ⬆️
postgres 81.53% <100.00%> (+<0.01%) ⬆️
presto 52.49% <75.00%> (+<0.01%) ⬆️
python 81.95% <100.00%> (+<0.01%) ⬆️
sqlite 81.30% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/db_engine_specs/presto.py 89.12% <100.00%> (ø)
superset/db_engine_specs/trino.py 71.68% <100.00%> (+0.88%) ⬆️
superset/utils/core.py 89.83% <100.00%> (+0.02%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d645579...299f6b0. Read the comment docs.

@john-bodley john-bodley force-pushed the john-bodley--extend-convert-dttm branch from 49a1db9 to a813bad Compare March 19, 2022 04:28
@john-bodley john-bodley force-pushed the john-bodley--extend-convert-dttm branch from a813bad to 299f6b0 Compare March 19, 2022 18:23
@zhaoyongjie zhaoyongjie self-requested a review March 21, 2022 13:45
Copy link
Member

@zhaoyongjie zhaoyongjie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. BTW, Postgresql has timestamp with time zone as well, we can add to pg specs in the future.

@john-bodley john-bodley merged commit 82a6811 into apache:master Mar 21, 2022
@john-bodley john-bodley deleted the john-bodley--extend-convert-dttm branch March 21, 2022 17:34
michael-hoffman-26 pushed a commit to nielsen-oss/superset that referenced this pull request Mar 23, 2022
Co-authored-by: John Bodley <john.bodley@airbnb.com>
villebro pushed a commit that referenced this pull request Apr 3, 2022
Co-authored-by: John Bodley <john.bodley@airbnb.com>
(cherry picked from commit 82a6811)
@mistercrunch mistercrunch added 🍒 1.5.0 🍒 1.5.1 🍒 1.5.2 🍒 1.5.3 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 2.0.0 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels lts-v1 size/L 🍒 1.5.0 🍒 1.5.1 🍒 1.5.2 🍒 1.5.3 🚢 2.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants