Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(postgres): remove redundant tz factory #17713

Merged
merged 2 commits into from
Dec 10, 2021

Conversation

villebro
Copy link
Member

SUMMARY

A recent PR #17290 bumped Psycopg2 from 2.8.5 to 2.9.1, which features many changes to how timezones are handled: https://www.psycopg.org/articles/2021/06/16/psycopg-29-released/. Most notable is the change replacing the internal FixedOffsetTimezone class with the regular datetime.timezone when creating datetime objects, which is the "normal behavior" when working with datetimes. As a consequence, the constructor call to cursor.tzinfo_factory previously received the offset in minutes, while it now receives it as a datetime.timedelta object. As the custom FixedOffsetTimezone was only needed to make the timezone serializable by PyArrow, we can now remove it, as datetime objects containing a timezone tzinfo are serializable by PyArrow:

>>> import pyarrow as pa
>>> from datetime import datetime, timezone
>>> buf = pa.serialize(datetime(year=2021, month=1, day=1, tzinfo=timezone.utc)).to_buffer()
sys:1: FutureWarning: 'pyarrow.serialize' is deprecated as of 2.0.0 and will be removed in a future version. Use pickle or the pyarrow IPC functionality instead.
>>> pa.deserialize(buf)
sys:1: FutureWarning: 'pyarrow.deserialize' is deprecated as of 2.0.0 and will be removed in a future version. Use pickle or the pyarrow IPC functionality instead.
datetime.datetime(2021, 12, 10, 0, 0)

AFTER

Now running the query SELECT TO_TIMESTAMP('2019-12-10 01:00:00.000000', 'YYYY-MM-DD HH24:MI:SS.US') works just fine:
image

BEFORE

Previously it failed with the following error:
image

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@codecov
Copy link

codecov bot commented Dec 10, 2021

Codecov Report

Merging #17713 (68f47f2) into master (8e69b2d) will decrease coverage by 0.00%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #17713      +/-   ##
==========================================
- Coverage   68.13%   68.13%   -0.01%     
==========================================
  Files        1653     1653              
  Lines       66375    66371       -4     
  Branches     7120     7120              
==========================================
- Hits        45226    45220       -6     
- Misses      19252    19254       +2     
  Partials     1897     1897              
Flag Coverage Δ
hive 81.78% <ø> (-0.01%) ⬇️
mysql 82.19% <ø> (-0.01%) ⬇️
postgres 82.20% <ø> (-0.01%) ⬇️
presto 82.06% <ø> (-0.01%) ⬇️
python 82.68% <ø> (-0.01%) ⬇️
sqlite 81.88% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/db_engine_specs/postgres.py 97.27% <ø> (-0.10%) ⬇️
superset/db_engine_specs/presto.py 89.97% <0.00%> (-0.42%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8e69b2d...68f47f2. Read the comment docs.

@villebro villebro merged commit 6a7e649 into apache:master Dec 10, 2021
@villebro villebro deleted the villebro/postgres-tz branch December 10, 2021 19:53
nickbp added a commit to scie-nz/walden that referenced this pull request Feb 9, 2022
Issue with psycopg2 version workaround: apache/superset#15768
Fixed in (unreleased) Superset: apache/superset#17713
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.5.0 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels preset-io size/XS 🚢 1.5.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants