Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: invalid conversion of timezone-aware datetime values to JSON #480

Merged
merged 4 commits into from
Jan 25, 2021

Conversation

plamut
Copy link
Contributor

@plamut plamut commented Jan 21, 2021

Fixes #476.

This PR fixes converting timezone-aware datetime values to an UTC JSON string.

PR checklist:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

@plamut plamut requested review from a team and shollyman and removed request for a team January 21, 2021 15:19
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Jan 21, 2021
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Jan 21, 2021
@plamut plamut changed the title Iss 476 fix: invalid conversion of timezone-aware datetime values to JSON Jan 21, 2021
setup.py Outdated Show resolved Hide resolved
@@ -315,13 +315,21 @@ def _timestamp_to_json_parameter(value):
def _timestamp_to_json_row(value):
"""Coerce 'value' to an JSON-compatible representation."""
if isinstance(value, datetime.datetime):
# For naive datetime objects UTC timezone is assumed, thus we format
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this always our assumption? Based on the unit tests, I suppose so?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't say if "always", but the _datetime_to_json() helper has been unchanged since 2017 and was supposedly doing its job fine with naive datetime objects in this time - at least I don't remember any reported issues related to timezones. The strftime() conversion just uses the datetime fields at their face value and ignores any tzinfo set.

Considering that naive datetimes are used most of the time (as opposed to timezone-aware datetime objects) and that users did not complain, the UTC timezone assumption was apparently correct behavior. The logic just failed to cover the case when somebody actually used a timezone-aware instance.

And indeed, unit tests expect the same, a naive datetime instance is supposed to be converted to a string representation with a zero offset from UTC (example).

@plamut plamut requested a review from tswast January 25, 2021 12:16
@plamut plamut merged commit 61b4385 into googleapis:master Jan 25, 2021
@plamut plamut deleted the iss-476 branch January 25, 2021 17:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
2 participants