fix: preserve timestamp microsecond precision with rows from REST API #402
Conversation
google-cla
bot
added
the
cla: yes
product-auto-label
bot
added
the
api: bigquery
| @@ -82,7 +82,18 @@ def _timestamp_from_json(value, field): | |||
| """Coerce 'value' to a datetime, if set or not nullable.""" | |||
| if _not_null(value, field): | |||
| # value will be a float in seconds, to microsecond precision, in UTC. | |||
| return _datetime_from_microseconds(1e6 * float(value)) | |||
There was a problem hiding this comment.
This will need to be modified to parse from an rfc timestamp string.
There was a problem hiding this comment.
Sorry, i didn't get the point. Before passing params["formatOptions.useInt64Timestamp"] = True in client.list_row, received a value in string with decimal in _timestamp_from_json method , but after passing it, received a value in string without decimal and call the method _datetime_from_microseconds(int(value)) which returns datetime object.
There was a problem hiding this comment.
Oh I see. Yes, that's expected.
There was a problem hiding this comment.
In this case, we should update the comment.
Also, are there cases where floating point values are still passed in? If not, we should clean this up now. If so, let's add a TODO to identify those cases and file an issue to clean them up.
google/cloud/bigquery/_helpers.py
Outdated
| elif value.isdigit(): | ||
| value = int(value) | ||
| else: | ||
| value = 1e6 * float(value) |
There was a problem hiding this comment.
Why is this case needed? Couldn't the above isdigit() check be removed and we always fall back to converting to integer? I don't expect floating point values to be serialized as strings in JSON.
…into bigquery_issue_395
google/cloud/bigquery/_helpers.py
Outdated
| return _datetime_from_microseconds(1e6 * float(value)) | ||
| # value will be a integer in seconds, to microsecond precision, in UTC. | ||
|
|
||
| # if value is not None: |
There was a problem hiding this comment.
Please remove commented out code.
…into bigquery_issue_395
tswast
changed the title
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Fixes #395