Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse to UTC for datetime transform #69

Merged
merged 4 commits into from
Apr 11, 2018

Conversation

dmosorast
Copy link
Contributor

The transformer requires datetime strings to be pegged at UTC, or it fails. This causes datetime strings with timezones to fail, since they have a time zone attached and aren't being shifted.

This adds a function to utils that tries to parse and peg at UTC before passing into strftime and uses it within string_to_datetime in transform.py.

@dmosorast dmosorast requested review from psantacl and KAllan357 April 10, 2018 20:15
@nick-mccoy
Copy link
Contributor

This also addresses this issue. We implemented a cross-platform fix similar to what was done here.

@dipamacharyya
Copy link

I'm running tap-onfleet --config config.json --catalog catalog.json but getting an unknown string format for my system time

Can someone help me fix this error?

INFO administrators: Starting sync
INFO GET request to https://onfleet.com/api/v2/admins
INFO METRIC: {"type": "counter", "metric": "record_count", "value": 0, "tags": {"endpoint": "administrators"}}
CRITICAL Unknown string format: 2022-10-10 11:13:18 Pacific Daylight Time
Traceback (most recent call last):
File "C:\python39\lib\runpy.py", line 197, in run_module_as_main
return run_code(code, main_globals, None,
File "C:\python39\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Python39\Scripts\tap-onfleet.exe_main.py", line 7, in
File "C:\python39\lib\site-packages\singer\utils.py", line 192, in wrapped
return fnc(*args, **kwargs)
File "C:\python39\lib\site-packages\tap_onfleet_init.py", line 91, in main
sync(client, parsed_args.catalog, state)
File "C:\python39\lib\site-packages\tap_onfleet_init.py", line 66, in sync
counter_value = sync_stream(state, instance)
File "C:\python39\lib\site-packages\tap_onfleet\sync.py", line 19, in sync_stream
for (stream, record) in instance.sync(state):
File "C:\python39\lib\site-packages\tap_onfleet\streams.py", line 103, in sync
self.update_bookmark(state, item[self.replication_key])
File "C:\python39\lib\site-packages\tap_onfleet\streams.py", line 55, in update_bookmark
if self.is_bookmark_old(state, value):
File "C:\python39\lib\site-packages\tap_onfleet\streams.py", line 61, in is_bookmark_old
return utils.strptime_with_tz(value) > utils.strptime_with_tz(current_bookmark)
File "C:\python39\lib\site-packages\singer\utils.py", line 21, in strptime_with_tz
d_object = dateutil.parser.parse(dtime)
File "C:\python39\lib\site-packages\dateutil\parser_parser.py", line 1368, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File "C:\python39\lib\site-packages\dateutil\parser_parser.py", line 643, in parse
raise ParserError("Unknown string format: %s", timestr)
dateutil.parser._parser.ParserError: Unknown string format: 2022-10-10 11:13:18 Pacific Daylight Time

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants