🐛 Source Monday: fails to parse backoff response

[awang]

Environment

• Airbyte version: 0.33.2-alpha
• OS Version / Instance: Ubuntu 20.04 (2 vCPU ARM)
• Deployment: Docker compose
• Source Connector and version: Monday
• Destination Connector and version: Postgres
• Severity: Very Low / Low / Medium / High / Critical
• Step where error happened: Sync job

Current Behavior

I’m getting a Normalization Failed error. If I try turning off basic normalization, I then get the below JSON decode error. Logs attached

Logs

LOG

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py", line 106, in read
    internal_config=internal_config,
  File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py", line 136, in _read_stream
    for record in record_iterator:
  File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py", line 208, in _read_full_refresh
    for record in records:
  File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/streams/http/http.py", line 351, in read_records
    response = self._send_request(request, request_kwargs)
  File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/streams/http/http.py", line 318, in _send_request
    return backoff_handler(user_backoff_handler)(request, request_kwargs)
  File "/usr/local/lib/python3.7/site-packages/backoff/_sync.py", line 94, in retry
    ret = target(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/backoff/_sync.py", line 94, in retry
    ret = target(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/streams/http/http.py", line 276, in _send
    if self.should_retry(response):
  File "/airbyte/integration_code/source_monday/source.py", line 54, in should_retry
    is_complex_query = response.json().get("errors")
  File "/usr/local/lib/python3.7/site-packages/requests/models.py", line 910, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/local/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Are you willing to submit a PR?

yes!

[alafanechere]

Hi @awang could you please share your source and destination catalogs?

docker run -it --rm --volume airbyte_workspace:/data busybox cat /data/<job-id>/<attempt-id>/source_catalog.json
docker run -it --rm --volume airbyte_workspace:/data busybox cat /data/<job-id>/<attempt-id>/destination_catalog.json

[awang]

@alafanechere sure!
catalogs.zip

[alafanechere]

The error is probably on the Monday connector side, an API response seems to not contain JSON and the connector fails while trying to deserialize json.

[bazarnov]

@alafanechere The source-monday has been migrated to the YAML Low-code and now we have the _safe_response_json() method added by this PR: https://github.com/airbytehq/airbyte/pull/18931/files This should be sufficient for such cases.

@lazebnyi I think this could be closed. Please confirm.

[bazarnov]

Closing because the empty response issue handler is already implemented starting 0.2.0, with migration to Low Code: #19586