🐛Source Freshdesk: Update parsing of start_date param (#14049)

* Upd parsing of start_date * Bump version * Upd unit tests * Upd timeout * Remove timeout * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
airbytehq · Jul 11, 2022 · 4e80a67 · 4e80a67
1 parent 7221842
commit 4e80a67
Show file tree

Hide file tree

Showing 7 changed files with 37 additions and 39 deletions.
diff --git a/airbyte-config/init/src/main/resources/seed/source_definitions.yaml b/airbyte-config/init/src/main/resources/seed/source_definitions.yaml
@@ -279,7 +279,7 @@
 - name: Freshdesk
   sourceDefinitionId: ec4b9503-13cb-48ab-a4ab-6ade4be46567
   dockerRepository: airbyte/source-freshdesk
-  dockerImageTag: 0.3.1
+  dockerImageTag: 0.3.2
   documentationUrl: https://docs.airbyte.io/integrations/sources/freshdesk
   icon: freshdesk.svg
   sourceType: api

diff --git a/airbyte-config/init/src/main/resources/seed/source_specs.yaml b/airbyte-config/init/src/main/resources/seed/source_specs.yaml
@@ -2487,7 +2487,7 @@
     supportsNormalization: false
     supportsDBT: false
     supported_destination_sync_modes: []
-- dockerImage: "airbyte/source-freshdesk:0.3.1"
+- dockerImage: "airbyte/source-freshdesk:0.3.2"
   spec:
     documentationUrl: "https://docs.airbyte.io/integrations/sources/freshdesk"
     connectionSpecification:

diff --git a/airbyte-integrations/connectors/source-freshdesk/Dockerfile b/airbyte-integrations/connectors/source-freshdesk/Dockerfile
@@ -34,5 +34,5 @@ COPY source_freshdesk ./source_freshdesk
 ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
 ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
 
-LABEL io.airbyte.version=0.3.1
+LABEL io.airbyte.version=0.3.2
 LABEL io.airbyte.name=airbyte/source-freshdesk
diff --git a/airbyte-integrations/connectors/source-freshdesk/README.md b/airbyte-integrations/connectors/source-freshdesk/README.md
@@ -47,10 +47,10 @@ and place them into `secrets/config.json`.
 
 ### Locally running the connector
 ```
-python main_dev.py spec
-python main_dev.py check --config secrets/config.json
-python main_dev.py discover --config secrets/config.json
-python main_dev.py read --config secrets/config.json --catalog sample_files/configured_catalog.json
+python main.py spec
+python main.py check --config secrets/config.json
+python main.py discover --config secrets/config.json
+python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json
 ```
 
 ### Unit Tests

diff --git a/airbyte-integrations/connectors/source-freshdesk/source_freshdesk/streams.py b/airbyte-integrations/connectors/source-freshdesk/source_freshdesk/streams.py
@@ -34,9 +34,7 @@ def __init__(self, authenticator: AuthBase, config: Mapping[str, Any], *args, **
         # Since this logic rely not on updated tickets, it can break tickets dependant streams - conversations.
         # So updated_since parameter will be always used in tickets streams. And start_date will be used too
         # with default value 30 days look back.
-        self.start_date = (
-            pendulum.parse(config.get("start_date")) if config.get("start_date") else pendulum.now() - pendulum.duration(days=30)
-        )
+        self.start_date = config.get("start_date") or (pendulum.now() - pendulum.duration(days=30)).strftime("%Y-%m-%dT%H:%M:%SZ")
 
     @property
     def url_base(self) -> str:
@@ -331,7 +329,6 @@ def next_page_token(self, response: requests.Response) -> Optional[Mapping[str,
         if next_page_token and int(next_page_token["page"]) > self.ticket_paginate_limit:
             # get last_record from latest batch, pos. -1, because of ACS order of records
             last_record_updated_at = response.json()[-1]["updated_at"]
-            last_record_updated_at = pendulum.parse(last_record_updated_at)
             # updating request parameters with last_record state
             next_page_token[self.cursor_filter] = last_record_updated_at
             next_page_token.pop("page")

diff --git a/airbyte-integrations/connectors/source-freshdesk/unit_tests/test_300_page.py b/airbyte-integrations/connectors/source-freshdesk/unit_tests/test_300_page.py
@@ -11,70 +11,70 @@
 def responses_fixtures():
     return [
         {
-            "url": "/api/tickets?per_page=1&updated_since=2002-02-10T22%3A21%3A44%2B00%3A00",
+            "url": "/api/tickets?per_page=1&updated_since=2002-02-10T22%3A21%3A44Z",
             "json": [{"id": 1, "updated_at": "2018-01-02T00:00:00Z"}],
             "headers": {
-                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=2&updated_since=2002-02-10T22%3A21%3A44%2B00%3A00>; rel="next"'
+                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=2&updated_since=2002-02-10T22%3A21%3A44Z>; rel="next"'
             },
         },
         {
-            "url": "/api/tickets?per_page=1&page=2&updated_since=2002-02-10T22%3A21%3A44%2B00%3A00",
+            "url": "/api/tickets?per_page=1&page=2&updated_since=2002-02-10T22%3A21%3A44Z",
             "json": [{"id": 2, "updated_at": "2018-02-02T00:00:00Z"}],
             "headers": {
-                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=3&updated_since=2002-02-10T22%3A21%3A44%2B00%3A00>; rel="next"'
+                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=3&updated_since=2002-02-10T22%3A21%3A44Z>; rel="next"'
             },
         },
         {
-            "url": "/api/tickets?per_page=1&updated_since=2018-02-02T00%3A00%3A00%2B00%3A00",
+            "url": "/api/tickets?per_page=1&updated_since=2018-02-02T00%3A00%3A00Z",
             "json": [{"id": 2, "updated_at": "2018-02-02T00:00:00Z"}],
             "headers": {
-                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=2&updated_since=2018-02-02T00%3A00%3A00%2B00%3A00>; rel="next"'
+                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=2&updated_since=2018-02-02T00%3A00%3A00Z>; rel="next"'
             },
         },
         {
-            "url": "/api/tickets?per_page=1&page=2&updated_since=2018-02-02T00%3A00%3A00%2B00%3A00",
+            "url": "/api/tickets?per_page=1&page=2&updated_since=2018-02-02T00%3A00%3A00Z",
             "json": [{"id": 3, "updated_at": "2018-03-02T00:00:00Z"}],
             "headers": {
-                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=3&updated_since=2018-02-02T00%3A00%3A00%2B00%3A00>; rel="next"'
+                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=3&updated_since=2018-02-02T00%3A00%3A00Z>; rel="next"'
             },
         },
         {
-            "url": "/api/tickets?per_page=1&updated_since=2018-03-02T00%3A00%3A00%2B00%3A00",
+            "url": "/api/tickets?per_page=1&updated_since=2018-03-02T00%3A00%3A00Z",
             "json": [{"id": 3, "updated_at": "2018-03-02T00:00:00Z"}],
             "headers": {
-                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=2&updated_since=2018-03-02T00%3A00%3A00%2B00%3A00>; rel="next"'
+                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=2&updated_since=2018-03-02T00%3A00%3A00Z>; rel="next"'
             },
         },
         {
-            "url": "/api/tickets?per_page=1&page=2&updated_since=2018-03-02T00%3A00%3A00%2B00%3A00",
+            "url": "/api/tickets?per_page=1&page=2&updated_since=2018-03-02T00%3A00%3A00Z",
             "json": [{"id": 4, "updated_at": "2019-01-03T00:00:00Z"}],
             "headers": {
-                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=3&updated_since=2018-03-02T00%3A00%3A00%2B00%3A00>; rel="next"'
+                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=3&updated_since=2018-03-02T00%3A00%3A00Z>; rel="next"'
             },
         },
         {
-            "url": "/api/tickets?per_page=1&updated_since=2019-01-03T00%3A00%3A00%2B00%3A00",
+            "url": "/api/tickets?per_page=1&updated_since=2019-01-03T00%3A00%3A00Z",
             "json": [{"id": 4, "updated_at": "2019-01-03T00:00:00Z"}],
             "headers": {
-                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=2&updated_since=2019-01-03T00%3A00%3A00%2B00%3A00>; rel="next"'
+                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=2&updated_since=2019-01-03T00%3A00%3A00Z>; rel="next"'
             },
         },
         {
-            "url": "/api/tickets?per_page=1&page=2&updated_since=2019-01-03T00%3A00%3A00%2B00%3A00",
+            "url": "/api/tickets?per_page=1&page=2&updated_since=2019-01-03T00%3A00%3A00Z",
             "json": [{"id": 5, "updated_at": "2019-02-03T00:00:00Z"}],
             "headers": {
-                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=3&updated_since=2019-01-03T00%3A00%3A00%2B00%3A00>; rel="next"'
+                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=3&updated_since=2019-01-03T00%3A00%3A00Z>; rel="next"'
             },
         },
         {
-            "url": "/api/tickets?per_page=1&updated_since=2019-02-03T00%3A00%3A00%2B00%3A00",
+            "url": "/api/tickets?per_page=1&updated_since=2019-02-03T00%3A00%3A00Z",
             "json": [{"id": 5, "updated_at": "2019-02-03T00:00:00Z"}],
             "headers": {
-                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=2&updated_since=2019-02-03T00%3A00%3A00%2B00%3A00>; rel="next"'
+                "Link": '<https://test.freshdesk.com/api/tickets?per_page=1&page=2&updated_since=2019-02-03T00%3A00%3A00Z>; rel="next"'
             },
         },
         {
-            "url": "/api/tickets?per_page=1&page=2&updated_since=2019-02-03T00%3A00%3A00%2B00%3A00",
+            "url": "/api/tickets?per_page=1&page=2&updated_since=2019-02-03T00%3A00%3A00Z",
             "json": [{"id": 6, "updated_at": "2019-03-03T00:00:00Z"}],
         },
     ]

diff --git a/docs/02-integrations/01-sources/freshdesk.md b/docs/02-integrations/01-sources/freshdesk.md
@@ -51,12 +51,13 @@ Please read [How to find your API key](https://support.freshdesk.com/support/sol
 
 ## Changelog
 
-| Version | Date       | Pull Request                                             | Subject                                                                        |
-|:--------|:-----------|:---------------------------------------------------------|:-------------------------------------------------------------------------------|
-| 0.3.1  | 2022-06-03 | [13332](https://github.com/airbytehq/airbyte/pull/13332)   | Add new streams      
-| 0.3.0  | 2022-05-30 | [12334](https://github.com/airbytehq/airbyte/pull/12334)   | Implement with latest CDK      
-| 0.2.11  | 2021-12-14 | [8682](https://github.com/airbytehq/airbyte/pull/8682)   | Migrate to the CDK                                                             |
-| 0.2.10  | 2021-12-06 | [8524](https://github.com/airbytehq/airbyte/pull/8524)   | Update connector fields title/description                                      |
-| 0.2.9   | 2021-11-16 | [8017](https://github.com/airbytehq/airbyte/pull/8017)   | Bugfix an issue that caused the connector not to sync more than 50000 contacts |
-| 0.2.8   | 2021-10-28 | [7486](https://github.com/airbytehq/airbyte/pull/7486)   | Include "requester" and "stats" fields in "tickets" stream                     |
-| 0.2.7   | 2021-10-13 | [6442](https://github.com/airbytehq/airbyte/pull/6442)   | Add start_date parameter to specification from which to start pulling data.    |
+| Version | Date       | Pull Request                                                | Subject                                                                        |
+|:--------|:-----------|:------------------------------------------------------------|:-------------------------------------------------------------------------------|
+| 0.3.2   | 2022-06-23 | [14049](https://github.com/airbytehq/airbyte/pull/14049)    | Update parsing of start_date                                                   |
+| 0.3.1   | 2022-06-03 | [13332](https://github.com/airbytehq/airbyte/pull/13332)    | Add new streams                                                                |
+| 0.3.0   | 2022-05-30 | [12334](https://github.com/airbytehq/airbyte/pull/12334)    | Implement with latest CDK                                                      |      
+| 0.2.11  | 2021-12-14 | [8682](https://github.com/airbytehq/airbyte/pull/8682)      | Migrate to the CDK                                                             |
+| 0.2.10  | 2021-12-06 | [8524](https://github.com/airbytehq/airbyte/pull/8524)      | Update connector fields title/description                                      |
+| 0.2.9   | 2021-11-16 | [8017](https://github.com/airbytehq/airbyte/pull/8017)      | Bugfix an issue that caused the connector not to sync more than 50000 contacts |
+| 0.2.8   | 2021-10-28 | [7486](https://github.com/airbytehq/airbyte/pull/7486)      | Include "requester" and "stats" fields in "tickets" stream                     |
+| 0.2.7   | 2021-10-13 | [6442](https://github.com/airbytehq/airbyte/pull/6442)      | Add start_date parameter to specification from which to start pulling data.    |