Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🐛 Source Zendesk Support: Fix schema User stream to pull all users #12122

Merged
merged 9 commits into from
Apr 20, 2022
Merged

🐛 Source Zendesk Support: Fix schema User stream to pull all users #12122

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
c4cbae4
Add SourceZendeskUserExportStream
danieldiamond Apr 19, 2022
abab552
Format streams
danieldiamond Apr 19, 2022
c41737b
Update zendesk-support README
danieldiamond Apr 19, 2022
a7217c3
Update dockerfile
danieldiamond Apr 19, 2022
88e3377
Update changelog to reflect additional fixes
danieldiamond Apr 19, 2022
73bb97d
Review updates
bazarnov Apr 19, 2022
97e548b
Fix typo from previous contribution
danieldiamond Apr 19, 2022
9efe7c4
Update unit tests
danieldiamond Apr 20, 2022
6e65ec3
Remove enum and default from spec
danieldiamond Apr 20, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-zendesk-support/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,5 +25,5 @@ COPY source_zendesk_support ./source_zendesk_support
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.2.5
LABEL io.airbyte.version=0.2.6
LABEL io.airbyte.name=airbyte/source-zendesk-support
22 changes: 12 additions & 10 deletions airbyte-integrations/connectors/source-zendesk-support/source_zendesk_support/streams.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -289,8 +289,8 @@ def read_records(

class SourceZendeskSupportFullRefreshStream(BaseSourceZendeskSupportStream):
"""
# endpoints don't provide the updated_at/created_at fields
# thus we can't implement an incremental logic for them
Endpoints don't provide the updated_at/created_at fields
Thus we can't implement an incremental logic for them
"""

page_size = 100
Expand Down Expand Up @@ -324,7 +324,7 @@ def request_params(self, next_page_token: Mapping[str, Any] = None, **kwargs) ->

class SourceZendeskSupportCursorPaginationStream(SourceZendeskSupportFullRefreshStream):
"""
# endpoints provide a cursor pagination and sorting mechanism
Endpoints provide a cursor pagination and sorting mechanism
"""

next_page_field = "next_page"
Expand Down Expand Up @@ -361,17 +361,17 @@ def request_params(
return params


class SourceZendeskTicketExportStream(SourceZendeskSupportCursorPaginationStream):
class SourceZendeskIncrementalExportStream(SourceZendeskSupportCursorPaginationStream):
"""Incremental Export from Tickets stream:
https://developer.zendesk.com/api-reference/ticketing/ticket-management/incremental_exports/#incremental-ticket-export-time-based

@ param response_list_name: the main nested entity to look at inside of response, defualt = response_list_name
@ param response_list_name: the main nested entity to look at inside of response, default = response_list_name
@ param sideload_param : parameter variable to include various information to response
more info: https://developer.zendesk.com/documentation/ticketing/using-the-zendesk-api/side_loading/#supported-endpoints
"""

cursor_field = "updated_at"
response_list_name: str = "tickets"
response_list_name: str = None
sideload_param: str = None

@staticmethod
Expand Down Expand Up @@ -410,11 +410,11 @@ def parse_response(self, response: requests.Response, **kwargs) -> Iterable[Mapp
yield record


class SourceZendeskSupportTicketEventsExportStream(SourceZendeskTicketExportStream):
class SourceZendeskSupportTicketEventsExportStream(SourceZendeskIncrementalExportStream):
"""Incremental Export from TicketEvents stream:
https://developer.zendesk.com/api-reference/ticketing/ticket-management/incremental_exports/#incremental-ticket-event-export

@ param response_list_name: the main nested entity to look at inside of response, defualt = "ticket_events"
@ param response_list_name: the main nested entity to look at inside of response, default = "ticket_events"
@ param response_target_entity: nested property inside of `response_list_name`, default = "child_events"
@ param list_entities_from_event : the list of nested child_events entities to include from parent record
@ param event_type : specific event_type to check ["Audit", "Change", "Comment", etc]
Expand All @@ -441,9 +441,10 @@ def parse_response(self, response: requests.Response, **kwargs) -> Iterable[Mapp
yield event


class Users(SourceZendeskSupportStream):
"""Users stream: https://developer.zendesk.com/api-reference/ticketing/users/users/"""
class Users(SourceZendeskIncrementalExportStream):
"""Users stream: https://developer.zendesk.com/api-reference/ticketing/ticket-management/incremental_exports/#incremental-user-export"""

response_list_name: str = "users"

class Organizations(SourceZendeskSupportStream):
"""Organizations stream: https://developer.zendesk.com/api-reference/ticketing/ticket-management/incremental_exports/"""
Expand All @@ -452,6 +453,7 @@ class Organizations(SourceZendeskSupportStream):
class Tickets(SourceZendeskTicketExportStream):
"""Tickets stream: https://developer.zendesk.com/api-reference/ticketing/ticket-management/incremental_exports/#incremental-ticket-export-time-based"""

response_list_name: str = "tickets"

class TicketComments(SourceZendeskSupportTicketEventsExportStream):
"""
Expand Down
5 changes: 3 additions & 2 deletions docs/integrations/sources/zendesk-support.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ This Source is capable of syncing the following core Streams:
* [Ticket Forms](https://developer.zendesk.com/rest_api/docs/support/ticket_forms)
* [Ticket Metrics](https://developer.zendesk.com/rest_api/docs/support/ticket_metrics)
* [Ticket Metric Events](https://developer.zendesk.com/api-reference/ticketing/tickets/ticket_metric_events/)
* [Users](https://developer.zendesk.com/rest_api/docs/support/users)
* [Users](https://developer.zendesk.com/api-reference/ticketing/ticket-management/incremental_exports/#incremental-user-export)

### Data type mapping

Expand Down Expand Up @@ -62,7 +62,7 @@ The Zendesk connector should not run into Zendesk API limitations under normal u

### Requirements

* `Subdomain` - this is your Zendesk subdomain that can be found in your account URL. For example, in `https://{MY_SUBDOMAIN}.zendesk.com/`, where `MY_SUBDOMAIN` is the value of your subdomain.
* `Subdomain` - this is your Zendesk subdomain that can be found in your account URL. For example, in `https://{MY_SUBDOMAIN}.zendesk.com/`, where `MY_SUBDOMAIN` is the value of your subdomain.
* `Authentication` - Zendesk service provides two authentication methods. Choose between: `OAuth2.0` or `API token`.
* Authentication using `OAuth2.0` (Only for Airbyte Cloud) - obtain `access_token` by authorising using your Zendesk Account credentials. Simply proceed by pressing "Authenticate your Zendesk Account" and complete the authentication.
* Authentication using `API Token`:
Expand All @@ -74,6 +74,7 @@ The Zendesk connector should not run into Zendesk API limitations under normal u

| Version | Date | Pull Request | Subject |
|:---------|:-----------| :----- |:-------------------------------------------------------|
| `0.2.6` | 2022-04-19 | [12122](https://github.com/airbytehq/airbyte/pull/12122) | Fixed the bug when only 100,000 Users are synced [11895](https://github.com/airbytehq/airbyte/issues/11895) and fixed bug when `start_date` is not used on user stream [12059](https://github.com/airbytehq/airbyte/issues/12059).
| `0.2.5` | 2022-04-05 | [11727](https://github.com/airbytehq/airbyte/pull/11727) | Fixed the bug when state was not parsed correctly
| `0.2.4` | 2022-04-04 | [11688](https://github.com/airbytehq/airbyte/pull/11688) | Small documentation corrections
| `0.2.3` | 2022-03-23 | [11349](https://github.com/airbytehq/airbyte/pull/11349) | Fixed the bug when Tickets stream didn't return deleted records
Expand Down