Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 Source Zendesk chat: fix chats stream is only pulling for first page #7210

Merged
merged 5 commits into from
Nov 1, 2021

Conversation

harshithmullapudi
Copy link
Contributor

What

To close #7161

How

Get next token from the response and use it in next call

Community member or Airbyter

  • Community member? Grant edit access to maintainers (instructions)
  • Secrets in the connector's spec are annotated with airbyte_secret
  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • docs/SUMMARY.md
    • docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
    • docs/integrations/README.md
    • airbyte-integrations/builds.md
  • PR name follows PR naming conventions
  • Connector added to connector index like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • Credentials added to Github CI. Instructions.
  • /test connector=connectors/<name> command is passing.
  • New Connector version released on Dockerhub by running the /publish command described here

Updating a connector

Community member or Airbyter

  • Grant edit access to maintainers (instructions)
  • Secrets in the connector's spec are annotated with airbyte_secret
  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
  • PR name follows PR naming conventions
  • Connector version bumped like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • Credentials added to Github CI. Instructions.
  • /test connector=connectors/<name> command is passing.
  • New Connector version released on Dockerhub by running the /publish command described here

Connector Generator

  • Issue acceptance criteria met
  • PR name follows PR naming conventions
  • If adding a new generator, add it to the list of scaffold modules being tested
  • The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates then checking in your changes
  • Documentation which references the generator is updated as needed.

@github-actions github-actions bot added the area/connectors Connector related issues label Oct 20, 2021
@harshithmullapudi harshithmullapudi changed the title fix: chats stream is only pulling for first page 🐛 Source Zendesk: fix chats stream is only pulling for first page Oct 20, 2021
@github-actions github-actions bot added the area/documentation Improvements or additions to documentation label Oct 21, 2021
@harshithmullapudi harshithmullapudi temporarily deployed to more-secrets October 21, 2021 09:29 Inactive
Copy link
Member

@marcosmarxm marcosmarxm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please run tests and publish the connector.

Copy link
Contributor

@sherifnada sherifnada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marcosmarxm marcosmarxm added this to the USE 2021-10-23 milestone Oct 25, 2021
@harshithmullapudi harshithmullapudi changed the title 🐛 Source Zendesk: fix chats stream is only pulling for first page 🐛 Source Zendesk chat: fix chats stream is only pulling for first page Oct 25, 2021
@harshithmullapudi
Copy link
Contributor Author

harshithmullapudi commented Oct 25, 2021

/test connector=connectors/source-zendesk-chat

🕑 connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1382585437
❌ connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1382585437
🐛 https://gradle.com/s/fvbhzjpijvmaa

@jrhizor jrhizor temporarily deployed to more-secrets October 25, 2021 19:19 Inactive
@harshithmullapudi
Copy link
Contributor Author

harshithmullapudi commented Oct 27, 2021

/test connector=connectors/source-zendesk-chat

🕑 connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1388477505
❌ connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1388477505
🐛 https://gradle.com/s/6hzhlkjgmblde

@harshithmullapudi harshithmullapudi temporarily deployed to more-secrets October 27, 2021 03:30 Inactive
@jrhizor jrhizor temporarily deployed to more-secrets October 27, 2021 03:31 Inactive
@harshithmullapudi harshithmullapudi temporarily deployed to more-secrets October 27, 2021 03:54 Inactive
@harshithmullapudi
Copy link
Contributor Author

harshithmullapudi commented Oct 27, 2021

/test connector=connectors/source-zendesk-chat

🕑 connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1388606515
❌ connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1388606515
🐛 https://gradle.com/s/5eououw76e7wi

@harshithmullapudi harshithmullapudi temporarily deployed to more-secrets October 27, 2021 04:29 Inactive
@jrhizor jrhizor temporarily deployed to more-secrets October 27, 2021 04:30 Inactive
@danieldiamond
Copy link
Contributor

danieldiamond commented Oct 29, 2021

  1. why does the overall class definition of this stream look different to the zendesk talk one?
  • you're moving api.py to streams.py, shouldn't we do this here as well: 🐛 Source Zendesk Talk: migrate to CDK #7173
  • looking at zendesk talk and chat (and haven't looked at support but imagine the same), it doesn't look very DRY. wondering if this can be remedied
  1. can the "fix" here (i imagine these lines) be applied to zendesk talk? i've been eagerly awaiting the talk connector to be fixed so anything to get that over the line!

@harshithmullapudi
Copy link
Contributor Author

harshithmullapudi commented Nov 1, 2021

/test connector=connectors/source-zendesk-chat

🕑 connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1407556604
❌ connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1407556604
🐛 https://gradle.com/s/f6fb6v6kp2vqg

@harshithmullapudi harshithmullapudi temporarily deployed to more-secrets November 1, 2021 10:51 Inactive
@jrhizor jrhizor temporarily deployed to more-secrets November 1, 2021 10:53 Inactive
@harshithmullapudi
Copy link
Contributor Author

harshithmullapudi commented Nov 1, 2021

/test connector=connectors/source-zendesk-chat

🕑 connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1407609469
❌ connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1407609469
🐛 https://gradle.com/s/vixpniau5fybk

@jrhizor jrhizor temporarily deployed to more-secrets November 1, 2021 11:09 Inactive
@harshithmullapudi
Copy link
Contributor Author

harshithmullapudi commented Nov 1, 2021

/test connector=connectors/source-zendesk-chat

🕑 connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1407649765
✅ connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1407649765
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 47     20    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  860    419    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                       Stmts   Miss  Cover
	 --------------------------------------------------------------
	 base_python/__init__.py                       13      0   100%
	 base_python/catalog_helpers.py                10      6    40%
	 base_python/cdk/__init__.py                    0      0   100%
	 base_python/cdk/abstract_source.py            83     59    29%
	 base_python/cdk/streams/__init__.py            0      0   100%
	 base_python/cdk/streams/auth/__init__.py       0      0   100%
	 base_python/cdk/streams/auth/core.py           8      1    88%
	 base_python/cdk/streams/auth/jwt.py            5      5     0%
	 base_python/cdk/streams/auth/oauth.py         37     26    30%
	 base_python/cdk/streams/auth/token.py          9      4    56%
	 base_python/cdk/streams/core.py               63     32    49%
	 base_python/cdk/streams/exceptions.py         10      2    80%
	 base_python/cdk/streams/http.py               67     33    51%
	 base_python/cdk/streams/rate_limiting.py      30     14    53%
	 base_python/cdk/utils/__init__.py              0      0   100%
	 base_python/cdk/utils/casing.py                4      0   100%
	 base_python/client.py                         56     33    41%
	 base_python/entrypoint.py                     70     56    20%
	 base_python/integration.py                    52     25    52%
	 base_python/logger.py                         33     19    42%
	 base_python/schema_helpers.py                 56     41    27%
	 base_python/source.py                         51     34    33%
	 main_dev.py                                    3      3     0%
	 --------------------------------------------------------------
	 TOTAL                                        660    393    40%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                              Stmts   Miss  Cover
	 /actions-runner/_work/airbyte/airbyte/airbyte-integrations/connectors/source-zendesk-chat/.venv/lib/python3.8/site-packages/coverage/control.py:761: CoverageWarning: No data was collected. (no-data-collected)
	 -----------------------------------------------------
	 main_dev.py                           6      6     0%
	   self._warn("No data was collected.", slug="no-data-collected")
	 source_zendesk_chat/__init__.py       2      2     0%
	 source_zendesk_chat/source.py        18     18     0%
	 source_zendesk_chat/streams.py      142    142     0%
	 -----------------------------------------------------
	 TOTAL                               168    168     0%

@jrhizor jrhizor temporarily deployed to more-secrets November 1, 2021 11:20 Inactive
@harshithmullapudi harshithmullapudi temporarily deployed to more-secrets November 1, 2021 11:28 Inactive
@harshithmullapudi
Copy link
Contributor Author

harshithmullapudi commented Nov 1, 2021

/publish connector=connectors/source-zendesk-chat

🕑 connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1407688044
❌ connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1407688044

@harshithmullapudi harshithmullapudi temporarily deployed to more-secrets November 1, 2021 11:32 Inactive
@jrhizor jrhizor temporarily deployed to more-secrets November 1, 2021 11:32 Inactive
@harshithmullapudi
Copy link
Contributor Author

harshithmullapudi commented Nov 1, 2021

/test connector=connectors/source-zendesk-chat

🕑 connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1407861751
✅ connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1407861751
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 47     20    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  860    419    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                       Stmts   Miss  Cover
	 --------------------------------------------------------------
	 base_python/__init__.py                       13      0   100%
	 base_python/catalog_helpers.py                10      6    40%
	 base_python/cdk/__init__.py                    0      0   100%
	 base_python/cdk/abstract_source.py            83     59    29%
	 base_python/cdk/streams/__init__.py            0      0   100%
	 base_python/cdk/streams/auth/__init__.py       0      0   100%
	 base_python/cdk/streams/auth/core.py           8      1    88%
	 base_python/cdk/streams/auth/jwt.py            5      5     0%
	 base_python/cdk/streams/auth/oauth.py         37     26    30%
	 base_python/cdk/streams/auth/token.py          9      4    56%
	 base_python/cdk/streams/core.py               63     32    49%
	 base_python/cdk/streams/exceptions.py         10      2    80%
	 base_python/cdk/streams/http.py               67     33    51%
	 base_python/cdk/streams/rate_limiting.py      30     14    53%
	 base_python/cdk/utils/__init__.py              0      0   100%
	 base_python/cdk/utils/casing.py                4      0   100%
	 base_python/client.py                         56     33    41%
	 base_python/entrypoint.py                     70     56    20%
	 base_python/integration.py                    52     25    52%
	 base_python/logger.py                         33     19    42%
	 base_python/schema_helpers.py                 56     41    27%
	 base_python/source.py                         51     34    33%
	 main_dev.py                                    3      3     0%
	 --------------------------------------------------------------
	 TOTAL                                        660    393    40%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                              Stmts   Miss  Cover
	 -----------------------------------------------------
	 main_dev.py                           6      6     0%
	 source_zendesk_chat/__init__.py       2      2     0%
	 source_zendesk_chat/source.py        18     18     0%
	 source_zendesk_chat/streams.py      144    144     0%
	 -----------------------------------------------------
	 TOTAL                               170    170     0%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------/actions-runner/_work/airbyte/airbyte/airbyte-integrations/connectors/source-zendesk-chat/.venv/lib/python3.8/site-packages/coverage/control.py:761: CoverageWarning: No data was collected. (no-data-collected)
	   self._warn("No data was collected.", slug="no-data-collected")

> Task :airbyte-integrations:connectors:source-zendesk-chat:airbytePythonTest

> Task :airbyte-integrations:bases:base-python:airbyteDocker
Requirement already satisfied: PyYAML==5.4 in /usr/local/lib/python3.7/site-packages (from base-python==0.0.0) (5.4)
Requirement already satisfied: pydantic==1.6.* in /usr/local/lib/python3.7/site-packages (from base-python==0.0.0) (1.6.1)
Requirement already satisfied: airbyte-protocol in /usr/local/lib/python3.7/site-packages (from base-python==0.0.0) (0.0.0)
Collecting jsonschema==3.2.0
  Downloading jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
Collecting requests
  Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting backoff
  Downloading backoff-1.11.1-py2.py3-none-any.whl (13 kB)
Collecting pytest
  Downloading pytest-6.2.5-py3-none-any.whl (280 kB)
Collecting pendulum
  Downloading pendulum-2.1.2-cp37-cp37m-manylinux1_x86_64.whl (155 kB)
Collecting six>=1.11.0
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting importlib-metadata
  Downloading importlib_metadata-4.8.1-py3-none-any.whl (17 kB)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (from jsonschema==3.2.0->base-python==0.0.0) (54.2.0)
Collecting pyrsistent>=0.14.0
  Downloading pyrsistent-0.18.0-cp37-cp37m-manylinux1_x86_64.whl (119 kB)
Collecting attrs>=17.4.0
  Downloading attrs-21.2.0-py2.py3-none-any.whl (53 kB)
Collecting zipp>=0.5
  Downloading zipp-3.6.0-py3-none-any.whl (5.3 kB)
Collecting typing-extensions>=3.6.4
  Downloading typing_extensions-3.10.0.2-py3-none-any.whl (26 kB)
Collecting python-dateutil<3.0,>=2.6
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting pytzdata>=2020.1
  Downloading pytzdata-2020.1-py2.py3-none-any.whl (489 kB)
Collecting iniconfig
  Downloading iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting py>=1.8.2
  Downloading py-1.10.0-py2.py3-none-any.whl (97 kB)
Collecting pluggy<2.0,>=0.12
  Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting packaging
  Downloading packaging-21.2-py3-none-any.whl (40 kB)
Collecting toml
  Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting pyparsing<3,>=2.0.2
  Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting idna<4,>=2.5
  Downloading idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
  Downloading charset_normalizer-2.0.7-py3-none-any.whl (38 kB)
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.7-py2.py3-none-any.whl (138 kB)
Building wheels for collected packages: base-python
  Building wheel for base-python (setup.py): started
  Building wheel for base-python (setup.py): finished with status 'done'
  Created wheel for base-python: filename=base_python-0.0.0-py3-none-any.whl size=23973 sha256=d85ecc9ed5c75877df619b300f9eb627cdff0c4c05a51097aaecf1782fbf97e5
  Stored in directory: /tmp/pip-ephem-wheel-cache-e7ra7xhi/wheels/c9/49/79/6c1c343ee620e764a007095404318d5c86926b58f2e692a49c
Successfully built base-python
Installing collected packages: zipp, typing-extensions, six, pyparsing, importlib-metadata, urllib3, toml, pytzdata, python-dateutil, pyrsistent, py, pluggy, packaging, iniconfig, idna, charset-normalizer, certifi, attrs, requests, pytest, pendulum, jsonschema, backoff, base-python
Successfully installed attrs-21.2.0 backoff-1.11.1 base-python-0.0.0 certifi-2021.10.8 charset-normalizer-2.0.7 idna-3.3 importlib-metadata-4.8.1 iniconfig-1.1.1 jsonschema-3.2.0 packaging-21.2 pendulum-2.1.2 pluggy-1.0.0 py-1.10.0 pyparsing-2.4.7 pyrsistent-0.18.0 pytest-6.2.5 python-dateutil-2.8.2 pytzdata-2020.1 requests-2.26.0 six-1.16.0 toml-0.10.2 typing-extensions-3.10.0.2 urllib3-1.26.7 zipp-3.6.0
�[91mWARNING: You are using pip version 21.0.1; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
�[0mRemoving intermediate container 547346ecfd81
 ---> e3efe760595f
Step 7/14 : ENV AIRBYTE_SPEC_CMD "base-python spec"
 ---> Running in 6a88969c5b60
Removing intermediate container 6a88969c5b60
 ---> c7b425d2d42e
Step 8/14 : ENV AIRBYTE_CHECK_CMD "base-python check"
 ---> Running in a61384e057aa
Removing intermediate container a61384e057aa
 ---> 1cb811c66a33
Step 9/14 : ENV AIRBYTE_DISCOVER_CMD "base-python discover"
 ---> Running in c418d6f922ca
Removing intermediate container c418d6f922ca
 ---> 22247407e3ef
Step 10/14 : ENV AIRBYTE_READ_CMD "base-python read"
 ---> Running in f5094d289631
Removing intermediate container f5094d289631
 ---> 39e29cd13b27
Step 11/14 : ENV AIRBYTE_ENTRYPOINT "/airbyte/base.sh"
 ---> Running in 81203806258c
Removing intermediate container 81203806258c
 ---> 40bc2bdd54fd
Step 12/14 : ENTRYPOINT ["/airbyte/base.sh"]
 ---> Running in 186efb2f7035
Removing intermediate container 186efb2f7035
 ---> 01452b990e0d
Step 13/14 : LABEL io.airbyte.version=0.1.8
 ---> Running in e12a2847bbaa
Removing intermediate container e12a2847bbaa
 ---> a29fc82b4d4d
Step 14/14 : LABEL io.airbyte.name=airbyte/integration-base-python
 ---> Running in 72524365e275
Removing intermediate container 72524365e275
 ---> 4dc2db6b8dcb
[Warning] One or more build-args [JDK_VERSION] were not consumed
Successfully built 4dc2db6b8dcb
Successfully tagged airbyte/integration-base-python:dev

> Task :airbyte-integrations:connectors:source-zendesk-chat:compileJava NO-SOURCE
> Task :airbyte-integrations:connectors:source-zendesk-chat:classes UP-TO-DATE
> Task :airbyte-integrations:connectors:source-zendesk-chat:jar
> Task :airbyte-integrations:connectors:source-zendesk-chat:assemble

> Task :airbyte-integrations:connectors:source-zendesk-chat:airbyteDocker
Execution optimizations have been disabled for task ':airbyte-integrations:connectors:source-zendesk-chat:airbyteDocker' to ensure correctness due to the following reasons:
  - Gradle detected a problem with the following location: '/actions-runner/_work/airbyte/airbyte/airbyte-integrations/connectors/source-zendesk-chat'. Reason: Task ':airbyte-integrations:connectors:source-zendesk-chat:airbyteDocker' uses this output of task ':airbyte-integrations:connectors:source-zendesk-chat:compileTestJava' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.2/userguide/validation_problems.html#implicit_dependency for more details about this problem.
  - Gradle detected a problem with the following location: '/actions-runner/_work/airbyte/airbyte/airbyte-integrations/connectors/source-zendesk-chat'. Reason: Task ':airbyte-integrations:connectors:source-zendesk-chat:airbyteDocker' uses this output of task ':airbyte-integrations:connectors:source-zendesk-chat:processTestResources' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.2/userguide/validation_problems.html#implicit_dependency for more details about this problem.
Sending build context to Docker daemon  54.27kB

Step 1/14 : FROM python:3.7-slim
 ---> 3d94f0f2ca79
Step 2/14 : RUN apt-get update && apt-get install -y bash && rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 134bd93be6f9
Step 3/14 : ENV CODE_PATH="source_zendesk_chat"
 ---> Running in 7619b3948912
Removing intermediate container 7619b3948912
 ---> bcdc57ecdd7b
Step 4/14 : ENV AIRBYTE_IMPL_MODULE="source_zendesk_chat"
 ---> Running in dae5712b48a5
Removing intermediate container dae5712b48a5
 ---> 5fc24face028
Step 5/14 : ENV AIRBYTE_IMPL_PATH="SourceZendeskChat"
 ---> Running in a43aabe0fb73
Removing intermediate container a43aabe0fb73
 ---> 62508375d47c
Step 6/14 : ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main_dev.py"
 ---> Running in af83b37d13cf
Removing intermediate container af83b37d13cf
 ---> 1208d07ef77a
Step 7/14 : WORKDIR /airbyte/integration_code
 ---> Running in a92e339d902e
Removing intermediate container a92e339d902e
 ---> 4b3ac205a699
Step 8/14 : COPY $CODE_PATH ./$CODE_PATH
 ---> 15a6dd0581bd
Step 9/14 : COPY main_dev.py ./
 ---> bc8472d95893
Step 10/14 : COPY setup.py ./
 ---> dba114b6ea3f
Step 11/14 : RUN pip install .
 ---> Running in 78f0645030ec
Processing /airbyte/integration_code
�[91m  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
�[0mCollecting airbyte-cdk~=0.1
  Downloading airbyte_cdk-0.1.32-py3-none-any.whl (66 kB)
Collecting pendulum==1.2.0
  Downloading pendulum-1.2.0.tar.gz (70 kB)
Collecting requests==2.25.1
  Downloading requests-2.25.1-py2.py3-none-any.whl (61 kB)
Collecting tzlocal
  Downloading tzlocal-4.1-py3-none-any.whl (19 kB)
Collecting python-dateutil
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting pytzdata
  Downloading pytzdata-2020.1-py2.py3-none-any.whl (489 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting idna<3,>=2.5
  Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.7-py2.py3-none-any.whl (138 kB)
Collecting chardet<5,>=3.0.2
  Downloading chardet-4.0.0-py2.py3-none-any.whl (178 kB)
Collecting Deprecated~=1.2
  Downloading Deprecated-1.2.13-py2.py3-none-any.whl (9.6 kB)
Collecting pydantic~=1.6
  Downloading pydantic-1.8.2-cp37-cp37m-manylinux2014_x86_64.whl (10.1 MB)
Collecting jsonref~=0.2
  Downloading jsonref-0.2-py3-none-any.whl (9.3 kB)
Collecting PyYAML~=5.4
  Downloading PyYAML-5.4.1-cp37-cp37m-manylinux1_x86_64.whl (636 kB)
Collecting vcrpy
  Downloading vcrpy-4.1.1-py2.py3-none-any.whl (40 kB)
Collecting jsonschema~=3.2.0
  Downloading jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
Collecting backoff
  Downloading backoff-1.11.1-py2.py3-none-any.whl (13 kB)
Collecting wrapt<2,>=1.10
  Downloading wrapt-1.13.3-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (79 kB)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (from jsonschema~=3.2.0->airbyte-cdk~=0.1->source-zendesk-chat==0.0.0) (57.0.0)
Collecting importlib-metadata
  Downloading importlib_metadata-4.8.1-py3-none-any.whl (17 kB)
Collecting attrs>=17.4.0
  Downloading attrs-21.2.0-py2.py3-none-any.whl (53 kB)
Collecting six>=1.11.0
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting pyrsistent>=0.14.0
  Downloading pyrsistent-0.18.0-cp37-cp37m-manylinux1_x86_64.whl (119 kB)
Collecting typing-extensions>=3.7.4.3
  Downloading typing_extensions-3.10.0.2-py3-none-any.whl (26 kB)
Collecting zipp>=0.5
  Downloading zipp-3.6.0-py3-none-any.whl (5.3 kB)
Collecting pytz-deprecation-shim
  Downloading pytz_deprecation_shim-0.1.0.post0-py2.py3-none-any.whl (15 kB)
Collecting backports.zoneinfo
  Downloading backports.zoneinfo-0.2.1-cp37-cp37m-manylinux1_x86_64.whl (70 kB)
Collecting tzdata
  Downloading tzdata-2021.5-py2.py3-none-any.whl (339 kB)
Collecting yarl
  Downloading yarl-1.7.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (271 kB)
Collecting multidict>=4.0
  Downloading multidict-5.2.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (160 kB)
Building wheels for collected packages: source-zendesk-chat, pendulum
  Building wheel for source-zendesk-chat (setup.py): started
  Building wheel for source-zendesk-chat (setup.py): finished with status 'done'
  Created wheel for source-zendesk-chat: filename=source_zendesk_chat-0.0.0-py3-none-any.whl size=12389 sha256=dedf9c43fda721ba8ea59d887574518c280898627f7b5f362c50c23e6848c68e
  Stored in directory: /tmp/pip-ephem-wheel-cache-726vj8mt/wheels/ae/94/dc/1c707c3e6652ba466c448a71a5a8bc364aeb95358e9f9979e6
  Building wheel for pendulum (setup.py): started
  Building wheel for pendulum (setup.py): finished with status 'done'
  Created wheel for pendulum: filename=pendulum-1.2.0-py2.py3-none-any.whl size=101641 sha256=0e76d929ea3911395fae3fec6c511d7df82a70b9d12e058ee30eac03417c714e
  Stored in directory: /root/.cache/pip/wheels/24/fd/5a/9ce1034716a841e76491624ae12e672c7bf4f2269ec5a1b905
Successfully built source-zendesk-chat pendulum
Installing collected packages: tzdata, backports.zoneinfo, zipp, typing-extensions, six, pytz-deprecation-shim, multidict, idna, yarl, wrapt, urllib3, tzlocal, PyYAML, pytzdata, python-dateutil, pyrsistent, importlib-metadata, chardet, certifi, attrs, vcrpy, requests, pydantic, pendulum, jsonschema, jsonref, Deprecated, backoff, airbyte-cdk, source-zendesk-chat
Successfully installed Deprecated-1.2.13 PyYAML-5.4.1 airbyte-cdk-0.1.32 attrs-21.2.0 backoff-1.11.1 backports.zoneinfo-0.2.1 certifi-2021.10.8 chardet-4.0.0 idna-2.10 importlib-metadata-4.8.1 jsonref-0.2 jsonschema-3.2.0 multidict-5.2.0 pendulum-1.2.0 pydantic-1.8.2 pyrsistent-0.18.0 python-dateutil-2.8.2 pytz-deprecation-shim-0.1.0.post0 pytzdata-2020.1 requests-2.25.1 six-1.16.0 source-zendesk-chat-0.0.0 typing-extensions-3.10.0.2 tzdata-2021.5 tzlocal-4.1 urllib3-1.26.7 vcrpy-4.1.1 wrapt-1.13.3 yarl-1.7.0 zipp-3.6.0
�[91mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
�[0m�[91mWARNING: You are using pip version 21.1.3; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
�[0mRemoving intermediate container 78f0645030ec
 ---> 4589aa22095a
Step 12/14 : ENTRYPOINT ["python", "/airbyte/integration_code/main_dev.py"]
 ---> Running in 8c52cdefed2f
Removing intermediate container 8c52cdefed2f
 ---> 1428fc7b416f
Step 13/14 : LABEL io.airbyte.version=0.1.3
 ---> Running in 9e15c5b89bba
Removing intermediate container 9e15c5b89bba
 ---> 2e16a25d0c93
Step 14/14 : LABEL io.airbyte.name=airbyte/source-zendesk-chat
 ---> Running in 8c967b61962f
Removing intermediate container 8c967b61962f
 ---> 541e54ac3660
[Warning] One or more build-args [JDK_VERSION] were not consumed
Successfully built 541e54ac3660
Successfully tagged airbyte/source-zendesk-chat:dev

> Task :airbyte-integrations:connectors:source-zendesk-chat:compileTestJava NO-SOURCE
> Task :airbyte-integrations:connectors:source-zendesk-chat:testClasses UP-TO-DATE
> Task :airbyte-integrations:connectors:source-zendesk-chat:test NO-SOURCE
> Task :airbyte-integrations:connectors:source-zendesk-chat:jacocoTestReport SKIPPED
> Task :airbyte-integrations:connectors:source-zendesk-chat:check UP-TO-DATE
> Task :airbyte-integrations:connectors:source-zendesk-chat:build

> Task :airbyte-integrations:connectors:source-zendesk-chat:sourceAcceptanceTest
============================= test session starts ==============================
platform linux -- Python 3.7.11, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /test_input
plugins: timeout-1.4.2, sugar-0.9.4
collected 14 items

test_core.py ...........                                                 [ 78%]
test_full_refresh.py .                                                   [ 85%]
test_incremental.py .s                                                   [100%]

======================== 13 passed, 1 skipped in 28.11s ========================

> Task :airbyte-integrations:connectors:source-zendesk-chat:integrationTest

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.2/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD SUCCESSFUL in 2m 29s
90 actionable tasks: 74 executed, 16 up-to-date

Publishing build scan...
https://gradle.com/s/utx3iebwcu2gi

@harshithmullapudi harshithmullapudi temporarily deployed to more-secrets November 1, 2021 12:25 Inactive
@jrhizor jrhizor temporarily deployed to more-secrets November 1, 2021 12:27 Inactive
@harshithmullapudi
Copy link
Contributor Author

harshithmullapudi commented Nov 1, 2021

/publish connector=connectors/source-zendesk-chat

🕑 connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1407890681
✅ connectors/source-zendesk-chat https://github.com/airbytehq/airbyte/actions/runs/1407890681

@jrhizor jrhizor temporarily deployed to more-secrets November 1, 2021 12:35 Inactive
@harshithmullapudi harshithmullapudi merged commit a80d388 into master Nov 1, 2021
@harshithmullapudi harshithmullapudi deleted the harshith/zendesk-fix-chats branch November 1, 2021 12:48
schlattk pushed a commit to schlattk/airbyte that referenced this pull request Jan 4, 2022
airbytehq#7210)

* fix: chats stream is only pulling for first page

* fix: integration tests

* fix: remove cred

* bump version in source definition

* Fixed: integration tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Source Zendesk: Chats api is only pulling 40 records
5 participants