Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎉 New Source: Kustomer #4550

Merged
merged 5 commits into from
Aug 5, 2021

Conversation

rparrapy
Copy link
Contributor

@rparrapy rparrapy commented Jul 5, 2021

What

Adds a Kustomer connector.

How

Wrapping the existing Singer tap.

Pre-merge Checklist

  • Issue acceptance criteria met
  • PR name follows PR naming conventions
  • Secrets are annotated with airbyte_secret in the connector's spec
  • Credentials added to Github CI if needed and not already present. instructions for injecting secrets into CI.
  • Unit & integration tests added as appropriate (and are passing)
    • Community members: please provide proof of this succeeding locally e.g: screenshot or copy-paste acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • /test connector=connectors/<name> command as documented here is passing.
    • Community members can skip this, Airbyters will run this for you.
  • Code reviews completed
  • Documentation updated
    • README.md
    • docs/SUMMARY.md if it's a new connector
    • Created or updated reference docs in docs/integrations/<source or destination>/<name>.
    • Changelog in the appropriate page in docs/integrations/.... See changelog example
    • docs/integrations/README.md contains a reference to the new connector
    • Build status added to build page
  • Build is successful
  • Connector version bumped like described here
  • New Connector version released on Dockerhub by running the /publish command described here
  • No major blockers
  • PR merged into master branch
  • Follow up tickets have been created
  • Associated tickets have been closed & stakeholders notified

This closes #4004

@github-actions github-actions bot added the area/connectors Connector related issues label Jul 5, 2021
@rparrapy rparrapy changed the title Initial commit for the Kustomer singer-based connector, closes #4004 Kustomer singer-based connector Jul 5, 2021
@rparrapy
Copy link
Contributor Author

rparrapy commented Jul 5, 2021

I cannot get the acceptance tests for incremental syncs in airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_incremental.py to pass, mainly because Singer's state does not conform with what those tests expect.

This is what the connector state looks like:

  {
    "bookmarks": {
      "conversations": "2019-12-26T18:35:41.583Z",
      "customers": "2019-12-26T18:39:38.186Z",
      "kobjects": "2019-12-26T18:24:18.283Z",
      "messages": "2019-12-26T18:39:45.018Z",
      "shortcuts": "2019-12-25T18:00:00Z",
      "tags": "2019-12-25T18:00:00Z",
      "users": "2019-12-25T18:00:00Z"
    },
    "currently_syncing": "teams"
  }

I noticed that the Github source that is referenced as an example for singer-based connectors does not run acceptance tests either. Is the acceptance test criteria relaxed for Singer-based connectors or else please let me know how to proceed to get this merged.

@rparrapy rparrapy changed the title Kustomer singer-based connector Add Kustomer singer-based connector Jul 5, 2021
Copy link
Contributor

@sherifnada sherifnada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM @rparrapy -- for incremental once https://github.com/airbytehq/airbyte/pull/4552/files is merged you should be able to move forward on this issue (you might need to docker pull the acceptance test image)

then can you post the output for the remaining tests?

session.raise_for_status()
return AirbyteConnectionStatus(status=Status.SUCCEEDED)
except requests.exceptions.RequestException as e:
return AirbyteConnectionStatus(status=Status.FAILED, message=f"An exception occurred: {str(e)}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return AirbyteConnectionStatus(status=Status.FAILED, message=f"An exception occurred: {str(e)}")
return AirbyteConnectionStatus(status=Status.FAILED, message=f"An exception occurred: {repr(e)}")

@marcosmarxm marcosmarxm changed the title Add Kustomer singer-based connector 🎉 New Source Kustomer (singer-based) connector Jul 9, 2021
@rparrapy rparrapy force-pushed the add-kustomer-singer-source branch from df1be94 to 4ee2abe Compare July 22, 2021 03:22
@github-actions github-actions bot added the area/documentation Improvements or additions to documentation label Jul 22, 2021
@rparrapy
Copy link
Contributor Author

Finally got some time to wrap this up.

CleanShot 2021-07-21 at 23 05 32@2x

@rparrapy
Copy link
Contributor Author

rparrapy commented Aug 3, 2021

If we could get this merged this week based on the green integration tests that'd be great for my current project.

@sherifnada
Copy link
Contributor

Apologies @rparrapy - have been slammed. Will do it tonight!

@sherifnada sherifnada mentioned this pull request Aug 5, 2021
@jrhizor jrhizor temporarily deployed to more-secrets August 5, 2021 05:45 Inactive
@sherifnada sherifnada changed the title 🎉 New Source Kustomer (singer-based) connector 🎉 New Source: Kustomer Aug 5, 2021
@sherifnada
Copy link
Contributor

@rparrapy this has been published. Thank you for the contribution!! 🎉

@sherifnada sherifnada merged commit de0d6fc into airbytehq:master Aug 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Kustomer connector
4 participants