Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source Hubspot: best practices #2537

Merged
merged 4 commits into from
Mar 19, 2021

Conversation

keu
Copy link
Contributor

@keu keu commented Mar 19, 2021

What

closes #2221

How

Describe the solution

Pre-merge Checklist

  • Run integration tests
  • Publish Docker images

Recommended reading order

  1. test.java
  2. component.ts
  3. the rest

@auto-assign auto-assign bot requested review from cgardens and michel-tricot March 19, 2021 20:55
@keu keu requested review from sherifnada and yevhenii-ldv and removed request for michel-tricot and cgardens March 19, 2021 20:55
@keu keu force-pushed the keu/source-hubspot-best-practices branch from 92a8bd6 to ce9a325 Compare March 19, 2021 20:58
@keu
Copy link
Contributor Author

keu commented Mar 19, 2021

/test connector=source-hubspot

🕑 source-hubspot https://github.com/airbytehq/airbyte/actions/runs/669228060
❌ source-hubspot https://github.com/airbytehq/airbyte/actions/runs/669228060

@keu
Copy link
Contributor Author

keu commented Mar 19, 2021

/test connector=source-hubspot

🕑 source-hubspot https://github.com/airbytehq/airbyte/actions/runs/669264135
❌ source-hubspot https://github.com/airbytehq/airbyte/actions/runs/669264135

return {"api_key": "wrong_key"}


def test__heal_check_with_wrong_token(wrong_credentials):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def test__heal_check_with_wrong_token(wrong_credentials):
def test__healh_check_with_wrong_token(wrong_credentials):

from source_hubspot.errors import HubspotInvalidAuth


@pytest.fixture(name="wrong_credentials")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some of these tests should be integration tests since they are making API calls (or they should be mocked)

@keu
Copy link
Contributor Author

keu commented Mar 19, 2021

/test connector=source-hubspot

🕑 source-hubspot https://github.com/airbytehq/airbyte/actions/runs/669307889
✅ source-hubspot https://github.com/airbytehq/airbyte/actions/runs/669307889

@keu keu merged commit 61c8805 into keu/source-hubspot-native Mar 19, 2021
@keu keu deleted the keu/source-hubspot-best-practices branch March 19, 2021 21:38
keu added a commit that referenced this pull request Mar 20, 2021
* #2150 Issue: created native connector with schema folder populated

* #2150 Issue: make format code

* first version

* fix few issues

* fix issues

* fix read issue

* format

* docs

* docker tags

* extend configured catalog for testing

* fix source definitions

* format

* fix call rate issue, add backoff for retry after

* add general backoff

* write secrets for new connector

* drop singer connector registration

* refactor streams, resolve properties in schemas at runtime

* replace deprecated endpoint for company contacts

* replace deprecated pipeline endpoint

* update comments

* update docs

* fix typo

* fix stream contact lists

* fix pagination and forms result fetching

* fix health_check

* format and update catalog

* revert changes

* drop singer based hubspot

* fix company contacts substream

* move deals to separate test

* fix deals tests

* remove dynamic fields from records

* move deals to catalog again

* extend CRMObjectStream with associations

* format

* update schemas with updated field, change engagement layout

* fix Campaign stream

* remove custom tests

* remove dependency

* remove oauth

* Source Hubspot: add incremental streams (#2425)

* add incremental

* add incremental

* polishing

* update docs

* fix docstring

* clean up

* fix incremental bookmark access

* fix incremental tests

* clean up

* add custom test for incremental, improve logging

* format

* Update airbyte-integrations/connectors/source-hubspot/source_hubspot/api.py

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Source Hubspot: best practices (#2537)

* fix error reporting and add unit tests

* fix test and refactor cursor fields

* format

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>

* restored configured_catalog.json

Co-authored-by: ykurochkin <y.kurochkin@zazmic.com>
Co-authored-by: Eugene Kulak <widowmakerreborn@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Source Hubspot: ensure best practices are applied (rate limiting, tests all streams etc.)
3 participants