Source Harvest: fix build #20572

davydov-d · 2022-12-16T14:47:50Z

What

Fix build for Source Harvest

How

Introduce an end_replication_date where possible
Set test_strictness_level to high

davydov-d · 2022-12-16T14:54:35Z

/test connector=connectors/source-harvest

🕑 connectors/source-harvest https://github.com/airbytehq/airbyte/actions/runs/3714135688
✅ connectors/source-harvest https://github.com/airbytehq/airbyte/actions/runs/3714135688
Python tests coverage:

Name                         Stmts   Miss  Cover
------------------------------------------------
source_harvest/auth.py          11      0   100%
source_harvest/__init__.py       2      0   100%
source_harvest/source.py        37      1    97%
source_harvest/streams.py      170     16    91%
------------------------------------------------
TOTAL                          220     17    92%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       140      5    96%   87, 93, 238, 242-243
	 source_acceptance_test/conftest.py                     208     92    56%   36, 42-44, 49, 54, 77, 83, 89-91, 110, 115-117, 123-125, 131-132, 137-138, 143, 149, 158-167, 173-178, 193, 217, 248, 254, 262-267, 275-280, 288-301, 306-312, 319-330, 337-353
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              402    115    71%   53, 58, 93-104, 109-116, 120-121, 125-126, 308, 346-363, 376-387, 391-396, 402, 435-440, 478-485, 528-530, 533, 598-606, 618-621, 626, 682-683, 689, 692, 728-738, 751-776
	 source_acceptance_test/tests/test_incremental.py       158     14    91%   52-59, 64-77, 240
	 source_acceptance_test/utils/asserts.py                 39      2    95%   62-63
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       133     33    75%   24-27, 46-47, 50-54, 57-58, 73-75, 78-80, 83-85, 88-90, 93-95, 124-125, 159-161, 208
	 source_acceptance_test/utils/json_schema_helper.py     107     13    88%   30-31, 38, 41, 65-68, 96, 120, 192-194
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1603    336    79%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:377: The previous and actual discovered catalogs are identical.
================== 32 passed, 1 skipped in 1102.31s (0:18:22) ==================

alafanechere

Thanks for enabling high test strictness level!
acceptance-test-config.yml looks good to me.
I have a few non-blocking comments: I'd like clarification about the motivation for introducing replication_end_date in the source.
Please close #19047 and #19160 once this PR is merged.

airbyte-integrations/connectors/source-harvest/source_harvest/source.py

docs/integrations/sources/harvest.md

alafanechere

I'd love the replication_end_date option to be available in the spec for consistency of configurations by SAT with the connector spec. Feel free to use the airbyte_hidden: true property if you want to hide this option from users.

davydov-d · 2022-12-16T15:30:15Z

I'd love the replication_end_date option to be available in the spec for consistency of configurations by SAT with the connector spec. Feel free to use the airbyte_hidden: true property if you want to hide this option from users.

this option completely slipped my mind. will do, thanks!

davydov-d · 2022-12-16T15:45:48Z

/test connector=connectors/source-harvest

🕑 connectors/source-harvest https://github.com/airbytehq/airbyte/actions/runs/3714508708
✅ connectors/source-harvest https://github.com/airbytehq/airbyte/actions/runs/3714508708
Python tests coverage:

Name                         Stmts   Miss  Cover
------------------------------------------------
source_harvest/auth.py          11      0   100%
source_harvest/__init__.py       2      0   100%
source_harvest/source.py        37      1    97%
source_harvest/streams.py      170     16    91%
------------------------------------------------
TOTAL                          220     17    92%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       140      5    96%   87, 93, 238, 242-243
	 source_acceptance_test/conftest.py                     208     92    56%   36, 42-44, 49, 54, 77, 83, 89-91, 110, 115-117, 123-125, 131-132, 137-138, 143, 149, 158-167, 173-178, 193, 217, 248, 254, 262-267, 275-280, 288-301, 306-312, 319-330, 337-353
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              402    115    71%   53, 58, 93-104, 109-116, 120-121, 125-126, 308, 346-363, 376-387, 391-396, 402, 435-440, 478-485, 528-530, 533, 598-606, 618-621, 626, 682-683, 689, 692, 728-738, 751-776
	 source_acceptance_test/tests/test_incremental.py       158     14    91%   52-59, 64-77, 240
	 source_acceptance_test/utils/asserts.py                 39      2    95%   62-63
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       133     33    75%   24-27, 46-47, 50-54, 57-58, 73-75, 78-80, 83-85, 88-90, 93-95, 124-125, 159-161, 208
	 source_acceptance_test/utils/json_schema_helper.py     107     13    88%   30-31, 38, 41, 65-68, 96, 120, 192-194
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1603    336    79%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:377: The previous and actual discovered catalogs are identical.
================== 32 passed, 1 skipped in 1113.77s (0:18:33) ==================

alafanechere

Thanks! Feel free to publish and merge once test pass.

davydov-d · 2022-12-16T16:45:35Z

/publish connector=connectors/source-harvest

🕑 Publishing the following connectors:
connectors/source-harvest
https://github.com/airbytehq/airbyte/actions/runs/3714921624

Connector	Did it publish?	Were definitions generated?
connectors/source-harvest	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

#17506 source Harvest: fix build

a10f9a3

octavia-squidington-iv added area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/harvest labels Dec 16, 2022

#17506 source harvest: upd changelog

3866f49

davydov-d requested review from artem1205, bazarnov, grubberr and a team December 16, 2022 14:50

alafanechere approved these changes Dec 16, 2022

View reviewed changes

airbyte-integrations/connectors/source-harvest/source_harvest/source.py Show resolved Hide resolved

docs/integrations/sources/harvest.md Outdated Show resolved Hide resolved

bazarnov approved these changes Dec 16, 2022

View reviewed changes

alafanechere requested changes Dec 16, 2022

View reviewed changes

#17506 source harvest: introduce replication_end_date in spec

2a953c7

davydov-d requested a review from alafanechere December 16, 2022 15:45

alafanechere approved these changes Dec 16, 2022

View reviewed changes

auto-bump connector version

dcf2adb

octavia-squidington-iii temporarily deployed to more-secrets December 16, 2022 17:26 — with GitHub Actions Inactive

octavia-squidington-iii temporarily deployed to more-secrets December 16, 2022 17:27 — with GitHub Actions Inactive

davydov-d merged commit dcda815 into master Dec 16, 2022

davydov-d deleted the ddavydov/#17506-source-harvest-fix-build branch December 16, 2022 18:34

This was referenced Dec 16, 2022

Source Harvest: enable high test strictness level in SAT #19047

Closed

Source Harvest: enable high test strictness level in SAT #19160

Closed

octavia-squidington-iii mentioned this pull request Dec 18, 2022

Bump Airbyte version from 0.40.25 to 0.40.26 #20630

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Source Harvest: fix build #20572

Source Harvest: fix build #20572

davydov-d commented Dec 16, 2022

davydov-d commented Dec 16, 2022 •

edited by github-actions bot

Loading

alafanechere left a comment

alafanechere left a comment

davydov-d commented Dec 16, 2022

davydov-d commented Dec 16, 2022 •

edited by github-actions bot

Loading

alafanechere left a comment

davydov-d commented Dec 16, 2022 •

edited by github-actions bot

Loading

Source Harvest: fix build #20572

Source Harvest: fix build #20572

Conversation

davydov-d commented Dec 16, 2022

What

How

davydov-d commented Dec 16, 2022 • edited by github-actions bot Loading

Build Passed

alafanechere left a comment

Choose a reason for hiding this comment

alafanechere left a comment

Choose a reason for hiding this comment

davydov-d commented Dec 16, 2022

davydov-d commented Dec 16, 2022 • edited by github-actions bot Loading

Build Passed

alafanechere left a comment

Choose a reason for hiding this comment

davydov-d commented Dec 16, 2022 • edited by github-actions bot Loading

davydov-d commented Dec 16, 2022 •

edited by github-actions bot

Loading

davydov-d commented Dec 16, 2022 •

edited by github-actions bot

Loading

davydov-d commented Dec 16, 2022 •

edited by github-actions bot

Loading