Source Facebook Marketing: Update schema ad_account #21149

artem1205 · 2023-01-09T12:50:05Z

What

Resolving https://github.com/airbytehq/oncall/issues/1276

How

Fix schema

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Secrets in the connector's spec are annotated with airbyte_secret
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Connector's README.md
- Connector's bootstrap.md. See description and examples
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub and connector version bumped by running the /publish command described here

artem1205 · 2023-01-09T13:37:37Z

/test connector=connectors/source-facebook-marketing

🕑 connectors/source-facebook-marketing https://github.com/airbytehq/airbyte/actions/runs/3874316401
✅ connectors/source-facebook-marketing https://github.com/airbytehq/airbyte/actions/runs/3874316401
Python tests coverage:

Name                                                        Stmts   Miss  Cover
-------------------------------------------------------------------------------
source_facebook_marketing/streams/__init__.py                   2      0   100%
source_facebook_marketing/__init__.py                           2      0   100%
source_facebook_marketing/spec.py                              41      1    98%
source_facebook_marketing/api.py                               96     12    88%
source_facebook_marketing/streams/base_streams.py             138     27    80%
source_facebook_marketing/streams/common.py                    44     16    64%
source_facebook_marketing/utils.py                             24      9    62%
source_facebook_marketing/source.py                            58     22    62%
source_facebook_marketing/streams/streams.py                  120     47    61%
source_facebook_marketing/streams/base_insight_streams.py     145     64    56%
source_facebook_marketing/streams/async_job.py                217    136    37%
source_facebook_marketing/streams/async_job_manager.py         78     60    23%
-------------------------------------------------------------------------------
TOTAL                                                         965    394    59%
Name                                                        Stmts   Miss  Cover
-------------------------------------------------------------------------------
source_facebook_marketing/utils.py                             24      0   100%
source_facebook_marketing/streams/common.py                    44      0   100%
source_facebook_marketing/streams/async_job.py                217      0   100%
source_facebook_marketing/streams/__init__.py                   2      0   100%
source_facebook_marketing/spec.py                              41      0   100%
source_facebook_marketing/__init__.py                           2      0   100%
source_facebook_marketing/streams/async_job_manager.py         78      3    96%
source_facebook_marketing/api.py                               96      9    91%
source_facebook_marketing/streams/base_insight_streams.py     145     16    89%
source_facebook_marketing/source.py                            58      7    88%
source_facebook_marketing/streams/base_streams.py             138     30    78%
source_facebook_marketing/streams/streams.py                  120     27    78%
-------------------------------------------------------------------------------
TOTAL                                                         965     92    90%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       141      5    96%   87, 93, 239, 243-244
	 source_acceptance_test/conftest.py                     211     95    55%   36, 42-44, 49, 54, 77, 83, 89-91, 110, 115-117, 123-125, 131-132, 137-138, 143, 149, 158-167, 173-178, 193, 217, 248, 254, 262-267, 275-285, 293-306, 311-317, 324-335, 342-358
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              402    115    71%   53, 58, 93-104, 109-116, 120-121, 125-126, 308, 346-363, 376-387, 391-396, 402, 435-440, 478-485, 528-530, 533, 598-606, 618-621, 626, 682-683, 689, 692, 728-738, 751-776
	 source_acceptance_test/tests/test_incremental.py       160     14    91%   56-63, 68-81, 244
	 source_acceptance_test/utils/asserts.py                 39      2    95%   62-63
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       133     33    75%   24-27, 46-47, 50-54, 57-58, 73-75, 78-80, 83-85, 88-90, 93-95, 124-125, 159-161, 208
	 source_acceptance_test/utils/json_schema_helper.py     107     13    88%   30-31, 38, 41, 65-68, 96, 120, 192-194
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1609    339    79%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:94: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:386: Backward compatibility tests are disabled for version 0.2.81.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_incremental.py:204: Skipping new incremental test based on acceptance-test-config.yml
============ 28 passed, 3 skipped, 31 warnings in 772.53s (0:12:52) ============

artem1205 · 2023-01-09T14:37:10Z

/publish connector=connectors/source-facebook-marketing

🕑 Publishing the following connectors:
connectors/source-facebook-marketing
https://github.com/airbytehq/airbyte/actions/runs/3874800869

Connector	Did it publish?	Were definitions generated?
connectors/source-facebook-marketing	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

alafanechere · 2023-01-09T14:52:42Z

airbyte-integrations/connectors/source-facebook-marketing/acceptance-test-config.yml

@@ -11,6 +11,8 @@ tests:
      status: "failed"
  discovery:
    - config_path: "secrets/config.json"
+      backward_compatibility_tests_config:


I need to perform additional tests to check the backward compatibility

alafanechere

Thanks for the fix.
Users facing normalization errors will have to refresh source schema that will get the right schema for the ad_account stream AND reset their data on save (only the ad_account stream will be reset).
I assume few user will be impacted because it only impacts users having non null value in the extended_credit_invoice_group.

* Source Facebook Marketing: Update schema ad_account * Source Facebook Marketing: Update docs; bump version * Source Facebook Marketing: disable backward compatibility * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

Source Facebook Marketing: Update schema ad_account

0e7056c

octavia-squidington-iv added area/connectors Connector related issues connectors/source/facebook-marketing labels Jan 9, 2023

Source Facebook Marketing: Update docs; bump version

5c2ce6c

octavia-squidington-iv added the area/documentation Improvements or additions to documentation label Jan 9, 2023

Source Facebook Marketing: disable backward compatibility

d26e49d

artem1205 requested review from arsenlosenko, bazarnov and grubberr January 9, 2023 14:10

bazarnov approved these changes Jan 9, 2023

View reviewed changes

arsenlosenko approved these changes Jan 9, 2023

View reviewed changes

alafanechere requested changes Jan 9, 2023

View reviewed changes

auto-bump connector version

ac4a3f6

octavia-squidington-iii temporarily deployed to more-secrets January 9, 2023 15:16 — with GitHub Actions Inactive

alafanechere approved these changes Jan 9, 2023

View reviewed changes

octavia-squidington-iii temporarily deployed to more-secrets January 9, 2023 18:02 — with GitHub Actions Inactive

octavia-squidington-iii temporarily deployed to more-secrets January 9, 2023 18:03 — with GitHub Actions Inactive

artem1205 merged commit bc93198 into master Jan 9, 2023

artem1205 deleted the artem1205/source-facebook-marketing-OC-1276 branch January 9, 2023 18:38

github-actions bot mentioned this pull request Jan 9, 2023

Bump helm chart version reference to 0.43.18 #21172

Closed

octavia-squidington-iii mentioned this pull request Jan 12, 2023

Bump Airbyte version from 0.40.27 to 0.40.28 #21359

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Source Facebook Marketing: Update schema ad_account #21149

Source Facebook Marketing: Update schema ad_account #21149

artem1205 commented Jan 9, 2023 •

edited

Loading

artem1205 commented Jan 9, 2023 •

edited by github-actions bot

Loading

artem1205 commented Jan 9, 2023 •

edited by github-actions bot

Loading

alafanechere Jan 9, 2023

alafanechere left a comment

Source Facebook Marketing: Update schema ad_account #21149

Source Facebook Marketing: Update schema ad_account #21149

Conversation

artem1205 commented Jan 9, 2023 • edited Loading

What

How

Pre-merge Checklist

Community member or Airbyter

Airbyter

artem1205 commented Jan 9, 2023 • edited by github-actions bot Loading

Build Passed

artem1205 commented Jan 9, 2023 • edited by github-actions bot Loading

alafanechere Jan 9, 2023

Choose a reason for hiding this comment

alafanechere left a comment

Choose a reason for hiding this comment

artem1205 commented Jan 9, 2023 •

edited

Loading

artem1205 commented Jan 9, 2023 •

edited by github-actions bot

Loading

artem1205 commented Jan 9, 2023 •

edited by github-actions bot

Loading