🐛 Source LinkedIn-Ads: Fix for `Campaigns/targetingCriteria` transformation, coerced `Creatives/variables/values` to string #6610

bazarnov · 2021-10-01T11:15:22Z

What

#6527 - New LinkedIn Ads Source Connector fails with Analytics Streams

How

edited utils.py transform_targeting_criteria() method.
edited utils.py transform_variables() method.
edited schema for creatives stream, ...variables/values to be of type string by default.
edited Dockerfile to fix PEP dependency issues.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions
Connector version bumped like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
Credentials added to Github CI. Instructions.
/test connector=connectors/<name> command is passing.
New Connector version released on Dockerhub by running the /publish command described here

…variables to string by default, bumped the version

…s-analytics-issue

bazarnov · 2021-10-01T11:24:21Z

/test connector=connectors/source-linkedin-ads

🕑 connectors/source-linkedin-ads https://github.com/airbytehq/airbyte/actions/runs/1294843472
✅ connectors/source-linkedin-ads https://github.com/airbytehq/airbyte/actions/runs/1294843472
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 47     20    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     111     11    90%
	 ------------------------------------------------------------------------
	 TOTAL                                                  856    416    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                       Stmts   Miss  Cover
	 --------------------------------------------------------------
	 base_python/__init__.py                       13      0   100%
	 base_python/catalog_helpers.py                10      6    40%
	 base_python/cdk/__init__.py                    0      0   100%
	 base_python/cdk/abstract_source.py            83     59    29%
	 base_python/cdk/streams/__init__.py            0      0   100%
	 base_python/cdk/streams/auth/__init__.py       0      0   100%
	 base_python/cdk/streams/auth/core.py           8      1    88%
	 base_python/cdk/streams/auth/jwt.py            5      5     0%
	 base_python/cdk/streams/auth/oauth.py         37     26    30%
	 base_python/cdk/streams/auth/token.py          9      4    56%
	 base_python/cdk/streams/core.py               63     32    49%
	 base_python/cdk/streams/exceptions.py         10      2    80%
	 base_python/cdk/streams/http.py               67     33    51%
	 base_python/cdk/streams/rate_limiting.py      30     14    53%
	 base_python/cdk/utils/__init__.py              0      0   100%
	 base_python/cdk/utils/casing.py                4      0   100%
	 base_python/client.py                         56     33    41%
	 base_python/entrypoint.py                     70     56    20%
	 base_python/integration.py                    52     25    52%
	 base_python/logger.py                         33     19    42%
	 base_python/schema_helpers.py                 56     41    27%
	 base_python/source.py                         51     34    33%
	 main_dev.py                                    3      3     0%
	 --------------------------------------------------------------
	 TOTAL                                        660    393    40%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                               Stmts   Miss  Cover
	 ------------------------------------------------------
	 source_linkedin_ads/__init__.py        2      0   100%
	 source_linkedin_ads/analytics.py      45      1    98%
	 source_linkedin_ads/source.py        153     66    57%
	 source_linkedin_ads/utils.py          86      5    94%
	 ------------------------------------------------------
	 TOTAL                                286     72    75%

antixar

Several refactoring comments.

airbyte-integrations/connectors/source-linkedin-ads/source_linkedin_ads/utils.py

bazarnov · 2021-10-05T07:55:09Z

/test connector=connectors/source-linkedin-ads

🕑 connectors/source-linkedin-ads https://github.com/airbytehq/airbyte/actions/runs/1306891604
✅ connectors/source-linkedin-ads https://github.com/airbytehq/airbyte/actions/runs/1306891604
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 47     20    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     111     11    90%
	 ------------------------------------------------------------------------
	 TOTAL                                                  856    416    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                       Stmts   Miss  Cover
	 --------------------------------------------------------------
	 base_python/__init__.py                       13      0   100%
	 base_python/catalog_helpers.py                10      6    40%
	 base_python/cdk/__init__.py                    0      0   100%
	 base_python/cdk/abstract_source.py            83     59    29%
	 base_python/cdk/streams/__init__.py            0      0   100%
	 base_python/cdk/streams/auth/__init__.py       0      0   100%
	 base_python/cdk/streams/auth/core.py           8      1    88%
	 base_python/cdk/streams/auth/jwt.py            5      5     0%
	 base_python/cdk/streams/auth/oauth.py         37     26    30%
	 base_python/cdk/streams/auth/token.py          9      4    56%
	 base_python/cdk/streams/core.py               63     32    49%
	 base_python/cdk/streams/exceptions.py         10      2    80%
	 base_python/cdk/streams/http.py               67     33    51%
	 base_python/cdk/streams/rate_limiting.py      30     14    53%
	 base_python/cdk/utils/__init__.py              0      0   100%
	 base_python/cdk/utils/casing.py                4      0   100%
	 base_python/client.py                         56     33    41%
	 base_python/entrypoint.py                     70     56    20%
	 base_python/integration.py                    52     25    52%
	 base_python/logger.py                         33     19    42%
	 base_python/schema_helpers.py                 56     41    27%
	 base_python/source.py                         51     34    33%
	 main_dev.py                                    3      3     0%
	 --------------------------------------------------------------
	 TOTAL                                        660    393    40%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                               Stmts   Miss  Cover
	 ------------------------------------------------------
	 source_linkedin_ads/__init__.py        2      0   100%
	 source_linkedin_ads/analytics.py      45      1    98%
	 source_linkedin_ads/source.py        153     66    57%
	 source_linkedin_ads/utils.py          82      5    94%
	 ------------------------------------------------------
	 TOTAL                                282     72    74%

bazarnov · 2021-10-06T08:45:50Z

/test connector=connectors/source-linkedin-ads

🕑 connectors/source-linkedin-ads https://github.com/airbytehq/airbyte/actions/runs/1311119674
✅ connectors/source-linkedin-ads https://github.com/airbytehq/airbyte/actions/runs/1311119674
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 47     20    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     111     11    90%
	 ------------------------------------------------------------------------
	 TOTAL                                                  856    416    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                       Stmts   Miss  Cover
	 --------------------------------------------------------------
	 base_python/__init__.py                       13      0   100%
	 base_python/catalog_helpers.py                10      6    40%
	 base_python/cdk/__init__.py                    0      0   100%
	 base_python/cdk/abstract_source.py            83     59    29%
	 base_python/cdk/streams/__init__.py            0      0   100%
	 base_python/cdk/streams/auth/__init__.py       0      0   100%
	 base_python/cdk/streams/auth/core.py           8      1    88%
	 base_python/cdk/streams/auth/jwt.py            5      5     0%
	 base_python/cdk/streams/auth/oauth.py         37     26    30%
	 base_python/cdk/streams/auth/token.py          9      4    56%
	 base_python/cdk/streams/core.py               63     32    49%
	 base_python/cdk/streams/exceptions.py         10      2    80%
	 base_python/cdk/streams/http.py               67     33    51%
	 base_python/cdk/streams/rate_limiting.py      30     14    53%
	 base_python/cdk/utils/__init__.py              0      0   100%
	 base_python/cdk/utils/casing.py                4      0   100%
	 base_python/client.py                         56     33    41%
	 base_python/entrypoint.py                     70     56    20%
	 base_python/integration.py                    52     25    52%
	 base_python/logger.py                         33     19    42%
	 base_python/schema_helpers.py                 56     41    27%
	 base_python/source.py                         51     34    33%
	 main_dev.py                                    3      3     0%
	 --------------------------------------------------------------
	 TOTAL                                        660    393    40%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                               Stmts   Miss  Cover
	 ------------------------------------------------------
	 source_linkedin_ads/__init__.py        2      0   100%
	 source_linkedin_ads/analytics.py      45      1    98%
	 source_linkedin_ads/source.py        153     66    57%
	 source_linkedin_ads/utils.py          83      5    94%
	 ------------------------------------------------------
	 TOTAL                                283     72    75%

antixar

looks nice!

bazarnov · 2021-10-06T10:24:25Z

/publish connector=connectors/source-linkedin-ads

🕑 connectors/source-linkedin-ads https://github.com/airbytehq/airbyte/actions/runs/1311460076
✅ connectors/source-linkedin-ads https://github.com/airbytehq/airbyte/actions/runs/1311460076

…mation, coerced `Creatives/variables/values` to string (airbytehq#6610) * fixed targettingCriteria transformation, changed values of creatives/variables to string by default, bumped the version

bazarnov added 3 commits October 1, 2021 13:48

fixed targettingCriteria transformation, changed values of creatives/…

2220772

…variables to string by default, bumped the version

fixed PEP issue for Dockerfile

befa6cb

Merge remote-tracking branch 'origin/master' into bazarnov/6527-li-ad…

ab767b5

…s-analytics-issue

bazarnov self-assigned this Oct 1, 2021

github-actions bot added the area/connectors Connector related issues label Oct 1, 2021

bazarnov linked an issue Oct 1, 2021 that may be closed by this pull request

New LinkedIn Ads Source Connector fails with Analytics Streams #6527

Closed

added changelog

c917339

github-actions bot added the area/documentation Improvements or additions to documentation label Oct 1, 2021

bazarnov temporarily deployed to more-secrets October 1, 2021 11:18 Inactive

jrhizor temporarily deployed to more-secrets October 1, 2021 11:26 Inactive

bazarnov requested review from antixar and lazebnyi October 1, 2021 11:46

antixar reviewed Oct 4, 2021

View reviewed changes

updated after review

692110a

bazarnov temporarily deployed to more-secrets October 4, 2021 15:04 Inactive

jrhizor temporarily deployed to more-secrets October 4, 2021 15:05 Inactive

...

5e55f79

bazarnov temporarily deployed to more-secrets October 5, 2021 07:56 Inactive

jrhizor temporarily deployed to more-secrets October 5, 2021 07:57 Inactive

bazarnov requested a review from antixar October 5, 2021 08:04

updated after review

79140a1

bazarnov temporarily deployed to more-secrets October 6, 2021 08:47 Inactive

jrhizor temporarily deployed to more-secrets October 6, 2021 08:48 Inactive

antixar approved these changes Oct 6, 2021

View reviewed changes

jrhizor temporarily deployed to more-secrets October 6, 2021 10:26 Inactive

bazarnov temporarily deployed to more-secrets October 6, 2021 10:35 Inactive

bazarnov merged commit 64d7cf1 into master Oct 6, 2021

bazarnov deleted the bazarnov/6527-li-ads-analytics-issue branch October 6, 2021 10:57

jrhizor mentioned this pull request Oct 7, 2021

Bump Airbyte version from 0.30.15-alpha to 0.30.16-alpha #6880

Merged

lazebnyi removed their request for review May 30, 2022 10:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🐛 Source LinkedIn-Ads: Fix for `Campaigns/targetingCriteria` transformation, coerced `Creatives/variables/values` to string #6610

🐛 Source LinkedIn-Ads: Fix for `Campaigns/targetingCriteria` transformation, coerced `Creatives/variables/values` to string #6610

bazarnov commented Oct 1, 2021

bazarnov commented Oct 1, 2021 •

edited by github-actions bot

Loading

antixar left a comment

bazarnov commented Oct 5, 2021 •

edited by github-actions bot

Loading

bazarnov commented Oct 6, 2021 •

edited by github-actions bot

Loading

antixar left a comment

bazarnov commented Oct 6, 2021 •

edited by github-actions bot

Loading

🐛 Source LinkedIn-Ads: Fix for Campaigns/targetingCriteria transformation, coerced Creatives/variables/values to string #6610

🐛 Source LinkedIn-Ads: Fix for Campaigns/targetingCriteria transformation, coerced Creatives/variables/values to string #6610

Conversation

bazarnov commented Oct 1, 2021

What

How

Pre-merge Checklist

Community member or Airbyter

Airbyter

bazarnov commented Oct 1, 2021 • edited by github-actions bot Loading

antixar left a comment

Choose a reason for hiding this comment

bazarnov commented Oct 5, 2021 • edited by github-actions bot Loading

bazarnov commented Oct 6, 2021 • edited by github-actions bot Loading

antixar left a comment

Choose a reason for hiding this comment

bazarnov commented Oct 6, 2021 • edited by github-actions bot Loading

🐛 Source LinkedIn-Ads: Fix for `Campaigns/targetingCriteria` transformation, coerced `Creatives/variables/values` to string #6610

🐛 Source LinkedIn-Ads: Fix for `Campaigns/targetingCriteria` transformation, coerced `Creatives/variables/values` to string #6610

bazarnov commented Oct 1, 2021 •

edited by github-actions bot

Loading

bazarnov commented Oct 5, 2021 •

edited by github-actions bot

Loading

bazarnov commented Oct 6, 2021 •

edited by github-actions bot

Loading

bazarnov commented Oct 6, 2021 •

edited by github-actions bot

Loading