Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🎉 Source Linkedin Ads - for adDirectSponsoredContents stream skip accounts which are not part of organization #18111

Merged
merged 6 commits into from
Oct 21, 2022
Merged

🎉 Source Linkedin Ads - for adDirectSponsoredContents stream skip accounts which are not part of organization #18111

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
fed7f03
for adDirectSponsoredContents stream skip accounts which are part of …
midavadim Oct 18, 2022
bfe64e4
updated PR number
midavadim Oct 18, 2022
a6aaae0
updated log message, added unit tests
midavadim Oct 19, 2022
623a36c
additional formatting
midavadim Oct 19, 2022
24c3dd5
updated connector version in source_definitions.yaml
midavadim Oct 19, 2022
dc858e5
auto-bump connector version
octavia-squidington-iii Oct 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion airbyte-config/init/src/main/resources/seed/source_definitions.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -566,7 +566,7 @@
- name: LinkedIn Ads
sourceDefinitionId: 137ece28-5434-455c-8f34-69dc3782f451
dockerRepository: airbyte/source-linkedin-ads
dockerImageTag: 0.1.11
dockerImageTag: 0.1.12
documentationUrl: https://docs.airbyte.com/integrations/sources/linkedin-ads
icon: linkedin.svg
sourceType: api
Expand Down
2 changes: 1 addition & 1 deletion airbyte-config/init/src/main/resources/seed/source_specs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5673,7 +5673,7 @@
path_in_connector_config:
- "credentials"
- "client_secret"
- dockerImage: "airbyte/source-linkedin-ads:0.1.11"
- dockerImage: "airbyte/source-linkedin-ads:0.1.12"
spec:
documentationUrl: "https://docs.airbyte.com/integrations/sources/linkedin-ads"
connectionSpecification:
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-linkedin-ads/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,5 +33,5 @@ COPY source_linkedin_ads ./source_linkedin_ads
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.11
LABEL io.airbyte.version=0.1.12
LABEL io.airbyte.name=airbyte/source-linkedin-ads
18 changes: 18 additions & 0 deletions airbyte-integrations/connectors/source-linkedin-ads/source_linkedin_ads/source.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,24 @@ def request_params(self, stream_state: Mapping[str, Any], stream_slice: Mapping[
params["q"] = self.search_param
return params

def read_records(
self, stream_state: Mapping[str, Any] = None, stream_slice: Optional[Mapping[str, Any]] = None, **kwargs
) -> Iterable[Mapping[str, Any]]:
stream_state = stream_state or {}
parent_stream = self.parent_stream(config=self.config)
for record in parent_stream.read_records(**kwargs):

if record.get("reference", "").startswith("urn:li:person"):
self.logger.warn(
f'Skip {record.get("name")} account, ORGANIZATION permissions required, but referenced to PERSON {record.get("reference")}'
)
continue

child_stream_slice = super(LinkedInAdsStreamSlicing, self).read_records(
stream_slice=get_parent_stream_values(record, self.parent_values_map), **kwargs
)
yield from self.filter_records_newer_than_state(stream_state=stream_state, records_slice=child_stream_slice)


class LinkedInAdsAnalyticsStream(IncrementalLinkedinAdsStream):
"""
Expand Down
13 changes: 13 additions & 0 deletions airbyte-integrations/connectors/source-linkedin-ads/unit_tests/source_tests/test_source.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,19 @@ def test_request_headers(self):
assert result == expected


class TestAccountUsers:
stream: AccountUsers = AccountUsers(TEST_CONFIG)

def test_state_checkpoint_interval(self):
assert self.stream.state_checkpoint_interval == 500

def test_get_updated_state(self):
state = self.stream.get_updated_state(
current_stream_state={"lastModified": "2021-01-01"}, latest_record={"lastModified": "2021-08-01"}
)
assert state == {"lastModified": "2021-08-01"}


class TestLinkedInAdsStreamSlicing:
@pytest.mark.parametrize(
"stream_cls, slice, expected",
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/sources/linkedin-ads.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,7 @@ After 5 unsuccessful attempts - the connector will stop the sync operation. In s

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:---------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------|
| 0.1.12 | 2022-10-18 | [18111](https://github.com/airbytehq/airbyte/pull/18111) | for adDirectSponsoredContents stream skip accounts which are part of organization |
| 0.1.11 | 2022-10-07 | [17724](https://github.com/airbytehq/airbyte/pull/17724) | Retry 429/5xx errors when refreshing access token |
| 0.1.10 | 2022-09-28 | [17326](https://github.com/airbytehq/airbyte/pull/17326) | Migrate to per-stream states. |
| 0.1.9 | 2022-07-21 | [14924](https://github.com/airbytehq/airbyte/pull/14924) | Remove `additionalProperties` field from schemas |
Expand Down