Source Facebook Marketing: remove unsupported fields (#20402)

* Source Facebook Marketing: remove unsupported fields * Source Facebook Marketing: fix * Source Facebook Marketing: refactor; raise error * Source Facebook marketing: docs update * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
airbytehq · Dec 15, 2022 · 6bf2448 · 6bf2448
1 parent f1546c2
commit 6bf2448
Show file tree

Hide file tree

Showing 5 changed files with 14 additions and 4 deletions.
diff --git a/airbyte-config/init/src/main/resources/seed/source_definitions.yaml b/airbyte-config/init/src/main/resources/seed/source_definitions.yaml
@@ -451,7 +451,7 @@
 - name: Facebook Marketing
   sourceDefinitionId: e7778cfc-e97c-4458-9ecb-b4f2bba8946c
   dockerRepository: airbyte/source-facebook-marketing
-  dockerImageTag: 0.2.78
+  dockerImageTag: 0.2.79
   documentationUrl: https://docs.airbyte.com/integrations/sources/facebook-marketing
   icon: facebook.svg
   sourceType: api

diff --git a/airbyte-config/init/src/main/resources/seed/source_specs.yaml b/airbyte-config/init/src/main/resources/seed/source_specs.yaml
@@ -3474,7 +3474,7 @@
     supportsNormalization: false
     supportsDBT: false
     supported_destination_sync_modes: []
-- dockerImage: "airbyte/source-facebook-marketing:0.2.78"
+- dockerImage: "airbyte/source-facebook-marketing:0.2.79"
   spec:
     documentationUrl: "https://docs.airbyte.com/integrations/sources/facebook-marketing"
     changelogUrl: "https://docs.airbyte.com/integrations/sources/facebook-marketing"

diff --git a/airbyte-integrations/connectors/source-facebook-marketing/Dockerfile b/airbyte-integrations/connectors/source-facebook-marketing/Dockerfile
@@ -13,5 +13,5 @@ ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
 ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
 
 
-LABEL io.airbyte.version=0.2.78
+LABEL io.airbyte.version=0.2.79
 LABEL io.airbyte.name=airbyte/source-facebook-marketing
diff --git a/...yte-integrations/connectors/source-facebook-marketing/source_facebook_marketing/source.py b/...yte-integrations/connectors/source-facebook-marketing/source_facebook_marketing/source.py
@@ -35,6 +35,7 @@
 from .utils import validate_end_date, validate_start_date
 
 logger = logging.getLogger("airbyte")
+UNSUPPORTED_FIELDS = {"unique_conversions", "unique_ctr", "unique_clicks"}
 
 
 class SourceFacebookMarketing(AbstractSource):
@@ -183,10 +184,18 @@ def get_custom_insights_streams(self, api: API, config: ConnectorConfig) -> List
         """return custom insights streams"""
         streams = []
         for insight in config.custom_insights or []:
+            insight_fields = set(insight.fields)
+            if insight_fields.intersection(UNSUPPORTED_FIELDS):
+                # https://github.com/airbytehq/oncall/issues/1137
+                mes = (
+                    f"Please remove Following fields from the Custom {insight.name} fields list due to possible "
+                    f"errors on Facebook side: {insight_fields.intersection(UNSUPPORTED_FIELDS)}"
+                )
+                raise ValueError(mes)
             stream = AdsInsights(
                 api=api,
                 name=f"Custom{insight.name}",
-                fields=list(set(insight.fields)),
+                fields=list(insight_fields),
                 breakdowns=list(set(insight.breakdowns)),
                 action_breakdowns=list(set(insight.action_breakdowns)),
                 action_breakdowns_allow_empty=config.action_breakdowns_allow_empty,

diff --git a/docs/integrations/sources/facebook-marketing.md b/docs/integrations/sources/facebook-marketing.md
@@ -131,6 +131,7 @@ Please be informed that the connector uses the `lookback_window` parameter to pe
 
 | Version | Date       | Pull Request                                             | Subject                                                                                                                                                                                                                                                                                           |
 |:--------|:-----------|:---------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| 0.2.79  | 2022-12-07 | [20402](https://github.com/airbytehq/airbyte/pull/20402) | Exclude Not supported fields from request                                                                                                                                                                                                                                                         |
 | 0.2.78  | 2022-12-07 | [20165](https://github.com/airbytehq/airbyte/pull/20165) | fix fields permission error                                                                                                                                                                                                                                                                       |
 | 0.2.77  | 2022-12-06 | [20131](https://github.com/airbytehq/airbyte/pull/20131) | update next cursor value at read start                                                                                                                                                                                                                                                            |
 | 0.2.76  | 2022-12-03 | [20043](https://github.com/airbytehq/airbyte/pull/20043) | Allows `action_breakdowns` to be an empty list - bugfix for #20016                                                                                                                                                                                                                                |