Google Analytics v4: Added support for segments and filters (#16920)

* Google Analytics v4: Added support for segments and filters

* Bumped version to 0.1.26 and updated docs

* Fix connection test

* Fix merge conflict (connector version)

airbyte-integrations/connectors/source-google-analytics-v4/Dockerfile

@@ -12,5 +12,5 @@ COPY main.py ./
 ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
 ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
 
-LABEL io.airbyte.version=0.1.27
-LABEL io.airbyte.name=airbyte/source-google-analytics-v4
+LABEL io.airbyte.version=0.1.28
+LABEL io.airbyte.name=airbyte/source-google-analytics-v4

airbyte-integrations/connectors/source-google-analytics-v4/integration_tests/configured_catalog_segment_filters.json

@@ -0,0 +1,15 @@
+{
+  "streams": [
+    {
+      "stream": {
+        "name": "new_users_per_day",
+        "json_schema": {},
+        "supported_sync_modes": ["incremental"],
+        "source_defined_cursor": true
+      },
+      "sync_mode": "incremental",
+      "cursor_field": ["ga_date"],
+      "destination_sync_mode": "append"
+    }
+  ]
+}

airbyte-integrations/connectors/source-google-analytics-v4/source_google_analytics_v4/source.py

@@ -103,6 +103,8 @@ def __init__(self, config: MutableMapping):
         self.view_id = config["view_id"]
         self.metrics = config["metrics"]
         self.dimensions = config["dimensions"]
+        self.segments = config.get("segments", list())
+        self.filtersExpression = config.get("filter", "")
         self._config = config
         self.dimensions_ref, self.metrics_ref = GoogleAnalyticsV4TypesList().read_records(sync_mode=None)
 
@@ -167,6 +169,8 @@ def request_body_json(
 
         metrics = [{"expression": metric} for metric in self.metrics]
         dimensions = [{"name": dimension} for dimension in self.dimensions]
+        segments = [{"segmentId": segment} for segment in self.segments]
+        filtersExpression = self.filtersExpression
 
         request_body = {
             "reportRequests": [
@@ -176,6 +180,8 @@ def request_body_json(
                     "pageSize": self.page_size,
                     "metrics": metrics,
                     "dimensions": dimensions,
+                    "segments": segments,
+                    "filtersExpression": filtersExpression,
                 }
             ]
         }
@@ -268,7 +274,7 @@ def lookup_data_type(self, field_type: str, attribute: str) -> str:
         """
         try:
             if field_type == "dimension":
-                if attribute.startswith(("ga:dimension", "ga:customVarName", "ga:customVarValue")):
+                if attribute.startswith(("ga:dimension", "ga:customVarName", "ga:customVarValue", "ga:segment")):
                     # Custom Google Analytics Dimensions that are not part of self.dimensions_ref. They are always
                     # strings
                     return "string"
@@ -602,6 +608,8 @@ def streams(self, config: MutableMapping[str, Any]) -> List[Stream]:
         for stream in config["ga_streams"]:
             config["metrics"] = stream["metrics"]
             config["dimensions"] = stream["dimensions"]
+            config["segments"] = stream.get("segments", list())
+            config["filter"] = stream.get("filter", "")
 
             # construct GAReadStreams sub-class for each stream
             stream_name = stream["name"]

docs/integrations/sources/google-analytics-v4.md

@@ -9,10 +9,19 @@ This connector supports GA4 properties through the [Analytics Data API v1](https
 * JSON credentials for the service account that has access to Google Analytics. For more details check [instructions](https://support.google.com/analytics/answer/1009702#zippy=%2Cin-this-article)
 * OAuth 2.0 credentials for the service account that has access to Google Analytics
 * Property ID
-* Custom reports in format `{"name": "<report-name>", "dimensions": ["<dimension-name>", ...], "metrics": ["metric-name", ...]}`
 * Date Range Start Date
 * Data request time increment in days (Optional)
 
+## Custom reports
+
+* Support for multiple custom reports
+* Custom reports in format `[{"name": "<report-name>", "dimensions": ["<dimension-name>", ...], "metrics": ["<metric-name>", ...]}]`
+* Custom report format when using segments and / or filters `[{"name": "<report-name>", "dimensions": ["<dimension-name>", ...], "metrics": ["<metric-name>", ...], "segments":  ["<segment-id-or-dynamic-segment-v3-format]", filter: "<filter-definition-v3-format>"}]`
+* When using segments, make sure you add the `ga:segment` dimension.
+* Custom reports: [Dimensions and metrics explorer](https://ga-dev-tools.web.app/dimensions-metrics-explorer/)
+* Custom reports: [Segment](https://developers.google.com/analytics/devguides/reporting/core/v3/reference#segment) and [filter](https://developers.google.com/analytics/devguides/reporting/core/v3/reference#filters) definition guide (v3 format)
+* Example: `[{"name": "sessions_example", "dimensions": ["ga:date", "ga:segment", "ga:deviceCategory"], "metrics": ["ga:pageviews","ga:sessions","ga:users","ga:transactions"], "segments": ["sessions::condition::ga:browser==Chrome"], "filter": "ga:deviceCategory==desktop"}]`
+
 ## Step 1: Set up Source
 
 ### Create a Service Account
@@ -63,6 +72,7 @@ added by default to any report. There are 8 default reports. To add more reports
 
 | Version | Date       | Pull Request                                             | Subject                                            |
 |:--------|:-----------|:---------------------------------------------------------|:---------------------------------------------------|
+| 0.0.4   | 2022-09-24 | [16920](https://github.com/airbytehq/airbyte/pull/16920) | Added segments and filters to custom reports       |
 | 0.0.3   | 2022-08-15 | [15229](https://github.com/airbytehq/airbyte/pull/15229) | Source Google Analytics Data Api: code refactoring |
 | 0.0.2   | 2022-07-27 | [15087](https://github.com/airbytehq/airbyte/pull/15087) | fix documentationUrl                               |
 | 0.0.1   | 2022-05-09 | [12701](https://github.com/airbytehq/airbyte/pull/12701) | Introduce Google Analytics Data API source         |