Skip to content

Commit

Permalink
#12013 expose isDataGOlden flag: reword docs
Browse files Browse the repository at this point in the history
  • Loading branch information
@davydov-d
davydov-d committed May 3, 2022
1 parent 37fc7f1 commit cebb43b
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion docs/integrations/sources/google-analytics-v4.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,9 @@ When sampling occurs, a warning is logged to the sync log.
## IsDataGolden

Google Analytics API may return provisional or incomplete data. When this occurs, the returned data will set the flag `isDataGolden` to false, and the connector will log a warning to the sync log.
The connector always syncs data from 2 days ago due to the fact it [takes](https://support.google.com/analytics/answer/1070983?hl=en#DataProcessingLatency&zippy=%2Cin-this-article) Google Analytics up to 48 hours to update the data. To determine whether data is finished processing or not, the `isDataGolden` flag is exposed and should be used.
The connector adds a lookback window of 2 days to ensure any previously synced non-golden data is re-synced with its potential updates. This is done because [Google Analytics takes up to 48 hours](https://support.google.com/analytics/answer/1070983?hl=en#DataProcessingLatency&zippy=%2Cin-this-article) to update the data. For example:
- If your last sync occurred 5 days ago and a sync kicks off today, it will attempt to sync data from 7 days ago up to the latest data available.
To determine whether data is finished processing or not, the `isDataGolden` flag is exposed and should be used.

## Reading Custom Reports

Expand Down

1 comment on commit cebb43b

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SonarQube Report

SonarQube report for Airbyte Connectors Source Google Analytics V4(#12426)

Measures

Name Value Name Value Name Value
Security Rating A Coverage 100.0 Duplicated Blocks 0
Vulnerabilities 0 Reliability Rating A Lines to Cover 2
Duplicated Lines (%) 0.0 Quality Gate Status OK Code Smells 16
Lines of Code 340 Bugs 0 Blocker Issues 0
Critical Issues 1 Major Issues 5 Minor Issues 10

Detected Issues

Rule File Description Message
python:mypy_attr_defined (MINOR) source_google_analytics_v4/source.py:232 Check that attribute exists Module has no attribute "parse" . Code line: prev_end_date = pendulum.parse(stream_state.get(self.curso...
python:mypy_list_item (MINOR) source_google_analytics_v4/source.py:248 Check list items in a list expression [item, ...] List item 0 has incompatible type "None"; expected "Dict[str, str]" . Code line: return date_slices or [None]
python:mypy_no_any_return (MINOR) source_google_analytics_v4/source.py:252 Reject returning value with "Any" type if return type is not "Any" Returning Any from function declared to return "List[MutableMapping[Any, Any]]" . Code line: return report_body.get("data", {}).get("rows", [])
python:mypy_no_any_return (MINOR) source_google_analytics_v4/source.py:432 Reject returning value with "Any" type if return type is not "Any" Returning Any from function declared to return "Iterable[Mapping[str, Any]]" . Code line: return super().read_records(sync_mode, cursor_field, stream_sl...
python:mypy_return (MINOR) source_google_analytics_v4/source.py:290 Check that function always returns a value Missing return statement . Code line: def lookup_data_format(attribute: str) -> Union[str, None]:
python:S1192 (CRITICAL) source_google_analytics_v4/source.py:291 String literals should not be duplicated Define a constant instead of duplicating this literal "ga:date" 4 times.
python:mypy_attr_defined (MINOR) source_google_analytics_v4/source.py:513 Check that attribute exists Module has no attribute "parse" . Code line: start_date = pendulum.parse(self.start_date).date()
python:mypy_arg_type (MINOR) source_google_analytics_v4/source.py:579 Check argument types in calls Argument 1 to "loads" has incompatible type "Optional[bytes]"; expected "Union[str, bytes]" . Code line: reports = json.loads(pkgutil.get_data("source_google_analytics...
python:mypy_import (MINOR) source_google_analytics_v4/source.py:16 Require that imported module can be found or has stubs Library stubs not installed for "requests" (or incompatible with Python 3.9) . Code line: import requests
python:S112 (MAJOR) source_google_analytics_v4/source.py:81 "Exception" and "BaseException" should not be raised Replace this generic exception class with a more specific one.
python:mypy_return (MINOR) source_google_analytics_v4/source.py:132 Check that function always returns a value Missing return statement . Code line: def next_page_token(self, response: requests.Response) -> Optional...
python:mypy_attr_defined (MINOR) source_google_analytics_v4/source.py:230 Check that attribute exists Module has no attribute "parse" . Code line: start_date = pendulum.parse(self.start_date).date()
python:S1871 (MAJOR) source_google_analytics_v4/source.py:275 Two branches in a conditional structure should not have exactly the same implementation Either merge this branch with the identical one on line "272" or change one of the implementations.
python:S1871 (MAJOR) source_google_analytics_v4/source.py:277 Two branches in a conditional structure should not have exactly the same implementation Either merge this branch with the identical one on line "272" or change one of the implementations.
python:S112 (MAJOR) source_google_analytics_v4/source.py:466 "Exception" and "BaseException" should not be raised Replace this generic exception class with a more specific one.
python:S112 (MAJOR) source_google_analytics_v4/source.py:471 "Exception" and "BaseException" should not be raised Replace this generic exception class with a more specific one.

Coverage (100.0%)

File Coverage File Coverage
source_google_analytics_v4/init.py 100.0 source_google_analytics_v4/source.py 91.7

Please sign in to comment.