Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

migrate GA source connectors to per-stream state #17326

Merged
merged 31 commits into from
Sep 29, 2022
Merged

migrate GA source connectors to per-stream state #17326

Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
beb5906
update state files
alafanechere Sep 28, 2022
f6fc7ec
bump amplitude
alafanechere Sep 28, 2022
6a31e30
source-bing-ads
alafanechere Sep 28, 2022
6ba2825
source-harvest
alafanechere Sep 28, 2022
89ec930
source-intercom
alafanechere Sep 28, 2022
fc28617
source-linked-ads
alafanechere Sep 28, 2022
72c550c
source-mailchimp
alafanechere Sep 28, 2022
25b6b90
source-surveymonkey
alafanechere Sep 28, 2022
86eb224
source-google-analytics-v4
alafanechere Sep 28, 2022
8a4a6f8
source-google-sheets
alafanechere Sep 28, 2022
de0e8b0
source-tiktok-marketing
alafanechere Sep 28, 2022
e745b80
source-zendesk-chat
alafanechere Sep 28, 2022
8360861
source-zendesk-support
alafanechere Sep 28, 2022
1fef511
Merge branch 'master' into augustin/migrate-ga-to-per-stream-state-2
alafanechere Sep 28, 2022
7e9e424
auto-bump connector version [ci skip]
octavia-squidington-iii Sep 28, 2022
a458b8a
auto-bump connector version [ci skip]
octavia-squidington-iii Sep 28, 2022
8cb3104
source-google-ads
alafanechere Sep 28, 2022
a7b78e2
auto-bump connector version [ci skip]
octavia-squidington-iii Sep 28, 2022
9adad92
auto-bump connector version [ci skip]
octavia-squidington-iii Sep 28, 2022
f1c3ea3
auto-bump connector version [ci skip]
octavia-squidington-iii Sep 28, 2022
2d1acaf
auto-bump connector version [ci skip]
octavia-squidington-iii Sep 28, 2022
469bf1b
auto-bump connector version [ci skip]
octavia-squidington-iii Sep 28, 2022
42d44c3
auto-bump connector version [ci skip]
octavia-squidington-iii Sep 28, 2022
8a1156d
auto-bump connector version [ci skip]
octavia-squidington-iii Sep 28, 2022
6213bfc
auto-bump connector version [ci skip]
octavia-squidington-iii Sep 28, 2022
385c45f
auto-bump connector version [ci skip]
octavia-squidington-iii Sep 28, 2022
4edb8fc
Merge branch 'augustin/migrate-ga-to-per-stream-state-2' of github.co…
alafanechere Sep 28, 2022
daa49b7
source-google-ads
alafanechere Sep 28, 2022
bde30a2
revert google-ads bing-ads
alafanechere Sep 29, 2022
8703598
Merge branch 'master' into augustin/migrate-ga-to-per-stream-state-2
alafanechere Sep 29, 2022
097a1e5
Merge branch 'master' into augustin/migrate-ga-to-per-stream-state-2
alafanechere Sep 29, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-amplitude/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ RUN pip install .
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.13
LABEL io.airbyte.version=0.1.14
LABEL io.airbyte.name=airbyte/source-amplitude
28 changes: 20 additions & 8 deletions airbyte-integrations/connectors/source-amplitude/integration_tests/sample_state.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,23 @@
{
"events": {
"event_time": "2021-05-27 11:59:53.710000"
[
{
"type": "STREAM",
"stream": {
"stream_state": { "event_time": "2021-05-27 11:59:53.710000" },
"stream_descriptor": { "name": "events" }
}
},
"active_users": {
"date": "2021-05-27"
{
"type": "STREAM",
"stream": {
"stream_state": { "date": "2021-05-27" },
"stream_descriptor": { "name": "active_users" }
}
},
"average_session_length": {
"date": "2021-05-27"
{
"type": "STREAM",
"stream": {
"stream_state": { "date": "2021-05-27" },
"stream_descriptor": { "name": "average_session_length" }
}
}
}
]
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-bing-ads/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ RUN pip install .
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.12
LABEL io.airbyte.version=0.1.13
LABEL io.airbyte.name=airbyte/source-bing-ads
36 changes: 22 additions & 14 deletions airbyte-integrations/connectors/source-bing-ads/integration_tests/state.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,22 +1,30 @@
{
"keyword_performance_report_hourly": {
"180278106": {
"TimePeriod": 1627820152
[
{
"type": "STREAM",
"stream": {
"stream_state": { "180278106": { "TimePeriod": 1627820152 } },
"stream_descriptor": { "name": "keyword_performance_report_hourly" }
}
},
"budget_summary_report_hourly": {
"180278106": {
"Date": 1627800152
{
"type": "STREAM",
"stream": {
"stream_state": { "180278106": { "Date": 1627800152 } },
"stream_descriptor": { "name": "budget_summary_report_hourly" }
}
},
"ad_performance_report_hourly": {
"180278106": {
"TimePeriod": 1627795152
{
"type": "STREAM",
"stream": {
"stream_state": { "180278106": { "TimePeriod": 1627795152 } },
"stream_descriptor": { "name": "ad_performance_report_hourly" }
}
},
"campaign_performance_report_hourly": {
"180278106": {
"TimePeriod": 1727810152
{
"type": "STREAM",
"stream": {
"stream_state": { "180278106": { "TimePeriod": 1727810152 } },
"stream_descriptor": { "name": "campaign_performance_report_hourly" }
}
}
}
]
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-google-analytics-v4/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ COPY main.py ./
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.25
LABEL io.airbyte.version=0.1.26
LABEL io.airbyte.name=airbyte/source-google-analytics-v4
100 changes: 86 additions & 14 deletions ...-integrations/connectors/source-google-analytics-v4/integration_tests/abnormal_state.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,86 @@
{
"website_overview": { "ga_date": "2050-05-01" },
"traffic_sources": { "ga_date": "2050-05-01" },
"pages": { "ga_date": "2050-05-01" },
"locations": { "ga_date": "2050-05-01" },
"monthly_active_users": { "ga_date": "2050-05-01" },
"four_weekly_active_users": { "ga_date": "2050-05-01" },
"two_weekly_active_users": { "ga_date": "2050-05-01" },
"weekly_active_users": { "ga_date": "2050-05-01" },
"daily_active_users": { "ga_date": "2050-05-01" },
"devices": { "ga_date": "2050-05-01" },
"users_per_day": { "ga_date": "2050-05-01" },
"new_users_per_day": { "ga_date": "2050-05-01" }
}
[
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2050-05-01" },
"stream_descriptor": { "name": "website_overview" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2050-05-01" },
"stream_descriptor": { "name": "traffic_sources" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2050-05-01" },
"stream_descriptor": { "name": "pages" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2050-05-01" },
"stream_descriptor": { "name": "locations" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2050-05-01" },
"stream_descriptor": { "name": "monthly_active_users" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2050-05-01" },
"stream_descriptor": { "name": "four_weekly_active_users" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2050-05-01" },
"stream_descriptor": { "name": "two_weekly_active_users" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2050-05-01" },
"stream_descriptor": { "name": "weekly_active_users" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2050-05-01" },
"stream_descriptor": { "name": "daily_active_users" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2050-05-01" },
"stream_descriptor": { "name": "devices" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2050-05-01" },
"stream_descriptor": { "name": "users_per_day" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2050-05-01" },
"stream_descriptor": { "name": "new_users_per_day" }
}
}
]
100 changes: 86 additions & 14 deletions ...te-integrations/connectors/source-google-analytics-v4/integration_tests/sample_state.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,86 @@
{
"website_overview": { "ga_date": "2021-02-11" },
"traffic_sources": { "ga_date": "2021-02-11" },
"pages": { "ga_date": "2021-02-11" },
"locations": { "ga_date": "2021-02-11" },
"monthly_active_users": { "ga_date": "2021-02-11" },
"four_weekly_active_users": { "ga_date": "2021-02-11" },
"two_weekly_active_users": { "ga_date": "2021-02-11" },
"weekly_active_users": { "ga_date": "2021-02-11" },
"daily_active_users": { "ga_date": "2021-02-11" },
"devices": { "ga_date": "2021-02-11" },
"users_per_day": { "ga_date": "2021-02-11" },
"new_users_per_day": { "ga_date": "2021-02-11" }
}
[
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2021-02-11" },
"stream_descriptor": { "name": "website_overview" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2021-02-11" },
"stream_descriptor": { "name": "traffic_sources" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2021-02-11" },
"stream_descriptor": { "name": "pages" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2021-02-11" },
"stream_descriptor": { "name": "locations" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2021-02-11" },
"stream_descriptor": { "name": "monthly_active_users" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2021-02-11" },
"stream_descriptor": { "name": "four_weekly_active_users" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2021-02-11" },
"stream_descriptor": { "name": "two_weekly_active_users" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2021-02-11" },
"stream_descriptor": { "name": "weekly_active_users" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2021-02-11" },
"stream_descriptor": { "name": "daily_active_users" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2021-02-11" },
"stream_descriptor": { "name": "devices" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2021-02-11" },
"stream_descriptor": { "name": "users_per_day" }
}
},
{
"type": "STREAM",
"stream": {
"stream_state": { "ga_date": "2021-02-11" },
"stream_descriptor": { "name": "new_users_per_day" }
}
}
]
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-google-sheets/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,5 @@ COPY google_sheets_source ./google_sheets_source
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.2.17
LABEL io.airbyte.version=0.2.18
LABEL io.airbyte.name=airbyte/source-google-sheets
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-harvest/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ RUN pip install .
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.10
LABEL io.airbyte.version=0.1.11
LABEL io.airbyte.name=airbyte/source-harvest
Loading