🎉 Source Gitlab: add `OAuth2.0` authentication support #7506

bazarnov · 2021-10-31T19:59:31Z

What

#7480 - Source GitLab: support oauth

How

added GitlabAuthenticator class inside of source.py
extended spec.json with oauthFlowSpecification
added GitlabOauthFlow.java
added GitlabOauthFlowTest.java unit_test
added GitlabOauthFlowIntegrationTest.java integration_test

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions
Connector version bumped like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
Credentials added to Github CI. Instructions.
/test connector=connectors/<name> command is passing.
New Connector version released on Dockerhub by running the /publish command described here

This change is

…part

bazarnov · 2021-11-01T08:30:23Z

/test connector=connectors/source-gitlab

🕑 connectors/source-gitlab https://github.com/airbytehq/airbyte/actions/runs/1407111797
✅ connectors/source-gitlab https://github.com/airbytehq/airbyte/actions/runs/1407111797
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 47     20    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  860    419    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                        Stmts   Miss  Cover
	 -----------------------------------------------
	 source_gitlab/__init__.py       2      2     0%
	 source_gitlab/source.py        46     46     0%
	 source_gitlab/streams.py      220    220     0%
	 -----------------------------------------------
	 TOTAL                         268    268     0%

airbyte-oauth/src/test/java/io/airbyte/oauth/flows/GitlabOAuthFlowTest.java

…b-oauth

bazarnov · 2021-11-01T15:08:14Z

/test connector=connectors/source-gitlab

🕑 connectors/source-gitlab https://github.com/airbytehq/airbyte/actions/runs/1408454039
✅ connectors/source-gitlab https://github.com/airbytehq/airbyte/actions/runs/1408454039
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 54     23    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  867    422    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                        Stmts   Miss  Cover
	 -----------------------------------------------
	 source_gitlab/__init__.py       2      2     0%
	 source_gitlab/source.py        46     46     0%
	 source_gitlab/streams.py      220    220     0%
	 -----------------------------------------------/actions-runner/_work/airbyte/airbyte/airbyte-integrations/connectors/source-gitlab/.venv/lib/python3.8/site-packages/coverage/control.py:761: CoverageWarning: No data was collected. (no-data-collected)
	 -----------------------------------------------
	   self._warn("No data was collected.", slug="no-data-collected")
	 TOTAL                         268    268     0%

alafanechere

👏
Feel free to publish, but do not merge before 10AM PST please.

davydov-d · 2023-01-18T14:28:45Z

/publish connector=connectors/source-gitlab

🕑 Publishing the following connectors:
connectors/source-gitlab
https://github.com/airbytehq/airbyte/actions/runs/3949845143

Connector	Did it publish?	Were definitions generated?
connectors/source-gitlab	❌	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

…byte into bazarnov/7480-gitlab-oauth

davydov-d · 2023-01-18T15:44:48Z

/publish connector=connectors/source-gitlab

🕑 Publishing the following connectors:
connectors/source-gitlab
https://github.com/airbytehq/airbyte/actions/runs/3950510305

Connector	Did it publish?	Were definitions generated?
connectors/source-gitlab	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

github-actions · 2023-01-18T15:55:39Z

Airbyte Code Coverage

File	Coverage [25.08%]	❌
GitlabOAuthFlow.java	71.15%	🍏
OAuthImplementationFactory.java	0%	❌

Total Project Coverage	26.77%	🍏

github-actions · 2023-01-18T16:24:47Z

Airbyte Code Coverage

File	Coverage [25.08%]	❌
GitlabOAuthFlow.java	71.15%	🍏
OAuthImplementationFactory.java	0%	❌

Total Project Coverage	26.77%	🍏

github-actions · 2023-01-18T17:05:10Z

Airbyte Code Coverage

File	Coverage [25.08%]	❌
GitlabOAuthFlow.java	71.15%	🍏
OAuthImplementationFactory.java	0%	❌

Total Project Coverage	26.77%	🍏

tybernstein · 2023-01-18T17:57:05Z

@davydov-d Is this good to merge? This roughly the time we said it would be merged

davydov-d · 2023-01-18T17:59:27Z

yep! :)

tybernstein · 2023-01-18T18:30:44Z

Thank you!

tybernstein · 2023-01-18T19:45:12Z

It appears that the option to OAuth is not available on Cloud and syncs are starting to break. @alafanechere, @brianjlai, and @davydov-d. Can you check this out?

davydov-d · 2023-01-18T19:52:53Z

@TBernstein4 the deployment to cloud has not been finished yet. I removed the OAuth config so the previous version should work as usual. Will restore the Oauth config as soon as the deployment succeeds

davydov-d · 2023-01-18T19:53:21Z

I'll make you know when the Oauth is on cloud

tybernstein · 2023-01-18T20:28:02Z

Thank you for doing that @davydov-d!

bazarnov added 5 commits October 29, 2021 15:18

added support of oauth2.0

e1cea7e

added OAuthFlow.java part

6934a2d

bumped version of the Dockerfile

ac6dd9e

corrected SAT

37af9c0

added unit_test for Oauth java part, added integration-test for java …

5ecff85

…part

bazarnov self-assigned this Oct 31, 2021

bazarnov linked an issue Oct 31, 2021 that may be closed by this pull request

Source GitLab: support oauth #7480

Closed

github-actions bot added the area/connectors Connector related issues label Oct 31, 2021

added changelog

26612c4

github-actions bot added the area/documentation Improvements or additions to documentation label Oct 31, 2021

bazarnov requested review from midavadim and antixar October 31, 2021 20:04

bazarnov temporarily deployed to more-secrets October 31, 2021 20:04 Inactive

bazarnov temporarily deployed to more-secrets October 31, 2021 20:12 Inactive

fix java part test

98e03ec

bazarnov temporarily deployed to more-secrets October 31, 2021 20:19 Inactive

jrhizor temporarily deployed to more-secrets November 1, 2021 08:33 Inactive

bazarnov requested a review from VitaliiMaltsev November 1, 2021 13:21

VitaliiMaltsev reviewed Nov 1, 2021

View reviewed changes

airbyte-oauth/src/test/java/io/airbyte/oauth/flows/GitlabOAuthFlowTest.java Outdated Show resolved Hide resolved

bazarnov added 2 commits November 1, 2021 17:07

updated after review

af49c77

Merge remote-tracking branch 'origin/master' into bazarnov/7480-gitla…

bd67cec

…b-oauth

bazarnov temporarily deployed to more-secrets November 1, 2021 15:09 Inactive

jrhizor temporarily deployed to more-secrets November 1, 2021 15:10 Inactive

bazarnov requested a review from VitaliiMaltsev November 1, 2021 15:16

VitaliiMaltsev approved these changes Nov 1, 2021

View reviewed changes

bazarnov temporarily deployed to more-secrets November 2, 2021 13:59 Inactive

bazarnov temporarily deployed to more-secrets November 2, 2021 14:33 Inactive

updated spec.json

ec26576

alafanechere approved these changes Jan 18, 2023

View reviewed changes

davydov-d added 2 commits January 18, 2023 17:39

#7480 sourcte gitlab: upd expected records

0da8ab6

Merge branch 'bazarnov/7480-gitlab-oauth' of github.com:airbytehq/air…

7052aed

…byte into bazarnov/7480-gitlab-oauth

davydov-d temporarily deployed to more-secrets January 18, 2023 15:43 — with GitHub Actions Inactive

davydov-d temporarily deployed to more-secrets January 18, 2023 15:44 — with GitHub Actions Inactive

auto-bump connector version

550f5ad

octavia-squidington-iii temporarily deployed to more-secrets January 18, 2023 16:14 — with GitHub Actions Inactive

octavia-squidington-iii temporarily deployed to more-secrets January 18, 2023 16:15 — with GitHub Actions Inactive

Merge branch 'master' into bazarnov/7480-gitlab-oauth

9df8fa1

davydov-d temporarily deployed to more-secrets January 18, 2023 16:54 — with GitHub Actions Inactive

davydov-d temporarily deployed to more-secrets January 18, 2023 16:55 — with GitHub Actions Inactive

davydov-d merged commit b69639e into master Jan 18, 2023

davydov-d deleted the bazarnov/7480-gitlab-oauth branch January 18, 2023 17:59

octavia-squidington-iii mentioned this pull request Jan 24, 2023

Bump Airbyte version from 0.40.28 to 0.40.29 #21767

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🎉 Source Gitlab: add `OAuth2.0` authentication support #7506

🎉 Source Gitlab: add `OAuth2.0` authentication support #7506

bazarnov commented Oct 31, 2021 •

edited by pmossman

Loading

bazarnov commented Nov 1, 2021 •

edited by github-actions bot

Loading

bazarnov commented Nov 1, 2021 •

edited by github-actions bot

Loading

alafanechere left a comment

davydov-d commented Jan 18, 2023 •

edited by github-actions bot

Loading

davydov-d commented Jan 18, 2023 •

edited by github-actions bot

Loading

github-actions bot commented Jan 18, 2023

github-actions bot commented Jan 18, 2023

github-actions bot commented Jan 18, 2023

tybernstein commented Jan 18, 2023

davydov-d commented Jan 18, 2023

tybernstein commented Jan 18, 2023

tybernstein commented Jan 18, 2023

davydov-d commented Jan 18, 2023

davydov-d commented Jan 18, 2023

tybernstein commented Jan 18, 2023

🎉 Source Gitlab: add OAuth2.0 authentication support #7506

🎉 Source Gitlab: add OAuth2.0 authentication support #7506

Conversation

bazarnov commented Oct 31, 2021 • edited by pmossman Loading

What

How

Pre-merge Checklist

Community member or Airbyter

Airbyter

bazarnov commented Nov 1, 2021 • edited by github-actions bot Loading

bazarnov commented Nov 1, 2021 • edited by github-actions bot Loading

alafanechere left a comment

Choose a reason for hiding this comment

davydov-d commented Jan 18, 2023 • edited by github-actions bot Loading

davydov-d commented Jan 18, 2023 • edited by github-actions bot Loading

github-actions bot commented Jan 18, 2023

Airbyte Code Coverage

github-actions bot commented Jan 18, 2023

Airbyte Code Coverage

github-actions bot commented Jan 18, 2023

Airbyte Code Coverage

tybernstein commented Jan 18, 2023

davydov-d commented Jan 18, 2023

tybernstein commented Jan 18, 2023

tybernstein commented Jan 18, 2023

davydov-d commented Jan 18, 2023

davydov-d commented Jan 18, 2023

tybernstein commented Jan 18, 2023

🎉 Source Gitlab: add `OAuth2.0` authentication support #7506

🎉 Source Gitlab: add `OAuth2.0` authentication support #7506

bazarnov commented Oct 31, 2021 •

edited by pmossman

Loading

bazarnov commented Nov 1, 2021 •

edited by github-actions bot

Loading

bazarnov commented Nov 1, 2021 •

edited by github-actions bot

Loading

davydov-d commented Jan 18, 2023 •

edited by github-actions bot

Loading

davydov-d commented Jan 18, 2023 •

edited by github-actions bot

Loading