Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source GitLab: Accept api_url with or without scheme #21373

Merged
merged 21 commits into from
Apr 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
702a9cc
feat: source-gitlab accept api_url w/ or w/o scheme
pgollangi Jan 13, 2023
1a5b842
Merge branch 'master' into feat/source-gitlab-api-url
pgollangi Jan 23, 2023
358b447
Merge branch 'master' into feat/source-gitlab-api-url
pgollangi Jan 26, 2023
55c2e36
Merge branch 'master' into feat/source-gitlab-api-url
pgollangi Jan 26, 2023
c7b6e24
🪟🔧 Refactor ConnectorsView (#21531)
Jan 26, 2023
336d285
Merge branch 'master' into feat/source-gitlab-api-url
pgollangi Jan 26, 2023
006d33c
feat: source-gitlab accept api_url w/ or w/o scheme
pgollangi Jan 13, 2023
43080dc
Revert allowedHosts missing value checks (#21923)
evantahler Jan 26, 2023
bbf7cbf
Merge branch 'feat/source-gitlab-api-url' of github.com:pgollangi/air…
pgollangi Jan 26, 2023
5f7b179
Merge branch 'master' into feat/source-gitlab-api-url
pgollangi Jan 26, 2023
56a2285
Merge branch 'master' of github.com:pgollangi/airbyte into feat/sourc…
pgollangi Jan 31, 2023
346947a
Merge branch 'feat/source-gitlab-api-url' of github.com:pgollangi/air…
pgollangi Jan 31, 2023
c1de0d9
Merge branch 'master' into feat/source-gitlab-api-url
archangelic Apr 19, 2023
5008c34
Merge branch 'master' into feat/source-gitlab-api-url
pgollangi Apr 20, 2023
62b5954
bump versions
archangelic Apr 20, 2023
983848b
Merge branch 'master' into feat/source-gitlab-api-url
archangelic Apr 20, 2023
01c6208
Merge branch 'master' into feat/source-gitlab-api-url
marcosmarxm Apr 25, 2023
92f82c3
update expected records
marcosmarxm Apr 25, 2023
11329b7
Merge branch 'master' into feat/source-gitlab-api-url
marcosmarxm Apr 27, 2023
a32edd5
auto-bump connector version
octavia-squidington-iii Apr 27, 2023
d9da5cb
Merge branch 'master' into feat/source-gitlab-api-url
pgollangi Apr 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -13719,7 +13719,7 @@
"sourceDefinitionId": "5e6175e5-68e1-4c17-bff9-56103bbb0d80",
"name": "Gitlab",
"dockerRepository": "airbyte/source-gitlab",
"dockerImageTag": "1.0.3",
"dockerImageTag": "1.0.4",
"documentationUrl": "https://docs.airbyte.com/integrations/sources/gitlab",
"icon": "gitlab.svg",
"sourceType": "api",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -754,7 +754,7 @@
- name: Gitlab
sourceDefinitionId: 5e6175e5-68e1-4c17-bff9-56103bbb0d80
dockerRepository: airbyte/source-gitlab
dockerImageTag: 1.0.3
dockerImageTag: 1.0.4
documentationUrl: https://docs.airbyte.com/integrations/sources/gitlab
icon: gitlab.svg
sourceType: api
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5284,7 +5284,7 @@
path_in_connector_config:
- "credentials"
- "client_secret"
- dockerImage: "airbyte/source-gitlab:1.0.3"
- dockerImage: "airbyte/source-gitlab:1.0.4"
spec:
documentationUrl: "https://docs.airbyte.com/integrations/sources/gitlab"
connectionSpecification:
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-gitlab/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,5 @@ COPY main.py ./

ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=1.0.3
LABEL io.airbyte.version=1.0.4
LABEL io.airbyte.name=airbyte/source-gitlab
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-gitlab/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

MAIN_REQUIREMENTS = ["airbyte-cdk", "vcrpy==4.1.1"]

TEST_REQUIREMENTS = ["pytest~=6.1", "connector-acceptance-test", "requests_mock"]
TEST_REQUIREMENTS = ["pytest~=6.1", "connector-acceptance-test", "requests_mock", "pytest-mock"]

setup(
name="source_gitlab",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import datetime
from abc import ABC
from typing import Any, Dict, Iterable, List, Mapping, MutableMapping, Optional, Tuple
from urllib.parse import urlparse

import pendulum
import requests
Expand Down Expand Up @@ -51,7 +52,13 @@ def request_params(

@property
def url_base(self) -> str:
return f"https://{self.api_url}/api/v4/"
parse_result = urlparse(self.api_url)
# Default scheme to "https" if URL doesn't contain
scheme = parse_result.scheme if parse_result.scheme else "https"
# hostname without a scheme will result in `path` attribute
# Use path if netloc is not detected
host = parse_result.netloc if parse_result.netloc else parse_result.path
return f"{scheme}://{host}/api/v4/"

@property
def availability_strategy(self) -> Optional["AvailabilityStrategy"]:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@
import pytest


@pytest.fixture
def config(mocker):
@pytest.fixture(params=["gitlab.com", "https://gitlab.com", "https://gitlab.com/api/v4"])
def config(request):
return {
"start_date": "2021-01-01T00:00:00Z",
"api_url": "gitlab.com",
"api_url": request.param,
"credentials": {
"auth_type": "access_token",
"access_token": "token"
Expand Down
2 changes: 1 addition & 1 deletion connectors.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@
| **Genesys** | <img alt="Genesys icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/genesys.svg" height="30" height="30"/> | Source | airbyte/source-genesys:0.1.0 | unknown | [docs](https://docs.airbyte.com/integrations/sources/genesys) | [connectors/source/genesys](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/genesys) | [source-genesys](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-genesys) | <small>`5ea4459a-8f1a-452a-830f-a65c38cc438d`</small> |
| **GetLago** | <img alt="GetLago icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/getlago.svg" height="30" height="30"/> | Source | airbyte/source-getlago:0.1.0 | alpha | [docs](https://docs.airbyte.com/integrations/sources/getlago) | [connectors/source/getlago](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/getlago) | [source-getlago](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-getlago) | <small>`e1a3866b-d3b2-43b6-b6d7-8c1ee4d7f53f`</small> |
| **GitHub** | <img alt="GitHub icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/github.svg" height="30" height="30"/> | Source | airbyte/source-github:0.4.8 | generally_available | [docs](https://docs.airbyte.com/integrations/sources/github) | [connectors/source/github](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/github) | [source-github](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-github) | <small>`ef69ef6e-aa7f-4af1-a01d-ef775033524e`</small> |
| **Gitlab** | <img alt="Gitlab icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/gitlab.svg" height="30" height="30"/> | Source | airbyte/source-gitlab:1.0.3 | beta | [docs](https://docs.airbyte.com/integrations/sources/gitlab) | [connectors/source/gitlab](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/gitlab) | [source-gitlab](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-gitlab) | <small>`5e6175e5-68e1-4c17-bff9-56103bbb0d80`</small> |
| **Gitlab** | <img alt="Gitlab icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/gitlab.svg" height="30" height="30"/> | Source | airbyte/source-gitlab:1.0.4 | beta | [docs](https://docs.airbyte.com/integrations/sources/gitlab) | [connectors/source/gitlab](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/gitlab) | [source-gitlab](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-gitlab) | <small>`5e6175e5-68e1-4c17-bff9-56103bbb0d80`</small> |
| **Glassfrog** | <img alt="Glassfrog icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/glassfrog.svg" height="30" height="30"/> | Source | airbyte/source-glassfrog:0.1.0 | alpha | [docs](https://docs.airbyte.com/integrations/sources/glassfrog) | [connectors/source/glassfrog](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/glassfrog) | [source-glassfrog](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-glassfrog) | <small>`cf8ff320-6272-4faa-89e6-4402dc17e5d5`</small> |
| **GoCardless** | <img alt="GoCardless icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/gocardless.svg" height="30" height="30"/> | Source | airbyte/source-gocardless:0.1.0 | alpha | [docs](https://docs.airbyte.com/integrations/sources/gocardless) | [connectors/source/gocardless](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/gocardless) | [source-gocardless](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-gocardless) | <small>`ba15ac82-5c6a-4fb2-bf24-925c23a1180c`</small> |
| **Gong** | <img alt="Gong icon" src="https://raw.githubusercontent.com/airbytehq/airbyte /master/airbyte-config-oss/init-oss/src/main/resources/icons/gong.svg" height="30" height="30"/> | Source | airbyte/source-gong:0.1.0 | alpha | [docs](https://docs.airbyte.com/integrations/sources/gong) | [connectors/source/gong](https://github.com/airbytehq/airbyte/issues?q=is:open+is:issue+label:connectors/source/gong) | [source-gong](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations/connectors/source-gong) | <small>`32382e40-3b49-4b99-9c5c-4076501914e7`</small> |
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/sources/gitlab.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ Gitlab has the [rate limits](https://docs.gitlab.com/ee/user/gitlab_com/index.ht

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:---------------------------------------------------------|:-------------------------------------------------------------------------------------------|
| 1.0.4 | 2023-04-20 | [21373](https://github.com/airbytehq/airbyte/pull/21373) | Accept api_url with or without scheme |
| 1.0.3 | 2023-02-14 | [22992](https://github.com/airbytehq/airbyte/pull/22992) |Specified date formatting in specification |
| 1.0.2 | 2023-01-27 | [22001](https://github.com/airbytehq/airbyte/pull/22001) | Set `AvailabilityStrategy` for streams explicitly to `None` |
| 1.0.1 | 2023-01-23 | [21713](https://github.com/airbytehq/airbyte/pull/21713) | Fix missing data issue |
Expand Down