Skip to content

Commit

Permalink
Organizations.transform added
Browse files Browse the repository at this point in the history 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
  • Loading branch information
@grubberr
grubberr committed Feb 23, 2022
1 parent 94b9e30 commit ea8bdb7
Showing 1 changed file with 4 additions and 12 deletions.
16 changes: 4 additions & 12 deletions airbyte-integrations/connectors/source-github/source_github/streams.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -331,6 +331,10 @@ def path(self, stream_slice: Mapping[str, Any] = None, **kwargs) -> str:
def parse_response(self, response: requests.Response, **kwargs) -> Iterable[Mapping]:
yield response.json()

def transform(self, record: MutableMapping[str, Any], stream_slice: Mapping[str, Any]) -> MutableMapping[str, Any]:
record["organization"] = stream_slice["organization"]
return record


class Repositories(Organizations):
"""
Expand All @@ -344,10 +348,6 @@ def parse_response(self, response: requests.Response, stream_slice: Mapping[str,
for record in response.json(): # GitHub puts records in an array.
yield self.transform(record=record, stream_slice=stream_slice)

def transform(self, record: MutableMapping[str, Any], stream_slice: Mapping[str, Any]) -> MutableMapping[str, Any]:
record["organization"] = stream_slice["organization"]
return record


class Tags(GithubStream):
"""
Expand All @@ -372,10 +372,6 @@ def parse_response(self, response: requests.Response, stream_slice: Mapping[str,
for record in response.json():
yield self.transform(record=record, stream_slice=stream_slice)

def transform(self, record: MutableMapping[str, Any], stream_slice: Mapping[str, Any]) -> MutableMapping[str, Any]:
record["organization"] = stream_slice["organization"]
return record


class Users(Organizations):
"""
Expand All @@ -389,10 +385,6 @@ def parse_response(self, response: requests.Response, stream_slice: Mapping[str,
for record in response.json():
yield self.transform(record=record, stream_slice=stream_slice)

def transform(self, record: MutableMapping[str, Any], stream_slice: Mapping[str, Any]) -> MutableMapping[str, Any]:
record["organization"] = stream_slice["organization"]
return record


# Below are semi incremental streams

Expand Down

1 comment on commit ea8bdb7

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SonarQube Report

SonarQube report for Airbyte Connectors Source Github(#10385)

Measures

Name Value Name Value Name Value
Quality Gate Status ERROR Coverage 25.7 Lines of Code 792
Reliability Rating A Bugs 0 Duplicated Lines (%) 0.0
Security Rating A Lines to Cover 74 Vulnerabilities 0
Duplicated Blocks 0 Code Smells 4 Blocker Issues 0
Critical Issues 6 Major Issues 6 Minor Issues 6

Detected Issues

Rule File Description Message
python:S2638 (CRITICAL) source_github/streams.py Method overrides should not change contracts Make parameter stream_state keyword-or-positional.
python:S2638 (CRITICAL) source_github/streams.py Method overrides should not change contracts Make parameter stream_slice keyword-or-positional.
python:S2638 (CRITICAL) source_github/streams.py Method overrides should not change contracts Make parameter next_page_token keyword-or-positional.
python:black_need_format (MINOR) source_github/source.py Please run one of the commands: "black --config ./pyproject.toml <path_to_updated_folder>" or "./gradlew format" 1 code part(s) should be updated.
flake8:E501 (MAJOR) source_github/streams.py line too long (82 > 79 characters) line too long (143 > 140 characters)
python:isort_need_format (MINOR) unit_tests/test_stream.py Please run one of the commands: "isort <path_to_updated_folder>" or "./gradlew format" 1 code part(s) should be updated.
python:S3776 (CRITICAL) source_github/streams.py:91 Cognitive Complexity of functions should not be too high Refactor this function to reduce its Cognitive Complexity from 25 to the 15 allowed.
flake8:E501 (MAJOR) source_github/streams.py line too long (82 > 79 characters) line too long (150 > 140 characters)
python:S5886 (MAJOR) source_github/streams.py:84 Function return types should be consistent with their type hint Return a value of type "Union[int, float]" instead of "NoneType" or update function "backoff_time" type hint.
python:S2638 (CRITICAL) source_github/streams.py Method overrides should not change contracts Add missing parameters organization project_id.
python:isort_need_format (MINOR) unit_tests/test_source.py Please run one of the commands: "isort <path_to_updated_folder>" or "./gradlew format" 1 code part(s) should be updated.
python:isort_need_format (MINOR) unit_tests/unit_test.py Please run one of the commands: "isort <path_to_updated_folder>" or "./gradlew format" 1 code part(s) should be updated.
python:isort_need_format (MINOR) main.py Please run one of the commands: "isort <path_to_updated_folder>" or "./gradlew format" 1 code part(s) should be updated.
python:mypy_unknown (MINOR) main.py Unknown error Duplicate module named "main" (also at "./fixtures/main.py")
flake8:E501 (MAJOR) fixtures/github.py line too long (82 > 79 characters) line too long (151 > 140 characters)
python:S5797 (CRITICAL) fixtures/github.py:79 Constants should not be used as conditions Replace this expression; used as a condition it will always be constant.
python:S112 (MAJOR) source_github/source.py:71 "Exception" and "BaseException" should not be raised Replace this generic exception class with a more specific one.
flake8:C901 (MAJOR) source_github/streams.py flake8:C901 'GithubStream.read_records' is too complex (14)

Coverage (25.7%)

File Coverage File Coverage
fixtures/github.py 0.0 fixtures/main.py 0.0
source_github/init.py 100.0 source_github/source.py 48.9
source_github/streams.py 53.4

Please sign in to comment.