Skip to content

Commit

Permalink
refactor: start_point_map -> start_point
Browse files Browse the repository at this point in the history 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
  • Loading branch information
@grubberr
grubberr committed Feb 23, 2022
1 parent f7468c7 commit 5122429
Showing 1 changed file with 8 additions and 10 deletions.
18 changes: 8 additions & 10 deletions airbyte-integrations/connectors/source-github/source_github/streams.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -242,13 +242,13 @@ def read_records(
stream_slice: Mapping[str, Any] = None,
stream_state: Mapping[str, Any] = None,
) -> Iterable[Mapping[str, Any]]:
start_point_map = {repo: self.get_starting_point(stream_state=stream_state, repository=repo) for repo in self.repositories}
start_point = self.get_starting_point(stream_state=stream_state, repository=stream_slice["repository"])
for record in super().read_records(
sync_mode=sync_mode, cursor_field=cursor_field, stream_slice=stream_slice, stream_state=stream_state
):
if record.get(self.cursor_field) > start_point_map[stream_slice["repository"]]:
if record[self.cursor_field] > start_point:
yield record
elif self.is_sorted_descending and record.get(self.cursor_field) < start_point_map[stream_slice["repository"]]:
elif self.is_sorted_descending and record[self.cursor_field] < start_point:
break


Expand Down Expand Up @@ -642,17 +642,15 @@ def read_records(
stream_slice: Mapping[str, Any] = None,
stream_state: Mapping[str, Any] = None,
) -> Iterable[Mapping[str, Any]]:
repository = stream_slice["repository"]
start_point_map = {
branch: self.get_starting_point(stream_state=stream_state, repository=repository, branch=branch)
for branch in self.branches_to_pull.get(repository, [])
}
start_point = self.get_starting_point(
stream_state=stream_state, repository=stream_slice["repository"], branch=stream_slice["branch"]
)
for record in super(SemiIncrementalGithubStream, self).read_records(
sync_mode=sync_mode, cursor_field=cursor_field, stream_slice=stream_slice, stream_state=stream_state
):
if record.get(self.cursor_field) > start_point_map[stream_slice["branch"]]:
if record[self.cursor_field] > start_point:
yield record
elif self.is_sorted_descending and record.get(self.cursor_field) < start_point_map[stream_slice["branch"]]:
elif self.is_sorted_descending and record[self.cursor_field] < start_point:
break


Expand Down

1 comment on commit 5122429

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SonarQube Report

SonarQube report for Airbyte Connectors Source Github(#10385)

Measures

Name Value Name Value Name Value
Duplicated Blocks 0 Vulnerabilities 0 Bugs 0
Reliability Rating A Quality Gate Status ERROR Security Rating A
Duplicated Lines (%) 0.0 Lines to Cover 80 Code Smells 4
Coverage 26.2 Lines of Code 798 Blocker Issues 0
Critical Issues 6 Major Issues 6 Minor Issues 6

Detected Issues

Rule File Description Message
python:S2638 (CRITICAL) source_github/streams.py Method overrides should not change contracts Make parameter stream_state keyword-or-positional.
python:S2638 (CRITICAL) source_github/streams.py Method overrides should not change contracts Make parameter stream_slice keyword-or-positional.
python:S2638 (CRITICAL) source_github/streams.py Method overrides should not change contracts Make parameter next_page_token keyword-or-positional.
python:black_need_format (MINOR) source_github/source.py Please run one of the commands: "black --config ./pyproject.toml <path_to_updated_folder>" or "./gradlew format" 1 code part(s) should be updated.
flake8:E501 (MAJOR) source_github/streams.py line too long (82 > 79 characters) line too long (143 > 140 characters)
python:isort_need_format (MINOR) unit_tests/test_stream.py Please run one of the commands: "isort <path_to_updated_folder>" or "./gradlew format" 1 code part(s) should be updated.
python:S3776 (CRITICAL) source_github/streams.py:91 Cognitive Complexity of functions should not be too high Refactor this function to reduce its Cognitive Complexity from 25 to the 15 allowed.
flake8:E501 (MAJOR) source_github/streams.py line too long (82 > 79 characters) line too long (150 > 140 characters)
python:S5886 (MAJOR) source_github/streams.py:84 Function return types should be consistent with their type hint Return a value of type "Union[int, float]" instead of "NoneType" or update function "backoff_time" type hint.
python:S2638 (CRITICAL) source_github/streams.py Method overrides should not change contracts Add missing parameters organization project_id.
python:isort_need_format (MINOR) unit_tests/test_source.py Please run one of the commands: "isort <path_to_updated_folder>" or "./gradlew format" 1 code part(s) should be updated.
python:isort_need_format (MINOR) unit_tests/unit_test.py Please run one of the commands: "isort <path_to_updated_folder>" or "./gradlew format" 1 code part(s) should be updated.
python:isort_need_format (MINOR) main.py Please run one of the commands: "isort <path_to_updated_folder>" or "./gradlew format" 1 code part(s) should be updated.
python:mypy_unknown (MINOR) main.py Unknown error Duplicate module named "main" (also at "./fixtures/main.py")
flake8:E501 (MAJOR) fixtures/github.py line too long (82 > 79 characters) line too long (151 > 140 characters)
python:S5797 (CRITICAL) fixtures/github.py:79 Constants should not be used as conditions Replace this expression; used as a condition it will always be constant.
python:S112 (MAJOR) source_github/source.py:71 "Exception" and "BaseException" should not be raised Replace this generic exception class with a more specific one.
flake8:C901 (MAJOR) source_github/streams.py flake8:C901 'GithubStream.read_records' is too complex (14)

Coverage (26.2%)

File Coverage File Coverage
fixtures/github.py 0.0 fixtures/main.py 0.0
source_github/init.py 100.0 source_github/source.py 48.9
source_github/streams.py 53.1

Please sign in to comment.