Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(ingest/gc): infinite loop query entities #12274

Merged
merged 2 commits into from
Jan 6, 2025
Merged

fix(ingest/gc): infinite loop query entities #12274

merged 2 commits into from
Jan 6, 2025

Conversation

anshbansal
Copy link
Collaborator

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@github-actions github-actions bot added the ingestion PR or Issue related to the ingestion of metadata label Jan 6, 2025
@datahub-cyborg datahub-cyborg bot added the needs-review Label for PRs that need review from a maintainer. label Jan 6, 2025
treff7es
treff7es reviewed Jan 6, 2025
View reviewed changes
metadata-ingestion/src/datahub/ingestion/source/gc/soft_deleted_entity_cleanup.py Outdated
@@ -260,6 +260,8 @@ def _get_soft_deleted_queries(self) -> Iterable[str]:
scroll_across_entities = result.get("scrollAcrossEntities")
if not scroll_across_entities:
break
if scroll_across_entities.get("count") == 0:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if scroll_across_entities.get("count") == 0:
if not scroll_across_entities or not scroll_across_entities.get("count"):

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can merge the above two condition checks

@datahub-cyborg datahub-cyborg bot added pending-submitter-merge and removed needs-review Label for PRs that need review from a maintainer. labels Jan 6, 2025
@anshbansal anshbansal merged commit 842c8f9 into master Jan 6, 2025
96 of 107 checks passed
@anshbansal anshbansal deleted the ab-fix-infinite-loop branch January 6, 2025 11:09
llance pushed a commit to llance/datahub that referenced this pull request Jan 13, 2025
@anshbansal @llance
fix(ingest/gc): infinite loop query entities (datahub-project#12274)
c146922
llance pushed a commit to llance/datahub that referenced this pull request Jan 13, 2025
@anshbansal @llance
fix(ingest/gc): infinite loop query entities (datahub-project#12274)
f9a18cd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@treff7es treff7es treff7es approved these changes

Assignees
No one assigned
Labels
ingestion PR or Issue related to the ingestion of metadata pending-submitter-merge
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@anshbansal @treff7es