Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

check last_pruned instead of is_pruning #2748

Merged
merged 5 commits into from
Oct 16, 2024
Merged

Conversation

rkuo-danswer
Copy link
Contributor

@rkuo-danswer rkuo-danswer commented Oct 9, 2024

Description

Potential workaround for race condition in wait for prune and some buggy blocking behavior in default HTTPServer.

How Has This Been Tested?

[Describe the tests you ran to verify your changes]

Accepted Risk

[Any know risks or failure modes to point out to reviewers]

Related Issue(s)

[If applicable, link to the issue(s) this PR addresses]

Checklist:

  • All of the automated tests pass
  • All PR comments are addressed and marked resolved
  • If there are migrations, they have been rebased to latest main
  • If there are new dependencies, they are added to the requirements
  • If there are new environment variables, they are added to all of the deployment methods
  • If there are new APIs that don't require auth, they are added to PUBLIC_ENDPOINT_SPECS
  • Docker images build and basic functionalities work
  • Author has done a final read through of the PR right before merge

Copy link

vercel bot commented Oct 9, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
internal-search ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 16, 2024 6:35pm

@rkuo-danswer rkuo-danswer marked this pull request as ready for review October 9, 2024 21:05
Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

This pull request modifies the pruning process for connector-credential pairs, focusing on using a 'last_pruned' timestamp instead of an 'is_pruning' boolean to address potential race conditions and improve concurrent request handling.

  • Updated backend/danswer/server/documents/cc_pair.py to return the last pruning timestamp instead of checking if pruning is in progress
  • Modified backend/tests/integration/common_utils/managers/cc_pair.py to use 'last_pruned' timestamp in CCPairManager for pruning status checks
  • Changed backend/tests/integration/tests/pruning/test_pruning.py to use ThreadingHTTPServer for improved concurrent request handling
  • Adjusted backend/tests/integration/connector_job_tests/slack/test_prune.py to capture a timestamp before pruning for more accurate status checks

4 file(s) reviewed, 1 comment(s)
Edit PR Review Bot Settings

backend/tests/integration/common_utils/managers/cc_pair.py Outdated Show resolved Hide resolved
@rkuo-danswer rkuo-danswer added this pull request to the merge queue Oct 16, 2024
Merged via the queue into main with commit 0a0215c Oct 16, 2024
7 checks passed
@rkuo-danswer rkuo-danswer deleted the bugfix/wait_for_prune branch October 16, 2024 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants