Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes and Enhancements for PostgreSQL and JSON Document Storage #837

Merged
merged 16 commits into from
Feb 18, 2025

Conversation

ParisNeo
Copy link
Contributor

Description:

This pull request addresses several issues related to document storage in PostgreSQL and JSON handling. The following changes have been implemented:

  1. Added Missing Drop Methods

    • Implemented drop methods for postgresql as well as json doc status.
  2. Fixed Key Filtering for PostgreSQL

    • Corrected the key filtering logic to enhance data retrieval accuracy.
  3. Fixed PostgreSQL Implementation for Status Gathering

    • Updated the status gathering implementation to ensure it functions correctly with PostgreSQL.

Linting:

  • Ensured that the code adheres to the linting standards for better readability and maintainability.

element["id"]: DocProcessingStatus(
content=result[0]["content"],
content_summary=element["content_summary"],
content_length=element["content_length"],
status=element["status"],
created_at=element["created_at"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you remove str()?
It is not needed here.
It comes from the other implementation.
Thanks 🙏🏻

@ParisNeo
Copy link
Contributor Author

How are we going to handle wiping out the database?

@YanSte YanSte self-requested a review February 18, 2025 11:01
@YanSte
Copy link
Contributor

YanSte commented Feb 18, 2025

Small reviews to apply after that I will merge.

element["id"]: DocProcessingStatus(
content=result[0]["content"],
content_summary=element["content_summary"],
content_length=element["content_length"],
status=element["status"],
created_at=element["created_at"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you remove str()?
It is not needed here.
It comes from the other implementation.
Thanks 🙏🏻

else:
exist_keys = []
new_keys = set([s for s in keys if s not in exist_keys])
print(f"keys: {keys}")
Copy link
Contributor

@YanSte YanSte Feb 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please remove all the print. 🙏🏻

@YanSte
Copy link
Contributor

YanSte commented Feb 18, 2025

By the way thanks for the documentation for Postgres.

@YanSte YanSte merged commit 99dc485 into HKUDS:main Feb 18, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants