feat: consolidate Postgres providers #494
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Create a common PostgresDatastore class to encapsulate all the DB query logic based on a given connection pool. Reuse the same logic across AlloyDB & CloudSQL postgres providers for better code maintainability and less redundancy.
The `format_sql` helper now supports both `tuple` and `dict` for its `params` argument. This change accommodates the AlloyDB and CloudSQL Postgres providers, which pass `params` as `dict` through `postgres_datastore.py`, unlike the non-cloud Postgres provider that uses `tuples`. The updated `format_sql` helper is used in `postgres_datastore.py` query handlers to format SQL queries, populate them with variables, and return them as traces.
|
I believe this needs to be merged before #493 |
Co-authored-by: Vishwaraj Anand <vishwaraj.anand00@gmail.com>
The `format_sql` helper now supports both `tuple` and `dict` for its `params` argument. This change accommodates the AlloyDB and CloudSQL Postgres providers, which pass `params` as `dict` through `postgres_datastore.py`, unlike the non-cloud Postgres provider that uses `tuples`. The updated `format_sql` helper is used in `postgres_datastore.py` query handlers to format SQL queries, populate them with variables, and return them as traces.
Co-authored-by: Vishwaraj Anand <vishwaraj.anand00@gmail.com>
kurtisvg
requested changes
Sep 19, 2024
To align with our existing AlloyDB and CloudSQL providers, we're updating the non-cloud Postgres provider to use SQLAlchemy. This consolidates all Postgres interactions through the common `postgres_datastore.py` reusing its query handlers.
This commit renames the `pool` variable to `async_engine` in all Postgres providers. Previously, these providers used `asyncpg` directly for connection pool management. They have now been migrated to SQLAlchemy, which uses `AsyncEngine` for connection management. This name change reflects the shift to SQLAlchemy and provides better clarity.
This is now possible since non-cloud `postgres.py` was the only file where tuple was being passed to `format_sql` prior to its consolidation, but after consolidation, it passes `dict` only.
…oyDB/CloudSQL providers.
kurtisvg
changed the title
kurtisvg
approved these changes
Sep 22, 2024
anubhav756
pushed a commit
that referenced
this pull request
Dec 3, 2024
🤖 I have created a release *beep* *boop* --- ## [0.3.0](v0.2.0...v0.3.0) (2024-12-03) ### Features * Add similarity threshold to amenity search ([#477](#477)) ([c49bef9](c49bef9)) * Add tracing for AlloyDB and CloudSQL Postgres providers ([#494](#494)) ([2fa03bc](2fa03bc)) * Consolidate postgres providers ([#493](#493)) ([a3b2c42](a3b2c42)) * Reuse connector object across different database connections in… ([#487](#487)) ([61c0f52](61c0f52)) * Switch the llm to ChatVertexAI ([#486](#486)) ([479c5e5](479c5e5)) ### Bug Fixes * Add test cases to improve coverage for postgres and over more tools. ([#508](#508)) ([20870ea](20870ea)) * Reuse connector object across different database connections. ([#484](#484)) ([2b05739](2b05739)), closes [#416](#416) * update close client function to async ([#505](#505)) ([b614276](b614276)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To align with our existing AlloyDB and CloudSQL providers, we're updating the non-cloud Postgres provider to use SQLAlchemy. This also consolidates AlloyDB and CloudSQL Postgres interactions through the non-Cloud Postgres client by reusing its query handlers.
Also, the
format_sqlhelper now acceptsdictfor itsparamsargument. This change accommodates the AlloyDB and CloudSQL Postgres providers, and now, post consolidation, non-cloud Postres client as well, which passparamsasdictthroughpostgres.py. The updatedformat_sqlhelper is used inpostgres.pyquery handlers to format SQL queries, populate them with variables, and return them as traces.