Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add PostgreSQL Retriever #2790

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

gogbog
Copy link

@gogbog gogbog commented Dec 12, 2024

Description

This PR adds a PostgreSQL retriever for Goff. For example, we use PostgreSQL in our projects, and it would be nice to stick to one database for everything. Most likely, there are other people who would benefit from this as well.

Implementation Details:

New Libraries:

  • I used PGX V5 for the database connection and connection pool.
  • Integration tests were done using testcontainers-go to create a PostgreSQL container instance

Current Retriever Options:

  • URI - Database connection string
  • Table - The table from which the flags will be fetched.
  • Column - The column from which the flags will be fetched.
  • Type - Currently hardcoded to JSON. In the future, we can extend this to support a more structured table approach.

@thomaspoignant , happy to share this and open to improving it further if you have any feedback!

Checklist

  • I have tested this code
  • I have added unit test to cover this code
  • I have added example in the examples folder of how to run the provider
  • I have updated the documentation (README.md and /website/docs)
  • I have followed the contributing guide

Copy link

netlify bot commented Dec 12, 2024

Deploy Preview for go-feature-flag-doc-preview canceled.

Name Link
🔨 Latest commit 80dd2ee
🔍 Latest deploy log https://app.netlify.com/sites/go-feature-flag-doc-preview/deploys/675d92c9da6aa5000829d2c1

@gogbog gogbog changed the title Feat/Add PostgreSQL Retriever feat/Add PostgreSQL Retriever Dec 12, 2024
@gogbog gogbog changed the title feat/Add PostgreSQL Retriever feat: Add PostgreSQL Retriever Dec 12, 2024
@gogbog gogbog force-pushed the feat/add-postgresql-receiver branch from 56bc75d to 6ea1e93 Compare December 12, 2024 15:57
Copy link
Owner

@thomaspoignant thomaspoignant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @gogbog this is a super nice start, I have left some small nit comments.

As we said in slack I would also like to have the structured version too in the same PR if possible. WDYT?

r.logger.Error("Flag key does not have a string value")
}
} else {
r.logger.Error("No 'flag' entry found")
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤓 nitpick: ‏I am not sure that it is really an error to have no flag.

Suggested change
r.logger.Error("No 'flag' entry found")
r.logger.Warn("No 'flag' entry found")

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed that.

if r.dbPool == nil {
r.status = retriever.RetrieverNotReady

poolConfig, err := pgxpool.ParseConfig(r.URI)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ question: ‏Do we need a connection pool here? We will have only 1 connection per retriever, I am not super familiar with pgxpool but what value is it bringing here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right. Changed it to direct connection instead of a pool. If we have single connection it won't bring any value.

return err
}

pool, err := pgxpool.NewWithConfig(ctx, poolConfig)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❔ question: ‏Same question for pgxpool.

}
defer rows.Close()

ffDocs := make(map[string]interface{})
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 suggestion: ‏it is not super explicit to me what ffDocs contains. I can't come with a better name, but it does not sound self explain what do we store in this map.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, it's not the best variable name. I've updated it to mappedFlagDocs as it feels more self-explanatory. I'm still not sure what name would be the best, If you have better suggestions, let me know.

Copy link

Copy link

codecov bot commented Dec 14, 2024

Codecov Report

Attention: Patch coverage is 78.31325% with 18 lines in your changes missing coverage. Please review.

Project coverage is 84.66%. Comparing base (d7e475b) to head (80dd2ee).

Files with missing lines Patch % Lines
retriever/postgresqlretriever/retriever.go 74.60% 11 Missing and 5 partials ⚠️
cmd/relayproxy/service/gofeatureflag.go 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2790      +/-   ##
==========================================
- Coverage   84.77%   84.66%   -0.11%     
==========================================
  Files         111      112       +1     
  Lines        5207     5289      +82     
==========================================
+ Hits         4414     4478      +64     
- Misses        628      641      +13     
- Partials      165      170       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants