Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial cut at database/sql compatible pgx integration #363

Merged
merged 3 commits into from
Sep 1, 2021
Merged

initial cut at database/sql compatible pgx integration #363

merged 3 commits into from
Sep 1, 2021

Conversation

nr-swilloughby
Copy link
Contributor

This adds a new integration for the PGX Postgres database driver. It instruments database transactions using PGX in the same way as we do with the PQ integration. Future work is still needed to support the other modes of operation offered by PGX.

Fixes Issue #179

// Use this package to instrument your PostgreSQL calls using the pgx
// library.
//
// USING WITH PGX AS A DATABASE/SQL DRIVER
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You think perhaps we need another section here to describe how it works with SQLx as well?

Comment on lines +198 to +207
{
dsn: "host=host_string port=port_string dbname=dbname_string",
expHost: "host_string",
expPortPathOrID: "5432",
expDatabaseName: "dbname_string",
env: map[string]string{
"PGHOST": "host_env",
"PGPORT": "port_env",
"PGDATABASE": "dbname_env",
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why the expected Port, Path or ID is "5432", while we have an environment variable PGPORT that's set to "port_env."

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same with the test below this one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What the integration code does is fall back to the default port if the provided one is not a valid port number. In this case, it's not an integer so the library uses the default Postgres port instead.

Copy link
Contributor

@rvanderwal-newrelic rvanderwal-newrelic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I only made a few comments. Also, tests pass and seem pretty sane.

The database/sql example works without a running database. However, the sqlx example needs a running database to function properly. I've suggested a change in this PR, Steve's working with it.

@marcesher
Copy link

marcesher commented Aug 26, 2021

We're very interested in this integration. I have a few questions for you all.

First, we use pgx Pool in our apps, eg:

dburl := "postgres://" + ...
cfg, err := pgxpool.ParseConfig(dburl)
pool, err := pgxpool.ConnectConfig(context.Background(), cfg)

...

err := pool.QueryRow(context.Background(),
		`SELECT foo from bar`, s).Scan(&foo)

Does this New Relic pgx integration work with Pool ?

Second, we use https://github.com/georgysavva/scany instead of sqlx, though our usage is fairly limited. A typical invocation looks like this:

var settings []SchedulerSetting
err := pgxscan.Select(context.Background(), pool, &settings,
`SELECT setting_name, setting_value, description
FROM scheduler_settings`)

Note that it passes an aforementioned pgx pool into it.

Any idea if the New Relic integration will work with scany if passing in a New Relic context?

Third, this is a general question and please don't take this as a criticism! In the past I've used New Relic with Python/Django apps, and the database integration was practically transparent. Is it simply the case that that kind of non-intrusive integration isn't possible with something like pgx due to how either Go itself works, or due to pgx more specifically?

Thanks for all this work! We're excited to use it.

@nr-swilloughby
Copy link
Contributor Author

nr-swilloughby commented Aug 27, 2021

@marcesher wrote:

We're very interested in this integration. I have a few questions for you all.

Thanks! We hope to provide what features will be useful and needed by our user community, and interactions like this are a great way for us to know what you'd most like to see.

First, we use pgx Pool in our apps, eg:
Does this New Relic pgx integration work with Pool ?

This nrpgx integration is intended to take the basic step of supporting the pgx driver in the same way our nrpq integration supports the pq driver; in other words, the application needs to use github.com/jackc/pgx/v4/stdlib and database/sql and make connections to the database using sql.Open.

However, the pgx package also supports directly connecting to the database using its own code instead of using the database/sql stuff. As I understand it, that would include the use of pools such as what you describe. Enhancing our integration to bring in support for all of the (non-database/sql) features of pgx is something we can look at given interest in having those features too.

Second, we use https://github.com/georgysavva/scany instead of sqlx, though our usage is fairly limited.
Note that it passes an aforementioned pgx pool into it.

Since this uses pools, the same answer above would apply.

Third, this is a general question and please don't take this as a criticism! In the past I've used New Relic with Python/Django apps, and the database integration was practically transparent. Is it simply the case that that kind of non-intrusive integration isn't possible with something like pgx due to how either Go itself works, or due to pgx more specifically?

In general terms, this has more to do with the differences between Python and Go.

Comment on lines +215 to +216
// fullParamPattern is a regular expression to match the key=value pairs of a
// parameterized DSN string.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love these comments.

@nr-swilloughby nr-swilloughby merged commit 05421c0 into newrelic:develop Sep 1, 2021
@nr-swilloughby nr-swilloughby deleted the 179_jackc_pgx branch September 1, 2021 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

v3 agent: Support jackc/pgx as a postgres driver
4 participants