-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Postgres Source: use native Postgres timeout if it's not set by the user #19291
Merged
VitaliiMaltsev
merged 10 commits into
master
from
vmaltsev/3063-postgres-source-timeouts
Nov 16, 2022
Merged
Changes from 8 commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
6817961
Postgres Source: use native Postgres timeout if it's not set by the user
VitaliiMaltsev 01625a7
refactoring
VitaliiMaltsev 0002cc1
Merge branch 'master' into vmaltsev/3063-postgres-source-timeouts
VitaliiMaltsev a016852
updated connection timeout logic and added tests for different dataso…
VitaliiMaltsev 21b1146
Merge remote-tracking branch 'origin/vmaltsev/3063-postgres-source-ti…
VitaliiMaltsev 7df7e83
fixed pmd
VitaliiMaltsev 3b6e70b
refactoring
VitaliiMaltsev e07e9e3
Merge branch 'master' into vmaltsev/3063-postgres-source-timeouts
VitaliiMaltsev 559d275
bump version
VitaliiMaltsev bebdfc8
auto-bump connector version
octavia-squidington-iii File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4,12 +4,15 @@ | |
|
||
package io.airbyte.db.factory; | ||
|
||
import static org.postgresql.PGProperty.CONNECT_TIMEOUT; | ||
|
||
import com.google.common.base.Preconditions; | ||
import com.zaxxer.hikari.HikariConfig; | ||
import com.zaxxer.hikari.HikariDataSource; | ||
import java.io.Closeable; | ||
import java.time.Duration; | ||
import java.util.Map; | ||
import java.util.Objects; | ||
import javax.sql.DataSource; | ||
|
||
/** | ||
|
@@ -61,7 +64,7 @@ public static DataSource create(final String username, | |
.withJdbcUrl(jdbcConnectionString) | ||
.withPassword(password) | ||
.withUsername(username) | ||
.withConnectionTimeoutMs(DataSourceBuilder.getConnectionTimeoutMs(connectionProperties)) | ||
.withConnectionTimeoutMs(DataSourceBuilder.getConnectionTimeoutMs(connectionProperties, driverClassName)) | ||
.build(); | ||
} | ||
|
||
|
@@ -196,12 +199,23 @@ private DataSourceBuilder() {} | |
* | ||
* @param connectionProperties custom jdbc_url_parameters containing information on connection | ||
* properties | ||
* @param driverClassName name of the JDBC driver | ||
* @return DataSourceBuilder class used to create dynamic fields for DataSource | ||
*/ | ||
private static long getConnectionTimeoutMs(final Map<String, String> connectionProperties) { | ||
private static long getConnectionTimeoutMs(final Map<String, String> connectionProperties, String driverClassName) { | ||
// TODO: the usage of CONNECT_TIMEOUT is Postgres specific, may need to extend for other databases | ||
if (driverClassName.equals(DatabaseDriver.POSTGRESQL.getDriverClassName())) { | ||
final String pgPropertyConnectTimeout = CONNECT_TIMEOUT.getName(); | ||
// If the PGProperty.CONNECT_TIMEOUT was set by the user, then take its value, if not take the | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suggestion: would remove the comment on #L206 and move this comment to above the |
||
// default | ||
if (connectionProperties.containsKey(pgPropertyConnectTimeout) | ||
&& (Long.parseLong(connectionProperties.get(pgPropertyConnectTimeout)) >= 0)) { | ||
return Duration.ofSeconds(Long.parseLong(connectionProperties.get(pgPropertyConnectTimeout))).toMillis(); | ||
} else { | ||
return Duration.ofSeconds(Long.parseLong(Objects.requireNonNull(CONNECT_TIMEOUT.getDefaultValue()))).toMillis(); | ||
} | ||
} | ||
final Duration connectionTimeout; | ||
// TODO: the usage of CONNECT_TIMEOUT_KEY is Postgres specific, may need to extend for other | ||
// databases | ||
connectionTimeout = | ||
connectionProperties.containsKey(CONNECT_TIMEOUT_KEY) ? Duration.ofSeconds(Long.parseLong(connectionProperties.get(CONNECT_TIMEOUT_KEY))) | ||
: CONNECT_TIMEOUT_DEFAULT; | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add a test for this new functionality in
DataSourceFactoryTest.java
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It appears these changes were caught within the build failures
To add onto what Akash said, since this is a postgres specific feature I would add a test primarily for postgres and keep a generic one to account for non-postgres behavior.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@akashkulk @ryankfu added tests for Postgres and MySQL to see the difference