Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs][Connector][Source][jdbc]Change the line boundary store value type to BigDecimal #4900

Merged
merged 9 commits into from
Aug 9, 2023
Merged

[Docs][Connector][Source][jdbc]Change the line boundary store value type to BigDecimal #4900

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
e9a03b4
[Docs][Connector][Source][jdbc]Change the line boundary store value t…
zhilinli123 Jun 7, 2023
a931a72
[Docs][Connector][Source][jdbc]Change the line boundary store value t…
zhilinli123 Jun 7, 2023
6bd5164
Merge branch 'apache:dev' into jdbc-doc-partitionkey-change
zhilinli123 Jul 31, 2023
bbbaefa
Merge remote-tracking branch 'origin/dev' into jdbc-doc-partitionkey-…
zhilinli123 Aug 8, 2023
3b12f4d
[Docs][Connector][Source][jdbc]Change the line boundary store value t…
zhilinli123 Aug 8, 2023
56c98eb
Merge remote-tracking branch 'origin/dev' into jdbc-doc-partitionkey-…
zhilinli123 Aug 9, 2023
309d5aa
Merge branch 'apache:dev' into jdbc-doc-partitionkey-change
zhilinli123 Aug 9, 2023
8c174b1
Merge remote-tracking branch 'origin/jdbc-doc-partitionkey-change' in…
zhilinli123 Aug 9, 2023
971019f
[Docs][Connector][Source][jdbc]Change the line boundary store value t…
zhilinli123 Aug 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/en/connector-v2/source/Jdbc.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,11 +76,11 @@ The time in seconds to wait for the database operation used to validate the conn

The column name for parallelism's partition, only support numeric type.

### partition_upper_bound [long]
### partition_upper_bound [BigDecimal]

The partition_column max value for scan, if not set SeaTunnel will query database get max value.

### partition_lower_bound [long]
### partition_lower_bound [BigDecimal]

The partition_column min value for scan, if not set SeaTunnel will query database get min value.

Expand Down
30 changes: 15 additions & 15 deletions docs/en/connector-v2/source/Mysql.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,24 +56,24 @@ Read external data source data through JDBC.

## Source Options

| Name | Type | Required | Default | Description |
|------------------------------|--------|----------|-----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| url | String | Yes | - | The URL of the JDBC connection. Refer to a case: jdbc:mysql://localhost:3306:3306/test |
| driver | String | Yes | - | The jdbc class name used to connect to the remote data source,<br/> if you use MySQL the value is `com.mysql.cj.jdbc.Driver`. |
| user | String | No | - | Connection instance user name |
| password | String | No | - | Connection instance password |
| query | String | Yes | - | Query statement |
| connection_check_timeout_sec | Int | No | 30 | The time in seconds to wait for the database operation used to validate the connection to complete |
| partition_column | String | No | - | The column name for parallelism's partition, only support numeric type,Only support numeric type primary key, and only can config one column. |
| partition_lower_bound | Long | No | - | The partition_column min value for scan, if not set SeaTunnel will query database get min value. |
| partition_upper_bound | Long | No | - | The partition_column max value for scan, if not set SeaTunnel will query database get max value. |
| partition_num | Int | No | job parallelism | The number of partition count, only support positive integer. default value is job parallelism |
| fetch_size | Int | No | 0 | For queries that return a large number of objects,you can configure<br/> the row fetch size used in the query toimprove performance by<br/> reducing the number database hits required to satisfy the selection criteria.<br/> Zero means use jdbc default value. |
| common-options | | No | - | Source plugin common parameters, please refer to [Source Common Options](common-options.md) for details |
| Name | Type | Required | Default | Description |
|------------------------------|------------|----------|-----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| url | String | Yes | - | The URL of the JDBC connection. Refer to a case: jdbc:mysql://localhost:3306:3306/test |
| driver | String | Yes | - | The jdbc class name used to connect to the remote data source,<br/> if you use MySQL the value is `com.mysql.cj.jdbc.Driver`. |
| user | String | No | - | Connection instance user name |
| password | String | No | - | Connection instance password |
| query | String | Yes | - | Query statement |
| connection_check_timeout_sec | Int | No | 30 | The time in seconds to wait for the database operation used to validate the connection to complete |
| partition_column | String | No | - | The column name for parallelism's partition, only support numeric type,Only support numeric type primary key, and only can config one column. |
| partition_lower_bound | BigDecimal | No | - | The partition_column min value for scan, if not set SeaTunnel will query database get min value. |
| partition_upper_bound | BigDecimal | No | - | The partition_column max value for scan, if not set SeaTunnel will query database get max value. |
| partition_num | Int | No | job parallelism | The number of partition count, only support positive integer. default value is job parallelism |
| fetch_size | Int | No | 0 | For queries that return a large number of objects,you can configure<br/> the row fetch size used in the query toimprove performance by<br/> reducing the number database hits required to satisfy the selection criteria.<br/> Zero means use jdbc default value. |
| common-options | | No | - | Source plugin common parameters, please refer to [Source Common Options](common-options.md) for details |

### Tips

> If partition_column is not set, it will run in single concurrency, and if partition_column is set, it will be executed in parallel according to the concurrency of tasks.
> If partition_column is not set, it will run in single concurrency, and if partition_column is set, it will be executed in parallel according to the concurrency of tasks , When your shard read field is a large number type such as bigint(30) and above and the data is not evenly distributed, it is recommended to set the parallelism level to 1 to ensure that the data skew problem is resolved

## Task Example

Expand Down
2 changes: 2 additions & 0 deletions release-note.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,8 @@

## Improve

- [Improve][Connector-V2][Jdbc-Source] Support for Decimal types as splict keys (#4634)

### Core

- [Core] [Spark] Push transform operation from Spark Driver to Executors (#4503)
Expand Down