Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🐛 BUG-4307: Fix incorrect handling of LONG RAW data type #7125

Merged
merged 5 commits into from
Oct 21, 2021
Merged

🐛 BUG-4307: Fix incorrect handling of LONG RAW data type #7125

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
98eea6c
BUG-4307: Fix incorrect handling of LONG RAW data type.
alexandertsukanov Oct 18, 2021
0dcab3f
BUG-4307: Added property useFetchSizeWithLongColumn to Oracle JDBC Dr…
alexandertsukanov Oct 20, 2021
7c0ada5
Merge branch 'master' into otsukanov/4307-incorrect_handling_of_LONG_…
alexandertsukanov Oct 20, 2021
4d7153d
BUG-4307: Added README.md and updated oracle.md
alexandertsukanov Oct 21, 2021
690bd3a
BUG-4307: Added README.md and updated oracle.md
alexandertsukanov Oct 21, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions airbyte-integrations/connectors/source-oracle/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Oracle Source

## Documentation
This is the repository for the Oracle only source connector in Java.
For information about how to use this connector within Airbyte, see [User Documentation](https://docs.airbyte.io/integrations/sources/oracle)

## Local development

#### Building via Gradle
From the Airbyte repository root, run:
```
./gradlew :airbyte-integrations:connectors:source-oracle:build
```

### Locally running the connector docker image

#### Build
Build the connector image via Gradle:
```
./gradlew :airbyte-integrations:connectors:source-oracle:airbyteDocker
```
When building via Gradle, the docker image name and tag, respectively, are the values of the `io.airbyte.name` and `io.airbyte.version` `LABEL`s in
the Dockerfile.

## Testing
We use `JUnit` for Java tests.

### Test Configuration
#### Acceptance Tests
To run acceptance and custom integration tests:
```
./gradlew :airbyte-integrations:connectors:source-oracle:integrationTest
```
9 changes: 9 additions & 0 deletions ...ctors/source-oracle/src/main/java/io/airbyte/integrations/source/oracle/OracleSource.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,15 @@ public static Source sshWrappedSource() {
public JsonNode toDatabaseConfig(final JsonNode config) {
final List<String> additionalParameters = new ArrayList<>();

/*
The property useFetchSizeWithLongColumn required to select LONG or LONG RAW columns.
Oracle recommends avoiding LONG and LONG RAW columns. Use LOB instead. They are included in Oracle only for legacy reasons.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice comments.

THIS IS A THIN ONLY PROPERTY. IT SHOULD NOT BE USED WITH ANY OTHER DRIVERS.
See https://docs.oracle.com/cd/E11882_01/appdev.112/e13995/oracle/jdbc/OracleDriver.html
https://docs.oracle.com/cd/B19306_01/java.102/b14355/jstreams.htm#i1014085
*/
additionalParameters.add("oracle.jdbc.useFetchSizeWithLongColumn=true");

final Protocol protocol = config.has("encryption")
? obtainConnectionProtocol(config.get("encryption"), additionalParameters)
: Protocol.TCP;
Expand Down
6 changes: 3 additions & 3 deletions ...test-integration/java/io/airbyte/integrations/source/oracle/OracleSourceDatatypeTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ protected void initTests() {
TestDataHolder.builder()
.sourceType("BINARY_DOUBLE")
.airbyteType(JsonSchemaPrimitive.NUMBER)
.addInsertValues("126.45d", "2.22507485850720E-308", "TO_BINARY_DOUBLE('1.79769313486231E+308')", "BINARY_DOUBLE_INFINITY")
.addInsertValues("126.45d", "2.22507485850720E-308", "1.79769313486231E+308d", "BINARY_DOUBLE_INFINITY")
.addExpectedValues("126.45", "0.0", "1.79769313486231E308", "Infinity")
.build());

Expand Down Expand Up @@ -266,8 +266,8 @@ protected void initTests() {
.sourceType("LONG")
.airbyteType(JsonSchemaPrimitive.STRING)
.fullSourceDataType("LONG RAW")
// @TODO stream fails when reading data back
// .addInsertValues("utl_raw.cast_to_raw('some content here')", "null")
.addInsertValues("utl_raw.cast_to_raw('some content here')", "null")
.addExpectedValues("c29tZSBjb250ZW50IGhlcmU=", null)
.build());

addDataTypeTestData(
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/sources/oracle.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ Airbite has the ability to connect to the Oracle source with 3 network connectiv

| Version | Date | Pull Request | Subject |
| :--- | :--- | :--- | :--- |
| 0.3.8 | 2021-10-13 | [7125](https://github.com/airbytehq/airbyte/pull/7125) | Fix incorrect handling of LONG RAW data type |
| 0.3.7 | 2021-10-01 | [6616](https://github.com/airbytehq/airbyte/pull/6616) | Added network encryption options |
| 0.3.6 | 2021-09-30 | [6585](https://github.com/airbytehq/airbyte/pull/6585) | Improved SSH Tunnel key generation steps |
| 0.3.5 | 2021-09-22 | [6356](https://github.com/airbytehq/airbyte/pull/6356) | Added option to connect to DB via SSH. |
Expand Down