Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Oracle performance and stability #437

Closed
hmiguim opened this issue Feb 14, 2020 · 1 comment
Closed

Improve Oracle performance and stability #437

hmiguim opened this issue Feb 14, 2020 · 1 comment
Assignees
Labels
enhancement Issue type oracle Specific database module

Comments

@hmiguim
Copy link
Member

hmiguim commented Feb 14, 2020

Problem:

We create a database in the Oracle DBMS with a single table with 5 columns one of them was a CLOB. We added 1 million rows to this table and export the database to SIARD. After fetching around 800.000 rows the JDBC driver started to utilize all the cores in the CPU and eventually the execution stopped with a deadlock.

Solution:

The Oracle JDBC driver have a property that can be set at the connection creation level or statement level called oracle.jdbc.defaultLobPrefetchSize. This property allows to configure how much of the LOB data is fetched the first time is requested. By default the value is 4K for BLOB and 4K to CLOB.

A performance benchmark can be found here.

The number of roundtrips that are made between the client and the server can decrease significantly if overriding this property value. By decreasing the number of roundtrips we also increase the performance.

A java property option was added dbptk.jdbc.oracle.lobPrefetchSize to allow the user to fine tune the DBPTK Developer to increase overall performance. Combine this property with dbptk.jdbc.fetchsize.default and tune the DBPTK Developer accordingly your system specifications and database specifications.

@hmiguim hmiguim added enhancement Issue type oracle Specific database module labels Feb 14, 2020
@hmiguim hmiguim self-assigned this Feb 14, 2020
hmiguim added a commit that referenced this issue Feb 14, 2020
…ter to Oracle JDBC import module #437 [skip ci]
@hmiguim
Copy link
Member Author

hmiguim commented Feb 14, 2020

Close at fd783b4

@hmiguim hmiguim closed this as completed Feb 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Issue type oracle Specific database module
Projects
None yet
Development

No branches or pull requests

1 participant