Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce precompilation work in the function Buffer #377

Closed
wants to merge 3 commits into from
Closed

Reduce precompilation work in the function Buffer #377

wants to merge 3 commits into from

Conversation

ronisbr
Copy link
Contributor

@ronisbr ronisbr commented Sep 24, 2023

Hi!

I managed to reduce the precompilation work in Julia 1.9 by just rewriting the function that creates a Buffer. Unfortunately, I have no ideia if those actions have performance consequences in other scenarios since I only have access to one MSSQL server (I cannot install docker at the moment here). For my use case, there was no measurable performance degradation when fetching data, but the time to fetch the first package was greatly reduced.

Here is the timing without the modifications in Julia 1.9:

julia> @time using ODBC
  0.129348 seconds (161.98 k allocations: 12.275 MiB, 9.80% compilation time)

...

julia> function test()
       DBInterface.execute(conn, "SELECT * FROM SYSOBJECTS WHERE xtype = 'U'")
       return nothing
       end
test (generic function with 1 method)

julia> t₀ = time(); test(); t₁ = time(); println(t₁ - t₀)
49.72623920440674

Hence, it takes 50s to retrieve all the tables in the MSSQL server. The second time is way faster.

Here is the timing with the modifications in Julia 1.9:

@time using ODBC
  0.118692 seconds (159.37 k allocations: 12.150 MiB, 10.20% compilation time)

julia> function test()
       DBInterface.execute(conn, "SELECT * FROM SYSOBJECTS WHERE xtype = 'U'")
       return nothing
       end
test (generic function with 1 method)

julia> t₀ = time(); test(); t₁ = time(); println(t₁ - t₀)
7.495282888412476

Note: I could reduce even further the time to fetch the first package here using precompilation statements. However, it did not provide useful gains, because the time to load the package went from 0.11s to more than 5s.

Closes #371

They were adding a huge amount of time during the package loading.
We rewrote the function to create a Buffer to reduce the compilation
time.
@ronisbr ronisbr changed the title Precompilation Reduce precompilation work in the function Buffer Sep 24, 2023
@ronisbr
Copy link
Contributor Author

ronisbr commented Sep 24, 2023

I really have no ideia why the tests are failing. The function:

Buffer(ctype::API.SQLSMALLINT, columnsize, rows, nullable)

provides the same answer before and after modifications for all possible inputs.

@ronisbr
Copy link
Contributor Author

ronisbr commented Sep 24, 2023

I triggered the tests in my fork without the modifications in this PR and they are also failing. Hence, it is not be related to this PR.

@ronisbr
Copy link
Contributor Author

ronisbr commented Oct 6, 2023

Bump!

@ronisbr
Copy link
Contributor Author

ronisbr commented Oct 29, 2023

Bump v2 :)

@ronisbr
Copy link
Contributor Author

ronisbr commented Nov 8, 2023

We do not need this anymore apparently. I tested in Julia v1.10-RC1 and this time was reduced to 0.7s!

@ronisbr ronisbr closed this Nov 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DBInterface.execute is taking a lot of time to pre-compile in Julia 1.9
1 participant