Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SqliteCommand allocates byte array per every query or prepare #26296

Open
vonzshik opened this issue Oct 9, 2021 · 4 comments
Open

SqliteCommand allocates byte array per every query or prepare #26296

vonzshik opened this issue Oct 9, 2021 · 4 comments

Comments

@vonzshik
Copy link
Contributor

vonzshik commented Oct 9, 2021

var sql = new byte[byteCount + 1];

I see 2 possible ways to remove this allocation:
a) ArrayPool<byte>.Shared
b) Allocate an array and keep it on bound connection

To limit the potential memory consumption after a big query there can be put a maximum limit for an array length (1mb ?), and in case of hitting it the array is allocated and dropped as it's done right now.

@roji
Copy link
Member

roji commented Oct 9, 2021

This doesn't seem to escape the method, so stackalloc? Since the byteCount could be anything, we'd limit to some reasonable limit, and beyond that ArrayPool may be sufficient...

@vonzshik
Copy link
Contributor Author

vonzshik commented Oct 9, 2021

One problem with ArrayPool is that you can't control whenever it's actually going to be purged, so you may get stuck with a huge array for quite a while (see dotnet/runtime#52098, dotnet/runtime#853 and dotnet/runtime#58974 for a few other issues).

@vonzshik
Copy link
Contributor Author

Sadly, stackalloc isn't going to cut here since you can't pass a stackalloc array to sqlite3_prepare_v2.

@bricelam
Copy link
Contributor

I have bigger plans to cache the conversion to UTF8 as part of #14044

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants