Out of memory when running in dotnet-alpine docker image

[mldisibio]

Posting as a discussion because I probably cannot supply enough detail for an issue, and really just looking for 'yeah that's possible'.
We know alpine images use MUSL instead of GLIBC. We know sqlite uses malloc() or realloc() when it needs memory. Is it possible that when compiling with dotnet -r linux-musl-x64 the e_sqlite library cannot correctly request memory from the docker alpine image?

I run an etl pipeline in docker/dotnet that concatenates sqlite tables into an attached master opened as an in-memory db. So yes, each process needs about 200Mb db memory. I run about 16 parallel processes each with their own instance of sqlite db. My host has over 1T memory, and this etl only needs a minimal workingset amount, less than 32G max.

I have a dotnet 3.1 container on debian using the 2.0.4 sqlitepcl.raw. The container is allocated 600Gb memory. The pipeline consumes max 32Gb memory and runs just processing the 16 databases in parallel. It can run for 24 hours without failing.

I took the exact same dotnet code using sqlitepcl.raw, compiled with linux-musl-x64 option, and ran on an dotnet/alpine 3.1 image with the same memory configuration. When the memory usage reached 7Gb after about 20 minutes, sqlite crashed with 7-SQLITE_NOMEM out of memory on all its threads. I never received that error when running on debian.

Not really looking to post a bug. Looking more if that sounds familiar, or possible, and if the bottom line advice is simply not to use alpine for memory intensive sqlite apps?

[ericsink]

Hmmm.

I'd say "yeah that's possible".

But mostly because there are so many pieces in play, I think a LOT of things are possible.

No, this problem doesn't sound familiar.

And I wouldn't go so far as advising "simply not to use alpine for memory intensive sqlite apps" without a clearer understanding of what is actually happening.

[mldisibio]

Some Links Sowing Doubt about Allocation Memory in Thread Intensive Apps with Musl
The author of musl has enumerated some issues with malloc, and apparently a rewrite was attempted about two years ago?
malloc expand heap
whats wrong with malloc
new spec
problems with multi-core, large mem performance
That last discussion article makes me wonder if musl's malloc qualifies as an 'alternative memory allocator' and if sqlite needs to be made aware of it

Someone else who gave up on musl with multi-threading and its jemalloc memory allocator

The issue may indeed be more the multithreading than actual memory requests, but the two are closely related as shown in this musl multi-threading issues benchmarked

Conclusion: seems to be an anecdotally documented issue with the musl library itself, and running parallel threads using sqlite on an alpine image is just another manifestation.