Experiment with different mallocs

[ViralBShah]

There are some very fast mallocs out there:

http://www.canonware.com/jemalloc/
http://goog-perftools.sourceforge.net/doc/tcmalloc.html

It would be useful to try out these instead of the system malloc, to see if we get a real performance boost. I do feel that using jemalloc, rather than the system one, may also make performance and memory behaviour more uniform across different OSes.

I see use of malloc in src/, src/flisp, and src/support. It would also be nice to refactor the code so that other malloc implementations can be experimented with easily.

I guess the libraries in external will continue using their own malloc.

[ViralBShah]

This doesn't seem necessary unless we have performance measurements that require the use of a faster malloc.

[oschulz]

Just saw that Google published a revamped version of TCMalloc (https://github.com/google/tcmalloc) - given current core counts, would a multi-threaded malloc like that be beneficial for Julia?

[oschulz]

See also https://twitter.com/dvyukov/status/1227874576434110471

[yuyichao]

It'll certainly not benefit most uses since they don't use malloc anyway.

[oschulz]

Ah, thanks - was just curious.

[oscardssmith]

Slack today came up with a benchmark where our allocation is really slow. If you make a 100000000 element undef vector repeatedly, Julia is about 10x slower than numpy. We believe the problem is the lack of interaction between the allocator and GC

[Seelengrab]

FYI, I "only" had a 2x drop in performance:

Projects $ python3
Python 3.7.5 (default, Apr 19 2020, 20:18:17)
[GCC 9.2.1 20191008] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import timeit
>>> timeit.timeit(stmt='np.empty(int(1e9), dtype=np.float64)', setup='import numpy as np')
66.47885020000103
>>> import numpy
>>> numpy.version.version
'1.18.3'
>>>
Projects $ julia -q
julia> versioninfo()
Julia Version 1.5.2
Commit 539f3ce943 (2020-09-23 23:17 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
Environment:
  JULIA_NUM_THREADS = 4

julia> using BenchmarkTools

julia>  dummy = zeros(Int64(1e9));

julia> @benchmark similar($dummy)
BenchmarkTools.Trial:
  memory estimate:  7.45 GiB
  allocs estimate:  2
  --------------
  minimum time:     114.000 μs (65.79% GC)
  median time:      162.300 μs (71.69% GC)
  mean time:        175.176 μs (76.59% GC)
  maximum time:     49.640 ms (99.85% GC)
  --------------
  samples:          10000
  evals/sample:     1

The vast amount of time is spent in GC, even for minimum time, which is why we suspect that the deallocation/reusing of allocated blocks is somehow slow.

[racinmat]

I wonder if #42566 would be solved or at least partially mitigated by using jemalloc, guys in LinkedIn were able to solve nasty memory leaks by replacing malloc from glibc with jemalloc https://engineering.linkedin.com/blog/2021/taming-memory-fragmentation-in-venice-with-jemalloc

[oschulz]

I'm not an expert on allocators, just came across this, in case it's of interest:

http://ithare.com/testing-memory-allocators-ptmalloc2-tcmalloc-hoard-jemalloc-while-trying-to-simulate-real-world-loads/

They seems to like jemalloc. :-) It's not a very recent comparison, though.