Massive memory leak with --localUse

[LaurentTreguier]

I am posting this issue here because it's with DCD that I experienced this, and that I seem to be able to reproduce the issue; however this could probably be a dsymbol, dparse or containers issue.
The crash occurs after building the server side with DMD on release (or release-debug) mode, when launching the server with:

./bin/dcd-server -I libdparse/src -I /usr/include/dmd/druntime/import -I /usr/include/dmd/phobos

(or wherever druntime and phobos are located), and then the client with:

./bin/dcd-client -u -c 2620 libdparse/src/dparse/ast.d

This cursor location points to the ASTVisitor symbol, which is all over the file, with 222 occurrences, to try to find local uses.
If the crash doesn't happen (for example by compiling in debug mode on my machine, or with LDC), another effect can be seen instead: memory usage goes through the roof. Up to almost 600Mb in my tests, each subsequent call to the client command bumping memory usage by 15 to 20MB, and it doesn't seem to ever go down.

The backtrace I captured, if it's of any use: bt.txt

Tested on Linux x86_64, with DMD 2.085.0

[ghost]

The crash only happens from DMD 2.085. Previously there was "just" the leak. DMD 2.085 comes with several GC changes, which reveal the bug.

[ghost]

The crash is unrelated, #589

[ghost]

The crash is fixed for 2.085.1 (or right now in stable). Time to fix the leak.

[LaurentTreguier]

Great !
Regarding the leak, from what I tested in DLS, it seems to happen when repeatedly using generateAutocompleteTrees()

[ghost]

Yes, I also think it's in dsymbol or at least not in DCD. I tried to tweak the DCD code for local use (ref in foreach, rewrite SymbolStuff.destroy, more scoped!..., GC.minimize, etc) and it didn't change anything.

Now one thing is that for the particular case of the ASTVisitor symbol and --localUse: as you noticed, there's about 200 calls to generateAutocompleteTrees() and the memory usage blows. But you can leave the editor running for hours, make hundreds of requests, that will also call generateAutocompleteTrees() and the same effect cant be observed.

A second interesting thing: if you make the --localUse request a second time, the memory usage does not explode again.