You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I was iterating over thousands of bzip2 files with a function like this:
function read_first_line(filename)
open(filename) do f
f2 = CodecBzip2.Bzip2DecompressorStream(f)
r = first(eachline(f2))
# close(f2)
end
end
The original form (with close(f2) commented out) would leak over 100 GB memory and begin swapping. Calling GC.gc() had no effect on the memory usage. But when using close(f2), the julia process memory usage remains under 1 GB.
So I'm guessing some memory isn't getting cleared up properly. I would expect that the f2 decompressed stream should be cleared from memory at some point after it goes out of scope, at least before the Julia process runs out of system memory or when GC.gc() is run.
The text was updated successfully, but these errors were encountered:
I was iterating over thousands of bzip2 files with a function like this:
The original form (with
close(f2)
commented out) would leak over 100 GB memory and begin swapping. Calling GC.gc() had no effect on the memory usage. But when usingclose(f2)
, the julia process memory usage remains under 1 GB.So I'm guessing some memory isn't getting cleared up properly. I would expect that the
f2
decompressed stream should be cleared from memory at some point after it goes out of scope, at least before the Julia process runs out of system memory or whenGC.gc()
is run.The text was updated successfully, but these errors were encountered: