[SPARK-1103] [WIP] Automatic garbage collection of RDD, shuffle and broadcast data#126

Closed

tdas wants to merge 51 commits intoapache:masterfrom tdas:state-cleanup

+2,571-469

Commits on Feb 5, 2014

Added unpersist method to Broadcast.
Roman Pastukhov
committed

Commits on Feb 6, 2014

Fix for Broadcast unpersist patch.
Roman Pastukhov
committed

Commits on Feb 14, 2014

Added ContextCleaner to automatically clean RDDs and shuffles when they fall out of scope. Also replaced TimeStampedHashMap to BoundedHashMaps and TimeStampedWeakValueHashMap for the necessary hash…
tdas
committed
Changed TimeStampedHashMap to use WrappedJavaHashMap.
tdas
committed

Commits on Mar 11, 2014

Commits on Mar 12, 2014

Removed unncessary TimeStampedHashMap from DAGScheduler, added try-catches in finalize() methods, and replaced ArrayBlockingQueue to LinkedBlockingQueue to avoid blocking in Java's finalizing thread.
tdas
committed

Commits on Mar 13, 2014

Modifications based on the comments on PR 126.
tdas
committed

Commits on Mar 17, 2014

Added try-catch in context cleaner and null value cleaning in TimeStampedWeakValueHashMap.
tdas
committed

Commits on Mar 18, 2014

Removed use of BoundedHashMap, and made BlockManagerSlaveActor cleanup shuffle metadata in MapOutputTrackerWorker.
tdas
committed

Commits on Mar 19, 2014

Style fix
tdas
committed

Commits on Mar 27, 2014

Commits on Mar 28, 2014

Commits on Mar 29, 2014

Commits on Mar 30, 2014

Merge github.com:apache/spark into cleanup
andrewor14
committed

Commits on Mar 31, 2014

Commits on Apr 1, 2014

Address TD's comments
andrewor14
committed

Commits on Apr 2, 2014

Commits on Apr 8, 2014