Fix rpn memory leak and dataType errors. #1657
Conversation
Codecov Report
@@ Coverage Diff @@
## master #1657 +/- ##
==========================================
+ Coverage 66% 66.02% +0.01%
==========================================
Files 92 92
Lines 7330 7331 +1
Branches 1107 1107
==========================================
+ Hits 4838 4840 +2
+ Misses 2176 2175 -1
Partials 316 316
Continue to review full report at Codecov.
|
| @@ -163,6 +163,8 @@ def forward(self, image_list, feature_maps): | |||
| anchors_in_image.append(anchors_per_feature_map) | |||
| anchors.append(anchors_in_image) | |||
| anchors = [torch.cat(anchors_per_image) for anchors_per_image in anchors] | |||
| # Clear the cache in case that memory leaks. | |||
| self._cache.clear() | |||
There was a problem hiding this comment.
I think this is a bit too drastic: the size of the cache is in general fairly small (less than 1MB per key), and we can get some speedup if we cache the most used sizes. This is effectively removing the caching altogether.
Maybe for a follow-up PR: What about using some variant of functools.lru_cache instead of manually keeping a dictionary for the cache?
This way, we could potentially keep save the 32 last sizes.
Current version causes memory leak, which can be easily repeated using the repository here. The reason is that
rpn._cachekeeps the pair with the model and the number of tensors it holds expands as proposal grows;During the process fixing the first problem, I found some data type errors in the development version and fix it.