Feature skip data iteration when caching scoring #557
Conversation
* indentation level * disable some pylint messages * unused fixtures
Before, net.infer was cached when using a scoring callback wiht use_caching=True. This way, the time to make an inference step was saved. However, there was still an iteration step over the data for each scoring callback. If iteration is slow, this could incur a significant overhead. Now net.forward_iter is cached instead. This way, the iteration over the data is skipped and the iteration overhead should be gone.
|
Can I tests this from some nightly build or do I have to build the package on my own? |
|
@marrrcin You would need to install from source, but it's not difficult: https://github.com/skorch-dev/skorch#from-source |
skorch/net.py
Outdated
| def _forward_output(self, yp, device): | ||
| if isinstance(yp, tuple): | ||
| return tuple(n.to(device) for n in yp) | ||
| return yp.to(device) |
There was a problem hiding this comment.
This is structurally very similar to skorch.utils.to_tensor. Maybe we should introduce a skorch.utils.to_device instead? This might also become handy if we support multiple GPUs in the future.
There was a problem hiding this comment.
Good point, I moved this to skorch.utils.to_device
Similar to the comment in cache_net_infer
... instead of having it as a method on NeuralNet. Add tests
|
@ottonemo I addressed your comment, pls review again. Before merging this, I should we consider making a new release? I wouldn't mind this new feature being only on master for some time in case it creates some trouble down the line. |
BenjaminBossan
deleted the
feature-skip-data-iteration-when-caching-scoring
branch
This is the proposed fix for #552.
As discussed there, my proposal is to cache
net.forward_iter, so that we can skip the iteration over the data.It would be very helpful if someone could think of any unintended side-effects this change could have. E.g., in theory, some code could rely on iterating over the data even in case of caching, but I can hardly imagine this happening in reality.
Also, I cannot test on GPU for the moment. I don't see how this would affect the outcome, but it could still be nice of someone verifies it works.
I deprecated
cache_net_inferbecause it was not "private". However, I made the new_cache_net_forward_iterprivate.