Applying hooks for activation volumes

[johanenforcer]

I am currently trying to implement a pytorch hook to retrieve the activation volume from certain layers, in this case I would like to retrieve the activation output from the penultimate module in specifically the yolov5m.yaml file. Specifically this is the C3 module and in this module I would like to get the activations from self.cv3, so the last convolution layer in this module. Since the modules are called and used from the yolo.py file, surely I would have to define the forward hook in this file, but I am not certain exactly where, as it feels weird to do it in either the init or forward instance method of the Model class.

The forward hook would look something like this:

activation = {}
def get_activation(name):
    def hook(model, input, output):
        activation[name] = output.detach()
    return hook

self.model[-2].cv3.register_forward_hook(get_activation('embedding'))  

Thanks in advance!

[glenn-jocher]

@johanenforcer this is a good question. I don't use many hooks myself so I can't really help you, but if you get this working please consider submitting a PR to help everyone out who is also trying to visualize feature maps!

[johanenforcer]

I know that its maybe hard to answer but do you have any hints on maybe what one might try? Otherwise, I guess a pretty bad way of circumventing this issue might be to implement something directly into the model architechture but that is a really ugly solution maybe

[glenn-jocher]

@johanenforcer well, we have requests often for 'feature map visualization', though the details seem to be pretty vague about what users are looking for.

Since there's naturally hundreds of layers, and each layer may have thousands of channels, the amount of viewable information is enormous.

We could try to insert a flag to save feature maps to jpg, or to matplotlib plot matrices, but as your saying this would normally involve introducing code into the model forward method, i.e.

if self.save_features:
    ...