Add support for returning the attention scores in encoder call. #1644
Labels
stat:contributions welcome
Add this label to feature request issues so they are separated out from bug reporting issues
type:feature
New feature or request
Is your feature request related to a problem? Please describe.
Need to extract self attention scores from a call to the encoder block.
Describe the solution you'd like
Add "return_attention_scores" arg to the call method of encoder (similar to the signature of the attention layer call). Return self attention scores if indicated.
Describe alternatives you've considered
No other way to cleanly extract attention scores IIUC.
Additional context
I can create PR myself if it helps. Not sure if this needs to be done on the decoder model in the same PR.
The text was updated successfully, but these errors were encountered: