Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for returning the attention scores in encoder call. #1644

Open
mmilanifard opened this issue May 22, 2024 · 1 comment
Open

Add support for returning the attention scores in encoder call. #1644

mmilanifard opened this issue May 22, 2024 · 1 comment
Assignees
Labels
stat:contributions welcome Add this label to feature request issues so they are separated out from bug reporting issues type:feature New feature or request

Comments

@mmilanifard
Copy link

Is your feature request related to a problem? Please describe.
Need to extract self attention scores from a call to the encoder block.

Describe the solution you'd like
Add "return_attention_scores" arg to the call method of encoder (similar to the signature of the attention layer call). Return self attention scores if indicated.

Describe alternatives you've considered
No other way to cleanly extract attention scores IIUC.

Additional context
I can create PR myself if it helps. Not sure if this needs to be done on the decoder model in the same PR.

@SuryanarayanaY SuryanarayanaY added the type:feature New feature or request label May 23, 2024
@SuryanarayanaY
Copy link

Hi @mmilanifard ,

Please feel free to create a PR. Keras Dev team will review and update if has to be done on decoder side also. Thanks!

@mattdangerw mattdangerw added help wanted Extra attention is needed stat:contributions welcome Add this label to feature request issues so they are separated out from bug reporting issues and removed keras-team-review-pending help wanted Extra attention is needed labels Jul 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stat:contributions welcome Add this label to feature request issues so they are separated out from bug reporting issues type:feature New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants