Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change modality one-hot encoding to a learnt encoding #26

Open
jacobbieker opened this issue Nov 29, 2021 · 0 comments
Open

Change modality one-hot encoding to a learnt encoding #26

jacobbieker opened this issue Nov 29, 2021 · 0 comments
Labels
enhancement New feature or request

Comments

@jacobbieker
Copy link
Member

Detailed Description

AS @JackKelly pointed out, the PerceiverIO paper uses a learnt embedding for the modality, while this uses a one-hot vector instead. Switching to a learnt embedding might be better?

Context

Possible Implementation

A torch.nn.Embedding with a vocab size set to the number of data sources

@jacobbieker jacobbieker added the enhancement New feature or request label Nov 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant