Fix mup for the layers with AttentionLayerMup#494

Merged

DomInvivo merged 8 commits intomainfrom fix_mup_attn

Dec 24, 2023

+29-11

Commits on Dec 19, 2023

Added rescaling of embed_dim. And force embed_dim to be the same as in_dim in gps layer
DomInvivo
committed

Commits on Dec 24, 2023

Removed networkx inenv.yml
DomInvivo
authored