comment

rwth-i6 · Nov 19, 2023 · 6e3983b · 6e3983b
1 parent 1887d18
commit 6e3983b
Showing 1 changed file with 1 addition and 0 deletions.
diff --git a/users/zeyer/experiments/exp2023_04_25_rf/README.md b/users/zeyer/experiments/exp2023_04_25_rf/README.md
@@ -83,3 +83,4 @@ TODO model changes:
 - Second decoder LSTM
 - ZoneoutLSTM use_zoneout_output=True
 - (cnnblstmf2)
+- QK Norm (as in QK Norm paper with L2 norm, or as in Scaling ViT paper with LayerNorm)