Seeing 'Parameter not found' when training UMT

[JianLiu91]

Hi,
I have trained a language model (with MLM). When I train UMT using the following command:

python train.py --exp_name unsupMT_enfr --dump_path ./dumped/ --reload_model 'my.pth,my.pth'

I get:

INFO - 10/24/19 09:43:11 - 0:00:05 - Parallel data - valid - en-fr: 3000
INFO - 10/24/19 09:43:11 - 0:00:05 - Parallel data - test - en-fr: 3003
INFO - 10/24/19 09:43:16 - 0:00:10 - Reloading encoder from ./dumped/test_enfr_mlm/vp9pq2birk/best-valid_mlm_ppl.pth ...
INFO - 10/24/19 09:43:20 - 0:00:14 - Reloading decoder from ./dumped/test_enfr_mlm/vp9pq2birk/best-valid_mlm_ppl.pth ...
WARNING - 10/24/19 09:43:20 - 0:00:15 - Parameter layer_norm15.0.weight not found.
WARNING - 10/24/19 09:43:20 - 0:00:15 - Parameter layer_norm15.0.bias not found.
WARNING - 10/24/19 09:43:20 - 0:00:15 - Parameter encoder_attn.0.q_lin.weight not found.
WARNING - 10/24/19 09:43:20 - 0:00:15 - Parameter encoder_attn.0.q_lin.bias not found.
WARNING - 10/24/19 09:43:20 - 0:00:15 - Parameter encoder_attn.0.k_lin.weight not found.
WARNING - 10/24/19 09:43:20 - 0:00:15 - Parameter encoder_attn.0.k_lin.bias not found.
WARNING - 10/24/19 09:43:20 - 0:00:15 - Parameter encoder_attn.0.v_lin.weight not found.
WARNING - 10/24/19 09:43:20 - 0:00:15 - Parameter encoder_attn.0.v_lin.bias not found.
WARNING - 10/24/19 09:43:20 - 0:00:15 - Parameter encoder_attn.0.out_lin.weight not found.
WARNING - 10/24/19 09:43:20 - 0:00:15 - Parameter encoder_attn.0.out_lin.bias not found.
WARNING - 10/24/19 09:43:20 - 0:00:15 - Parameter layer_norm15.1.weight not found.
WARNING - 10/24/19 09:43:20 - 0:00:15 - Parameter layer_norm15.1.bias not found.
WARNING - 10/24/19 09:43:20 - 0:00:15 - Parameter encoder_attn.1.q_lin.weight not found.
WARNING - 10/24/19 09:43:20 - 0:00:15 - Parameter encoder_attn.1.q_lin.bias not found.
WARNING - 10/24/19 09:43:20 - 0:00:15 - Parameter encoder_attn.1.k_lin.weight not found.
WARNING - 10/24/19 09:43:20 - 0:00:15 - Parameter encoder_attn.1.k_lin.bias not found.
..

It seems that the decoder is not initialized. Is it OK?

[glample]

Yes, these warning correspond to the source attention weights, that cannot be pretrained for the decoder. So this is fine.

[JianLiu91]

Yes, these warning correspond to the source attention weights, that cannot be pretrained for the decoder. So this is fine.

Hi, thank you very much for your reply. It addresses my concerns.

By the way, you provide a pre-trained en-fr MLM model https://dl.fbaipublicfiles.com/XLM/mlm_enfr_1024.pth.
How many epochs do you use to train it?