Reproduciblity while using a pre-traineed model converted to analog.

[matifali]

For inference, while using an ideal device, even setting the rpu_config.device.construction_seed=0 gives a different test accuracy on every run.

What can I do to make my results reproducible?

[maljoras]

The construction seed is only setting the seed for the construction, which includes device to device variability for analog training. In case of ideal device it thus does not do anything. The PcmLikeNoise model uses the torch seed during inference. So if you set e.g. torch.manual_seed(123) then you should get reproducible results. However, in case of a non-ideal forward, an internal C++ seed is used additionally, which currently is not accessible from python. So then only the programing noise / drift is reproducible (using the torch seed) but not the analog MVM noise (e.g. forward.out_noise or forward.w_noise).

[matifali]

Thanks for your answer. Yes, I am setting the torch.manual_seed(SOME_SEED), but the results are not reproducible. Yes, I am using out_noise and w_noise during hardware-aware training, but for inference of the same model, I am setting both of them to zero and only applying PcmLikeNoise for programming and drift noise.

What could be wrong here?