Issue#2878: Adds Siamese Network example

[DeepC004]

Related to #2878

Description:
I have added an example on Siamese Networks in the examples folder along with requirements.txt and README.md.
I have tested the network locally with different parameters and set the default parameters to ones which perform the best.
Note: I am also working on Google Colab Notebook for the same and will add it to pytorch-ignite/examples soon.

Check list:

• New tests are added (if a new feature is added)
• New doc strings: description and/or example code are in RST format
• Documentation is updated (if required)

[DeepC004]

@vfdev-5 please check the PR. I am unable to understand the origin of error. Is the code formatter adding the following line, ✨ 1 file would be formatted, 267 files already formatted ✨, by itself since it does not show up when formatter is run locally

[vfdev-5]

@DeepC004 sure, I'll check the PR as soon as possible. To reformat please do

bash ./tests/run_code_style.sh install
bash ./tests/run_code_style.sh fmt

[DeepC004]

Thank you. I was running the code formatter in the tests folder earlier, hence the code formatter was not picking up all the files for formatting.

[vfdev-5]

Note: I am also working on Google Colab Notebook for the same and will add it to pytorch/examples soon.

@DeepC004 it should be added to https://github.com/pytorch-ignite/examples not pytorch/examples

See #2878 (comment)

[vfdev-5]

Thanks for the PR @DeepC004 !
I left few comments on improving the code and also suggested few changes. Let me know what you think.

[DeepC004]

@vfdev-5 I have made the changes. May you please review them

[vfdev-5]

@DeepC004 Thanks ! Can you share here training results (logs, metrics), does model converge etc ?

[vfdev-5]

@DeepC004 I just run it locally and it is very very slow. Can you run some profiling to get an idea where is a bottleneck ?
Try this tool: https://pytorch-ignite.ai/how-to-guides/03-time-profiling/#handler-based-profiling-using-handlerstimeprofiler

[DeepC004]

Yes, even I ran it locally before submitting and it took me a couple hours to complete the training (with cuda enabled). The loss drops down in ranges of 0.02 after the first epoch itself, and the average test loss came out to be around 0.00006 after 2 epochs of training. I will add time profiling to check for bottlenecks.

[vfdev-5]

Sounds good.

I will add time profiling to check for bottlenecks.

The point is not to add profiling into the code but just understand why GPU is almost idle during the run

[DeepC004]

I trained the model for 2 epochs and these are the results. Around 98.5% of time is spent in data flow tasks. I tried optimizations like using num_workers and setting pin_memory to true when creating DataLoader object, but there isn't much improvement. Is there any workaround for this problem?

[vfdev-5]

Thanks for the update, @DeepC004 !
The bottleneck is with dataset and __getitem__ method most probably. Can you check if we could improve the code, maybe take a look how it is done in other repos.

[DeepC004]

@vfdev-5 I have made the changes. The run time is much better now!

[vfdev-5]

@DeepC004 I left few more comments.
By the way have you committed the changes, because I see a lot of unaddressed comments

[DeepC004]

@DeepC004 I left few more comments. By the way have you committed the changes, because I see a lot of unaddressed comments

Oh, I believe I commited the wrong file with incomplete changes. I will update it.

[DeepC004]

@vfdev-5 May you please review the changes.

[vfdev-5]

@DeepC004 thanks for pinging. I left other comments to address

[vfdev-5]

Few other nits and it will be good to go

[vfdev-5]

LGTM, thanks @DeepC004
Next step is to integrate this example into CI.
Please, check this PR as example: #2899

[DeepC004]

I will look into it. Thank you!