feat: enable resume download for hf_hub_download

[lonelam]

enable auto resume download by default to prevent always restart on downloading large file with poor network connection.

[lopagela]

Suggesting to make it False by default in settings.py to do not explicitly require it in the chain of settings*.yaml

[pabloogc]

I think going for a very simple solution here is way better. Just making it always resume the download seems like a healthy default.

That way you don't have to overcomplicate with a setting that realistically nobody will need to change.

[lonelam]

@pabloogc , thank you for your review and suggestions. I agree that resuming downloads by default is a prudent approach. However, to address scenarios where resuming may not be ideal, such as when the model file in the HuggingFace repository has been updated, I propose adding an optional setting in the LocalSettings class. This would allow users to choose whether to resume downloads from the last breakpoint or start afresh, thereby ensuring flexibility and robustness in handling different situations.

[pabloogc]

@pabloogc , thank you for your review and suggestions. I agree that resuming downloads by default is a prudent approach. However, to address scenarios where resuming may not be ideal, such as when the model file in the HuggingFace repository has been updated, I propose adding an optional setting in the LocalSettings class. This would allow users to choose whether to resume downloads from the last breakpoint or start afresh, thereby ensuring flexibility and robustness in handling different situations.

I still wouldn't expose it as a setting, instead pass a parameter to the setup script scripts/setup --resume=true/false defaulting to true should be enough for 99.9% of scenarios.

[lonelam]

Well, '--resume'/'--no-resume' are added as arguments for the setup script in the new commit. The default value is true.