Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Static interpolator_wandb_ckpt_filename causes file not found error for interpolators with num_predictions!=20 #8

Closed
Saydemr opened this issue Sep 9, 2024 · 1 comment

Comments

@Saydemr
Copy link

Saydemr commented Sep 9, 2024

Hi,

If num_predictions is set to some other value than 20, the code cannot find the .ckpt file on wandb due to static nature of given checkpoint filenames. Readme assumes that if wandb.run_id is given for a cloud run, then the checkpoint file is automatically found. According to readme, filepath is only required for local .ckpt files.

Particularly this can be changed to best-val_${module.num_predictions}ens_mems_ipol_avg_crps.ckpt

Thanks.

@salvaRC
Copy link
Collaborator

salvaRC commented Oct 4, 2024

Thank you for pointing this out! This is indeed is a bit annoying but you could always fix it by explicitly specifying diffusion.interpolator_wandb_ckpt_filename=<ckpt-filename-on-wandb>, where <ckpt-filename-on-wandb> could be something like "best-val_20ens_mems_ipol_avg_crps.ckpt" or "best-val_10ens_mems_ipol_avg_crps.ckpt" or any other ckpt that you want to use.

Particularly this can be changed to best-val_${module.num_predictions}ens_mems_ipol_avg_crps.ckpt

The suggested fix would become a problem when the number of predictions during testing (e.g. 50) is different than what was used to validate the interpolator during its training (by default 20). Feel free to suggest alternative approaches though (e.g. modify the code that finds the ckpt on wandb to be able to search amongst all ckpts on wandb).

Let me know if you face any issues or have more suggestions!

@Saydemr Saydemr closed this as completed Oct 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants