Missing test h5ad files needed for continue pretraining pipeline

[WhenMelancholy]

Hi Team, I greatly appreciate your work on this project. The code is very well organized and the documentation is comprehensive, which made it easy to get started. Thank you for making this valuable contribution open source.

While implementing the continue pretraining pipeline, I noticed that some h5ad files required for testing are missing. Specifically, I encountered the following error when trying to run the validation:

FileNotFoundError: [Errno 2] Unable to synchronously open file (unable to open file: name = '.../data/gNNpgpo6gATjuxTE7CCp.h5ad', errno = 2, error message = 'No such file or directory')

I also found there are some other datasets used for validation but not included in the repository:

# testdatasets=['/R4ZHoQegxXdSFNFY5LGe.h5ad', '/SHV11AEetZOms4Wh7Ehb.h5ad',
# '/V6DPJx8rP3wWRQ43LMHb.h5ad', '/Gz5G2ETTEuuRDgwm7brA.h5ad', '/YyBdEsN89p2aF4xJY1CW.h5ad',
# '/SO5yBTUDBgkAmz0QbG8K.h5ad', '/r4iCehg3Tw5IbCLiCIbl.h5ad', '/SqvXr3i3PGXM8toXzUf9.h5ad',
# '/REIyQZE6OMZm1S3W2Dxi.h5ad', '/rYZ7gs0E0cqPOLONC8ia.h5ad', '/FcwMDDbAQPNYIjcYNxoc.h5ad',
# '/fvU5BAMJrm7vrgDmZM0z.h5ad', '/gNNpgpo6gATjuxTE7CCp.h5ad'],

I wasn't able to locate these test files in the repository. Would you be willing to provide these h5ad files or instructions on how to obtain them? This would be extremely helpful for those of us looking to build upon your excellent work.

Thanks again for creating and sharing this project!

[jkobject]

Hi @WhenMelancholy,

Oh right, this is quite omission on my side. to explain, these files are coming from the cellxgene lamindb dataset used during my run. I get them from cellxgene but their IDs as is make them completely useless for you.

I thus need to share with you these files, also I will update the package and README to showcase this:

• gNNpgpo6gATjuxTE7CCp #denoising test
• yBCKp6HmXuHa0cZptMo7 #omnipath grn test

respectively they are these 2 datasets on cellxgene, preprocessed with scdataloader's preprocessor:

• https://cellxgene.cziscience.com/e/322c816f-608a-4da1-99c4-36ca4ad45146.cxg/
• https://cellxgene.cziscience.com/e/02f8bb42-0322-4906-b9ad-b92a5ce73a2d.cxg/

[jkobject]

for simplicity for you, I have also moved them to hf:
https://huggingface.co/jkobject/scPRINT/tree/main