All my self-trained sisr Models

[Phhofm]

For convenience, I thought Id provide here a compressed archive file with most of my released (single image super resolution) models in it.

This is meant for inference, and most of these can simply be run with chaiNNer, which would be my recommendation.

Or they can be used in any other project that uses the Spandrel library, like chaiNNer, Automatic1111 Stable Diffusion Web UI, dgenearte, to some extend SD.Next (has its own chainner-extension), and maybe soon ComfyUI.

Most of these files are in the safetensors format. If you need them in another format (.pth, or onnx, or ncnn) you can convert most of these in chaiNNer.

This release currently contains 111 of my self-trained model files -
From my first released model 4xLSDIRCompact on the 11.03.2023
to my latest released model 4xRealWebPhoto_v4_dat2 on the 04.04.2024

You can find the details of these models on their respective release information in this repo (I am still in the process of creating github releases in this repo for my already released models, so i am still catching up, but you can find their info on openmodeldb or in our discord server's (Enhance Everything!) #model-releases channel history in the meantime)

Since the zip archive is currently 3.03GB, and Github Release allows a max file size of 2GB, this is split into currently 2 zip files with a max 2GB file size.

I will keep these archives updated by overwriting them here if I release newer models.

---

This discussion was created from the release All my self-trained sisr Models.

[przemoc]

Thank you for releasing all those SISR models! Keep doing the great job you do, it's appreciated by the community.

I have a few suggestions regarding phhofm_sisr_models archive. The idea is great, one can look at https://github.com/Phhofm/models/releases/tag/all_models instead of tracking each release separately all the time. Very useful. But regenerating one big archive every few weeks or months to add a few models seems a bit wasteful, both on your side (you re-archive and re-upload stuff that was already archived before) and on receiving side (folks need to download whole archive even if they want to get 5 new models as they downloaded older version, etc.).

From practical PoV it would be more efficient and manageable if the archive was meant to be incremental from the get go. To achieve that, please consider doing following stuff:

• preserve mtime (aka Last Write Time) of released models, and do not overwrite them by current time when creating archives; otherwise one cannot sort by date of modification and clearly see from when each model is
• prefer archive formats that support some form of indexing files or table of contents, so that people can check its content without downloading whole archive - tarball (.tar) is unfortunately not such format due to how it was designed, ZIP (.zip) is better in this regard due to its central directory
• instead of splitting one big archive post creation and recreating it from time to time, consider organizing them based on date of release, something like below, so that you only upload new archives and never have to delete old ones (once you'll switch to this way of organizing things and unless you want to combine some recent ones):

phhofm_sisr_models_2019-2021 - all SISR models released in 2019, 2020 and 2021, unless they do not fit in 2GB, then split into years:
phhofm_sisr_models_2019, phhofm_sisr_models_2020...
and if they still do not fit in 2GB, then split into quarters (3 months groups):
phhofm_sisr_models_2021--M01-03, phhofm_sisr_models_2021--M04-06, ...
(purposefully not using Q1, Q2, ... to have better alphabetical sorting)
and if they still do not fit in 2GB, then split into months:
phhofm_sisr_models_2021--M07
I feel like there is no point in going lower than month granularity, because if someone needs the latest one right now and cannot afford waiting, they can simply go to respective release page and download single model.
Such arrangement would require to never prepare and upload archive for given month, quarter or year unless it has already finished, because basic concept is to never alter already sent archive. If I already have file phhofm_sisr_models_2021--M07.zip locally, then I could rely on the fact that same named file on GH would never diverge and have more files for instance at some point in future.

After some time you could combine recent months into quarters if you'd like to (the only time you would be re-archiving and reuploading some stuff, but only fraction, not everything) and they would still fit in 2GB limit, and with that users would be still able to easily figure out that they do not need to download M07-09 if they already had separate M07, M08 and M09, etc., so it would be more of a convenience to folks that are downloading your models for the very first time, so they would have to download a few files less. But it would be optional and not necessary on your side.

[Phhofm]

Hey, thank you for writing and your suggestions :)

You are absolutely right, what I am doing is very suboptimal, I agree.
Having archives based on release date that I wont update after makes a lot of sense, Ill see when I can apply it.
(PS At the beginning I had zip files in here, but one time when I was uploading the new zip files as an asset, it was extremely (like unusable for 2gb files) slow. Same with other archive types it tried like 7z. But upload speed was normal with tar, this is why I had uploaded tar archives because I could upload them in a reasonable time frame in contrast.)

Was also thinking, since all assets in the archives are files that are already uploaded on github, I could instead maybe make a python script that, given a folder path, checks the files in that folder and downloads all my yet missing models into the folder.
Syncing a local folder to be up-to-date and including all my latest models by downloading the respective files. Then the step of me manually archiving and uploading these archive files here would be omitted. On a new release I could just add an new entry in the python script. A user could get all my (new, or missing) models at once by running the script, and maintenance effort for me should be pretty minimal this way. That was just another idea.

Anyway thanks again for your thoughts :)

[Phhofm]

Thanks for your input, I updated the release now: https://github.com/Phhofm/models/releases/tag/all_models

I replaced all archive files with a python script that syncs an input folder with my models, basically concurrently downloading all my models into a specified folder, skipping all already existing files.

This python script also self-updates at the beginning of execution, so the user of the script automatically gets the newest version/newest releases and does not need to manually check for new model releases of mine or for a new version of this script.

[przemoc]

Thank you for creating the script.

Please consider supporting preserving release date in downloaded files, so that looking at mtime / Last Write Time / Date Modified attribute of file (column in your favorite file manager) tells when the model was actually released (I don't feel like preserving time of release in given day gives that much additional value, so just defaulting to noon in UTC should be sufficient, date is what really matters, but feel free to keep time too if you want). I'm not sure if your GitHub upload dates are always the same as model release dates (I think you were creating some releases later, so the answer is most likely "no"). Therefore release date may be metadata worth being additionally stored in the script too.

_{Side-note: BTW I strongly suggest using ISO8601 (RFC 3339) date format %F aka %Y-%m-%d, e.g. 2024-09-11, for date purposes, which has advantage over others (like often used %d.%m.%Y in Europe or %m/%d/%Y in US) that its lexicographical order matches chronological order and is unambiguous (no concerns whether day or month goes first). From engineering perspective it is the best format to display date and I recommend using it everywhere, especially in any IT-related contexts.}

You may also think about reorganizing your model map in the script to make it more manageable. E.g.

'4xNomos2_hq_drct-l': {
    'date': '2024-09-08',
    'name': '4xNomos2_hq_drct-l',
    'path': 'https://github.com/Phhofm/models/releases/download/4xNomos2_hq_drct-l/',
    'files': [
        '4xNomos2_hq_drct-l.pth',
        '4xNomos2_hq_drct-l.safetensors',
    ]
}

Then handling option 1, 2 or 3 only need have a logic to filter files entries accordingly (by appropriate suffixes) and build full paths from them.

In theory you could simplify it further by hardcoding path beginning, but then it would be less flexible if you started releasing some models outside of GitHub at some point, or if others would want to consume such map, so I think it's not worth it.

Such model map could be actually stored as JSON uploaded to your all_models release, which script would be downloading. Then most of the time you could update JSON instead of updating the whole script. And others could use such JSON in their apps to easily find your models without having to cut and parse python script for that.

[leonary]

@Phhofm Dear author, the models you have trained are truly numerous and incredibly useful. I have a request: the DRCT upscaling model(https://openmodeldb.info/models/4x-Nomos2-hq-drct-l) you developed does not perform well on images with noise, compression artifacts, or distortions. Could you kindly consider training a model similar to your HAT model(https://openmodeldb.info/models/4x-Nomos8kHAT-L-otf), enabling the DRCT model to work effectively on low-quality images? I would be immensely grateful for your help.