"sshs_model_hash" printed on the LoRa header is wrong and causes a lot of mismatch problems with Civitai cross posting and Auto1111

[diodiogod]

your trainer is printing a wrong sshs_model_hash on any LoRa trained: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 this makes every model trained and uploaded to Civitai reference this random lora: https://civitai.com/models/16612?modelVersionId=19613

Automatic1111 and Forge will respect the hash printed on the lora header, but Civitai will calculate the correct one (consdering the file after the header) and this will cause a missmatch.

Erasing the "sshs_model_hash" from the file header for exemple using this https://xypher7.github.io/lora-metadata-viewer/
Will NOT solve the problem because now Forge will calculate the hash based on the file (from my understanding I'm not a dev). And since you changed the file header, you changed the hash. So now you get images printed with the wrong hash, again.

As of right now the only way I could think of solving cross posting on Civitai is:

1- upload the Lora on civitai (because I don't know any other way to get the correct hash, 7zip hash for example will show me the file hash not the hash disregarding the header)

2-wait to scan and give me the correct hash

3-remove the file from civitai

4-edit the 12 char hash of the header (that would still technically be fake because the rest is wrong) https://xypher7.github.io/lora-metadata-viewer/

5- upload again on Civitai

This is a lot of trouble, and the damage is kind of done already for the previous Loras uploaded with this trainer. I don't see how that could be fixed for the old ones, but maybe it can be fixed for the new ones. I hope. lol
Thanks!!

[ali0une]

Hi there.

Please try to fix this, many LoRA created with Ai-Toolkit share this same e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 sha256sum and it's really anoying when recalling generated images to have the wrong LoRA added to the prompt.

[Brioch]

e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 is the SHA256 hash of an empty string so there's obviously something wrong in the metadata generation.

I hacked together a script to fix the hash of a safetensors LoRA file. Download the file https://gist.github.com/Brioch/aa9600b470a0c66b3345f6030539d4c9 to your ai-toolkit dir then run it with python fix_safetensors_hash.py your_model.safetensors. I'm no python developer so backup your lora before trying it. It will print infos about expected sshs_model_hash and the actual one and create a new safetensors file with the fixed hash.

[diodiogod]

e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 is the SHA256 hash of an empty string so there's obviously something wrong in the metadata generation.

I hacked together a script to fix the hash of a safetensors LoRA file. Download the file https://gist.github.com/Brioch/aa9600b470a0c66b3345f6030539d4c9 to your ai-toolkit dir then run it with python fix_safetensors_hash.py your_model.safetensors. I'm no python developer so backup your lora before trying it. It will print infos about expected sshs_model_hash and the actual one and create a new safetensors file with the fixed hash.

Oh wow, thank you so much, I'll try it for sure for my next LoRa. I hope Ostris will implement this or some other fix soon.

[acezard]

e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 is the SHA256 hash of an empty string so there's obviously something wrong in the metadata generation.

I hacked together a script to fix the hash of a safetensors LoRA file. Download the file https://gist.github.com/Brioch/aa9600b470a0c66b3345f6030539d4c9 to your ai-toolkit dir then run it with python fix_safetensors_hash.py your_model.safetensors. I'm no python developer so backup your lora before trying it. It will print infos about expected sshs_model_hash and the actual one and create a new safetensors file with the fixed hash.

I am no Python dev either, I had to update the main function like this with the help of chatgpt4:

import os

def main():
    if len(sys.argv) < 2:
        print("Usage: python fix_safetensors_hash.py <path_to_safetensors_file>")
        return

    file = sys.argv[1]
    metadata = load_metadata_from_safetensors(file)
    
    # Retrieve the original hash from metadata
    hash = metadata.get("sshs_model_hash", None)
    if not hash:
        print("No 'sshs_model_hash' found in metadata.")
        return
    
    print(f"sshs_model_hash of the file is: {hash}")

    # Compute the current hash of the file
    hash_value = load_and_hash_safetensors(file)

    if hash_value != hash:
        print(f"sshs_model_hash of the file should be: {hash_value}")

        metadata["sshs_model_hash"] = hash_value
        metadata = {k: str(v) for k, v in metadata.items()}
        tensors = {}

        with safe_open(file, framework="pt") as f:
            for k in f.keys():
                tensors[k] = f.get_tensor(k)

        # Create new filename
        directory = os.path.dirname(file)
        base_name = os.path.basename(file)
        newFile = os.path.join(directory, "fixed_" + base_name)

        # Ensure the output directory exists (though it should if input is valid)
        os.makedirs(directory, exist_ok=True)

        # Save the fixed file
        save_file(tensors, newFile, metadata)

        print(f"Fixed file saved to {newFile}. New hash is {hash_value}.")
    else:
        print("Hashes match.")

if __name__ == "__main__":
    main()

else it would fail like that:

(venv) anc@desktop:~/ai-toolkit$ python fix_safetensors_hash.py output/lora_flux_24gb/lora_flux_24gb.safetensors
sshs_model_hash of the file is: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
sshs_model_hash of the file should be: 7da01367640aae9956da7a29e3db483063b2e8d3b9a27efaf2929a159e9fd9d2
Traceback (most recent call last):
  File "/home/anc/ai-toolkit/fix_safetensors_hash.py", line 85, in <module>
    main()
  File "/home/anc/ai-toolkit/fix_safetensors_hash.py", line 78, in main
    save_file(tensors, newFile, metadata)
  File "/home/anc/ai-toolkit/venv/lib/python3.12/site-packages/safetensors/torch.py", line 286, in save_file
    serialize_file(_flatten(tensors), filename, metadata=metadata)
safetensors_rust.SafetensorError: Error while serializing: IoError(Os { code: 2, kind: NotFound, message: "No such file or directory" })
[ble: exit 1]

After this change it now seems to work and create a lora with correct hash 👍 :

(venv) anc@desktop:~/ai-toolkit$ python fix_safetensors_hash.py output/lora_flux_24gb/lora_flux_24gb.safetensors
sshs_model_hash of the file is: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
sshs_model_hash of the file should be: 7da01367640aae9956da7a29e3db483063b2e8d3b9a27efaf2929a159e9fd9d2
Fixed file saved to output/lora_flux_24gb/fixed_lora_flux_24gb.safetensors. New hash is 7da01367640aae9956da7a29e3db483063b2e8d3b9a27efaf2929a159e9fd9d2.

[Brioch]

After this change it now seems to work and create a lora with correct hash 👍 :

@acezard Fair enough, I forgot to say it expected the file to be in the same directory as the script. Thanks for the fix. I have updated my gist.

[ali0une]

@acezard, @Brioch many thanks for providing a way to fix the already trained LoRAs. Works 100% for me.

ai-toolkit still needs to be fixed not to generate this bogus sshs_model_hash.

[DeverStyle]

your trainer is printing a wrong sshs_model_hash on any LoRa trained: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 this makes every model trained and uploaded to Civitai reference this random lora: https://civitai.com/models/16612?modelVersionId=19613

Automatic1111 and Forge will respect the hash printed on the lora header, but Civitai will calculate the correct one (consdering the file after the header) and this will cause a missmatch.

Erasing the "sshs_model_hash" from the file header for exemple using this https://xypher7.github.io/lora-metadata-viewer/ Will NOT solve the problem because now Forge will calculate the hash based on the file (from my understanding I'm not a dev). And since you changed the file header, you changed the hash. So now you get images printed with the wrong hash, again.

As of right now the only way I could think of solving cross posting on Civitai is:

1- upload the Lora on civitai (because I don't know any other way to get the correct hash, 7zip hash for example will show me the file hash not the hash disregarding the header)

2-wait to scan and give me the correct hash

3-remove the file from civitai

4-edit the 12 char hash of the header (that would still technically be fake because the rest is wrong) https://xypher7.github.io/lora-metadata-viewer/

5- upload again on Civitai

This is a lot of trouble, and the damage is kind of done already for the previous Loras uploaded with this trainer. I don't see how that could be fixed for the old ones, but maybe it can be fixed for the new ones. I hope. lol Thanks!!

Using the metadata viewer, a lot of the info is just wrong, this is what I get

{
  "format": "pt",
  "software": {
    "name": "ai-toolkit",
    "repo": "https://github.com/ostris/ai-toolkit",
    "version": "0.1.0"
  },
  "version": 1,
  "sshs_model_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "sshs_legacy_hash": "e3b0c442",
  "ss_output_name": "my_lora_name",
  "name": "my_lora_name",
  "ss_base_model_version": "sd_1.5",
  "training_info": {
    "step": 500,
    "epoch": 499
  }
}

[diodiogod]

Isn't this really fixed yet? At least remove all this wrong metadata and leave it empty... it would be better.