Move convert.py to examples/convert-legacy-llama.py

[Galunid]

There's little use for this one, and a lot of confusion.

[ggerganov]

We should update readme instructions, ggml-ci and anything else that references the script. "GGUF My Repo" would also need an update: https://huggingface.co/spaces/ggml-org/gguf-my-repo/blob/main/app.py#L29

[mofosyne]

No torch?

My understanding is that it's for safetensors

[cebtenzzre]

Given that .pth stands for pytorch, I think this name could be misleading. Why not call it convert-llama.py?

[Galunid]

convert-legacy-llama.py maybe? I'd like to avoid giving impression that it can be used for converting llama 3

[Galunid]

No torch?

My understanding is that it's for safetensors

No torch as in no torch dependency, though I can see why it'd be confusing. I'll try to come up with something better. Suggestions welcome ;)

[Galunid]

Relevant huggingface space PR https://huggingface.co/spaces/ggml-org/gguf-my-repo/discussions/68

[teleprint-me]

I always just named it convert-torch.py because that's what it did. I mean, if you really want to be clear about it, convert-torch-to-gguf.py. The issue is that it supports safetensors as well and python 3.12 breaks the CRC compatibility with the torch conversion.

[compilade]

convert-hf-to-gguf.py imports should be changed to reflect the rename

llama.cpp/convert-hf-to-gguf.py

Line 27 in 88e06d0

from convert import LlamaHfVocab

[teleprint-me]

It should be moved to vocab.py. Anything related to the vocab that's still needed.

[mofosyne]

Hmmmm... Do we have a conversion reference table so people can identify what file type to use with each script?

If not for code repo docs maybe that's a wiki entry

[mofosyne]

Not against this idea as convert.py is indeed a bit confusing, but this PR is still appearing broken with the CI.

I do wonder if we could somehow consolidate all these into one convert.py... and have convert.py call specific convert-*.py based on detection of the model source etc...

[mofosyne]

If you decide to go though the consolidation approach, see if you can also consolidate some aspect like how we generate default naming convention gguf.

This will make it easier to update it as the standards evolve.

[teleprint-me]

Vocab factory is missing too. Not sure how to handle that yet.

[Galunid]

Vocab factory is missing too. Not sure how to handle that yet.

I don't think there's any use for vocab factory in convert-hf-to-gguf.py, nor any other scripts. I think it's fine to leave it in convert-llama-legacy.py since that's the only place we can expect to use it.

[compilade]

Looks good to me. But I think https://huggingface.co/spaces/ggml-org/gguf-my-repo/discussions/68 should be merged first to avoid breaking that HF space.

@ggerganov, @phymbert, @Vaibhavs10

[teleprint-me]

Awesome! I've been waiting for this to be merged so I can integrate this into my PR. This will be incredibly useful so I don't need to duplicate or reinvent APIs.

[mofosyne]

Hoping this wont break my PR #7499 (review)

Looks like a good idea and good to refactor at this stage. Let's merge in once the hugging face PR gets merged in.

---

But let's merge if hf have not merged theirs in still by the time the main branch is stable.

[Vaibhavs10]

Heya! @mofosyne, @Galunid and @ggerganov - I just merged a PR (https://huggingface.co/spaces/ggml-org/gguf-my-repo/discussions/73) to default to the convert-hf script.

Good from my side to merge this!

[Vaibhavs10]

Also, thanks a ton for taking this up and opening the PR! 🤗

[mofosyne]

Well, it's now or never. Let's see how many convert.py PRs we have to fix, but looking forward to a cleaner codebase.