Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

convert-*.py: GGUF Naming Convention Refactor and Metadata Override Refactor #7499

Merged
merged 66 commits into from
Jul 18, 2024
Merged

convert-*.py: GGUF Naming Convention Refactor and Metadata Override Refactor #7499

merged 66 commits into from
Jul 18, 2024

Commits on Jul 15, 2024

  1. convert-*.py: licence -> license

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    d3a936f View commit details
    Browse the repository at this point in the history

  2. convert-*.py: add --get-outfile command and refactor

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    dbb1b47 View commit details
    Browse the repository at this point in the history

  3. convert-*.py: add basename and finetune metadata

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    a42c2b7 View commit details
    Browse the repository at this point in the history

  4. convert-*.py: model card metadata

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    916872f View commit details
    Browse the repository at this point in the history

  5. convert-*.py: metadata class moved to utility

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    4d5f18a View commit details
    Browse the repository at this point in the history

  6. convert-*.py: encoding_scheme --> output_type

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    5c263cb View commit details
    Browse the repository at this point in the history

  7. convert-*.py: parse model card in metadata util. Add license_link and… 

    … license_name to kv store
    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    b36e391 View commit details
    Browse the repository at this point in the history

  8. convert-*.py: add base_version and add tags

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    8f73408 View commit details
    Browse the repository at this point in the history

  9. convert-*.py: add parameter size class

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    0f1d50f View commit details
    Browse the repository at this point in the history

  10. convert-*.py: add datasets and language to KV store

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    684c604 View commit details
    Browse the repository at this point in the history

  11. convert-*.py: move per model weight estimation away from util back to… 

    … main script

plus some refactoring
    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    b1927ee View commit details
    Browse the repository at this point in the history

  12. convert-*.py: enable --model-name direct metadata override

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    f7c2079 View commit details
    Browse the repository at this point in the history

  13. convert-*.py: add general.organization to kv store

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    5a86dfa View commit details
    Browse the repository at this point in the history

  14. convert-*.py: add quantized_by and enhance heuristics

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    dd15712 View commit details
    Browse the repository at this point in the history

  15. convert-*.py: adjust help message

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    b0553f4 View commit details
    Browse the repository at this point in the history

  16. convert-*.py: use heuristics to parse _name_or_path

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    4d5cd06 View commit details
    Browse the repository at this point in the history

  17. convert-*.py: base_model is actually in spec for model cards

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    32e80e0 View commit details
    Browse the repository at this point in the history

  18. convert-*.py: refactor parameter weight class

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    54918ad View commit details
    Browse the repository at this point in the history

  19. convert-*.py: need to include self in per_model_weight_count_estimati… 

    …on()
    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    39472a0 View commit details
    Browse the repository at this point in the history

  20. convert-*.py: add heuristic to directory name fallback 

    Also add source_url for huggingface url
    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    3625a42 View commit details
    Browse the repository at this point in the history

  21. convert-*.py: add unittest to metadata class

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    91e65d9 View commit details
    Browse the repository at this point in the history

  22. convert-*.py: adjusted authorship KV store

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    d060fcd View commit details
    Browse the repository at this point in the history

  23. convert-*.py: separated unit test, hf_repo to repo_url

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    eaa47f5 View commit details
    Browse the repository at this point in the history

  24. convert-*.py: Remove self.model_name that was left in since last rebase

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    e973443 View commit details
    Browse the repository at this point in the history

  25. convert_hf_to_gguf.py: optional, dataclass removed from type as it wa… 

    …s unused
    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    5011eef View commit details
    Browse the repository at this point in the history

  26. convert_hf_to_gguf.py: rebase error correction

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    2f23927 View commit details
    Browse the repository at this point in the history

  27. convert_hf_to_gguf.py: Remove code that is already in fill_templated_… 

    …filename() and GGUFWriter()
    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    4dc8ddd View commit details
    Browse the repository at this point in the history

  28. gguf_writer.py: generate tensor uuid if missing

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    007708e View commit details
    Browse the repository at this point in the history

  29. test: remove test_gguf.py and remove test_generate_any_missing_uuid()

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    7ecb8f0 View commit details
    Browse the repository at this point in the history

  30. convert-*.py: autogenerate general.uuid if missing

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    fdc5a3f View commit details
    Browse the repository at this point in the history

  31. convert-*.py: write_tensors() --> prepare_tensors_for_writing()

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    2a976e1 View commit details
    Browse the repository at this point in the history

  32. convert-*.py: refactor per model weight count estimation

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    59a01df View commit details
    Browse the repository at this point in the history

  33. convert-*.py: pyright type fixes

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    dd14b8f View commit details
    Browse the repository at this point in the history

  34. Apply suggestions from code review 

    Co-authored-by: compilade <git@compilade.net>
    @mofosyne @compilade
    mofosyne and compilade committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    74383ba View commit details
    Browse the repository at this point in the history

  35. convert-*.py: cast not required if Metadata.load_metadata_override re… 

    …turned a dict[str, Any] instead of a dict[str, object]

Co-authored-by: compilade <git@compilade.net>
    @mofosyne @compilade
    mofosyne and compilade committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    4c91d07 View commit details
    Browse the repository at this point in the history

  36. convert-*.py: Removing the redundant metadata is not None from all co… 

    …nditions, and indenting them.

Co-authored-by: compilade <git@compilade.net>
    @mofosyne @compilade
    mofosyne and compilade committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    6eb08ac View commit details
    Browse the repository at this point in the history

  37. convert-*.py: parameter_class_attribute --> size_label

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    f8b5931 View commit details
    Browse the repository at this point in the history

  38. convert-*.py: remove redundant gguf_writer.add_name() calls

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    64707b6 View commit details
    Browse the repository at this point in the history

  39. convert-*.py: prepare_tensors_for_writing() --> prepare_tensors() 

    > Especially since it can be used for other purposes than "for writing", like preparing the tensors to then count and sum all their sizes.

Co-authored-by: compilade <git@compilade.net>
    @mofosyne @compilade
    mofosyne and compilade committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    04c4fff View commit details
    Browse the repository at this point in the history

  40. convert-*.py: import cast from typing and other refactor

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    f2b425c View commit details
    Browse the repository at this point in the history

  41. convert-*.py: remove autogenerated uuid

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    ad217d7 View commit details
    Browse the repository at this point in the history

  42. Update convert_hf_to_gguf.py 

    Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
    @mofosyne @ngxson
    mofosyne and ngxson committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    60278e4 View commit details
    Browse the repository at this point in the history

  43. Update convert_hf_to_gguf.py 

    Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
    @mofosyne @ngxson
    mofosyne and ngxson committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    aa4e589 View commit details
    Browse the repository at this point in the history

  44. Update constants.py : spacing correction

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    2c06030 View commit details
    Browse the repository at this point in the history

  45. constants.py : Revert removal of backward compatibility KEY_GENERAL_S… 

    …OURCE_URL
    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    8156835 View commit details
    Browse the repository at this point in the history

  46. convert-*.py: remove reference to uuid generation

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    ccff6c7 View commit details
    Browse the repository at this point in the history

  47. Apply suggestions from code review 

    Co-authored-by: compilade <git@compilade.net>
    @mofosyne @compilade
    mofosyne and compilade committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    455c0e5 View commit details
    Browse the repository at this point in the history

  48. convert-*.py: dict_item --> Iterable

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    5ab1a84 View commit details
    Browse the repository at this point in the history

  49. convert-*.py: update nix package to add python frontmatter

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    5cdb03b View commit details
    Browse the repository at this point in the history

  50. convert-*.py: add logger and refactor load_model_card()

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    9954b64 View commit details
    Browse the repository at this point in the history

  51. convert-*.py: quantized_by in model card is not relevant for converte… 

    …d gguf
    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    abc351c View commit details
    Browse the repository at this point in the history

  52. convert-*.py: pathlib.Path exist() --> is_file() or is_dir()

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    144a7ec View commit details
    Browse the repository at this point in the history

  53. covert-*.py: per_model_weight_count_estimation() tensor arg type is I… 

    …terable[tuple[str, LazyTensor]]
    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    8629b7b View commit details
    Browse the repository at this point in the history

  54. covert-*.py: flake8 newline missing

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    4e37611 View commit details
    Browse the repository at this point in the history

  55. convert-*.py: more rigorous regexp for get_model_id_components()

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    f98f109 View commit details
    Browse the repository at this point in the history

  56. convert-*.py: flake8 remove blank line

    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    3b1766a View commit details
    Browse the repository at this point in the history

  57. gguf-py : use pyyaml instead of python-frontmatter 

    HF transformers already depends on pyyaml for model cards,
so it should already be in the environment
of the users of the convert scripts, unlike python-frontmatter.

This should be completely equivalent since the model cards
seem to use only YAML and never TOML.
    @compilade @mofosyne
    compilade authored and mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    78a42fb View commit details
    Browse the repository at this point in the history

  58. convert_hf : use GGUFWriter to count model parameters

    @compilade @mofosyne
    compilade authored and mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    417d7a7 View commit details
    Browse the repository at this point in the history

  59. metadata.py: account for decimal point in size label within model id … 

    …components
    @mofosyne
    mofosyne committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    9a925b5 View commit details
    Browse the repository at this point in the history

  60. Update convert_hf_to_gguf.py 

    It might help with the convert_lora_to_gguf.py script if default values were added here

Co-authored-by: compilade <git@compilade.net>
    @mofosyne @compilade
    mofosyne and compilade committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    c7b3616 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2024

  1. Merge branch 'master' into refactor-convert-py

    @mofosyne
    mofosyne committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    5da16bb View commit details
    Browse the repository at this point in the history

  2. convert-*.py: Add naming_convention_vocab_only()

    @mofosyne
    mofosyne committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    eb0bf6b View commit details
    Browse the repository at this point in the history

  3. convert_lora_to_gguf.py: remove model_name parameter. Doesn't exist i… 

    …n LoraModel()
    @mofosyne
    mofosyne committed Jul 16, 2024
    Configuration menu
    Copy the full SHA
    7e9271c View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2024

  1. gguf-py : extract metadata from model name more resiliently 

    Using more than one regex to annotate the parts of the name,
this way, the order doesn't have to be fixed
and this should work correctly for more edge cases.

Also, the total parameter count of the model is used to figure out
if a size label is not actually a size label, but a context size.

* convert_lora : fix duplicate model type key
    @compilade
    compilade committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    2c18a9a View commit details
    Browse the repository at this point in the history

  2. gguf-py : fix flake8 lint

    @compilade
    compilade committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    4c9932c View commit details
    Browse the repository at this point in the history

  3. gguf-py : handle more name metadata extraction edge cases 

    * gguf-py : output the split plan on stdout when using dry_run

* convert_hf : unify vocab naming convention with the standard one

This also adds a way to name LoRA models.
    @compilade
    compilade committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    73899f7 View commit details
    Browse the repository at this point in the history