Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build(deps): update huggingface-hub requirement from <0.27 to <0.28 in /requirements #2871

Merged
merged 1 commit into from
Dec 17, 2024

Conversation

dependabot[bot]
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github Dec 16, 2024

Updates the requirements on huggingface-hub to permit the latest version.

Release notes

Sourced from huggingface-hub's releases.

[v0.27.0] DDUF tooling, torch model loading helpers & multiple quality of life improvements and bug fixes

📦 Introducing DDUF tooling

DDUF (DDUF's Diffusion Unified Format) is a single-file format for diffusion models that aims to unify the different model distribution methods and weight-saving formats by packaging all model components into a single file. We will soon have a detailed documentation for that.

The huggingface_hub library now provides tooling to handle DDUF files in Python. It includes helpers to read and export DDUF files, and built-in rules to validate file integrity.

How to write a DDUF file?

>>> from huggingface_hub import export_folder_as_dduf
Export "path/to/FLUX.1-dev" folder as a DDUF file
>>> export_folder_as_dduf("FLUX.1-dev.dduf", folder_path="path/to/FLUX.1-dev")

How to read a DDUF file?

>>> import json
>>> import safetensors.torch
>>> from huggingface_hub import read_dduf_file
Read DDUF metadata (only metadata is loaded, lightweight operation)
>>> dduf_entries = read_dduf_file("FLUX.1-dev.dduf")
Returns a mapping filename <> DDUFEntry
>>> dduf_entries["model_index.json"]
DDUFEntry(filename='model_index.json', offset=66, length=587)
Load the model_index.json content
>>> json.loads(dduf_entries["model_index.json"].read_text())
{'_class_name': 'FluxPipeline', '_diffusers_version': '0.32.0.dev0', '_name_or_path': 'black-forest-labs/FLUX.1-dev', 'scheduler': ['diffusers', 'FlowMatchEulerDiscreteScheduler'], 'text_encoder': ['transformers', 'CLIPTextModel'], 'text_encoder_2': ['transformers', 'T5EncoderModel'], 'tokenizer': ['transformers', 'CLIPTokenizer'], 'tokenizer_2': ['transformers', 'T5TokenizerFast'], 'transformer': ['diffusers', 'FluxTransformer2DModel'], 'vae': ['diffusers', 'AutoencoderKL']}
Load VAE weights using safetensors
>>> with dduf_entries["vae/diffusion_pytorch_model.safetensors"].as_mmap() as mm:
...     state_dict = safetensors.torch.load(mm)

⚠️ Note that this is a very early version of the parser. The API and implementation can evolve in the near future. 👉 More details about the API in the documentation here.

DDUF parser v0.1 by @​Wauplin in #2692

💾 Serialization

Following the introduction of the torch serialization module in 0.22.* and the support of saving torch state dict to disk in 0.24.*, we now provide helpers to load torch state dicts from disk. By centralizing these functionalities in huggingface_hub, we ensure a consistent implementation across the HF ecosystem while allowing external libraries to benefit from standardized weight handling.

>>> from huggingface_hub import load_torch_model, load_state_dict_from_file
load state dict from a single file
>>> state_dict = load_state_dict_from_file("path/to/weights.safetensors")
</tr></table>

... (truncated)

Commits
  • 6bfa5dd Release: v0.27.0
  • ea8cb06 Release: v0.27.0.rc1
  • 875b3a8 Merge branch 'main' into v0.27-release
  • c4e6079 [InferenceClient] Keep generate_kwargs instead of generation_parameters (...
  • 9100950 Release: v0.27.0.rc0
  • 6be2b3e 📝Translating docs to Simplified Chinese (#2705)
  • ca3f674 remove context manager when loading shards and handle mlx weights (#2709)
  • 4b0b179 DDUF parser v0.1 (#2692)
  • b75f8d9 [Serialization] support loading torch state dict from disk (#2687)
  • 51b866f Fix test_list_spaces_linked (#2707)
  • Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

📚 Documentation preview 📚: https://torchmetrics--2871.org.readthedocs.build/en/2871/

@dependabot dependabot bot requested a review from SkafteNicki as a code owner December 16, 2024 23:49
@dependabot dependabot bot added the test / CI testing or CI label Dec 16, 2024
@dependabot dependabot bot requested a review from a team December 16, 2024 23:49
Copy link

codecov bot commented Dec 17, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99%. Comparing base (cd24d2b) to head (f654d20).

Additional details and impacted files
@@            Coverage Diff            @@
##           master   #2871      +/-   ##
=========================================
+ Coverage      69%     99%     +30%     
=========================================
  Files         346      14     -332     
  Lines       19128     176   -18952     
=========================================
- Hits        13226     175   -13051     
+ Misses       5902       1    -5901     

@mergify mergify bot added the ready label Dec 17, 2024
Updates the requirements on [huggingface-hub](https://github.com/huggingface/huggingface_hub) to permit the latest version.
- [Release notes](https://github.com/huggingface/huggingface_hub/releases)
- [Commits](huggingface/huggingface_hub@v0.0.1...v0.27.0)

---
updated-dependencies:
- dependency-name: huggingface-hub
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot dependabot bot force-pushed the dependabot-pip-requirements-huggingface-hub-lt-0.28 branch from f654d20 to 8cd2957 Compare December 17, 2024 10:22
@Borda Borda merged commit 57ef645 into master Dec 17, 2024
56 of 63 checks passed
@Borda Borda deleted the dependabot-pip-requirements-huggingface-hub-lt-0.28 branch December 17, 2024 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready test / CI testing or CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant