Do not upload NeuronModelForCausalLM weights when they can be reconstructed from the hub #413
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
When serializing a model, the checkpoint files (i.e. the original weights formatted for neuron) and the compiled artifacts are stored under two folder in the model directory. This modifies the NeuronForCausalLM.push_to_hub method to exclude the folder containing the checkpoint files if they can be instead reconstructed from a model on the hub. Note that it is still possible to upload checkpoint files by using the huggingface_cli directly on the saved model folder instead of push_to_hub. To allow optimum-neuron to fetch and reconstruct the checkpoint when fetchin a neuron model from the hub, the original repository that was used to export the model and its revision are stored in the neuron config.
dacorvo
force-pushed
the
do_not_upload_model_weights
branch
from
298c840 to
87c7341
Compare
dacorvo
force-pushed
the
do_not_upload_model_weights
branch
from
87c7341 to
f07d0b1
Compare
JingyaHuang
approved these changes
Jan 18, 2024
| "model_id, revision", | ||
| [ | ||
| ["dacorvo/tiny-random-gpt2-neuronx", "1b3456cf877cc42c053ee8464f1067021eccde4b"], | ||
| ["dacorvo/tiny-random-gpt2-neuronx-no-checkpoint", "78eb2313ab7e149bbc22ff32257db93ba09e3033"], |
There was a problem hiding this comment.
For the test with no checkpoint, shall we check there is no checkpoint after push to hub?
There was a problem hiding this comment.
It is a download here: the check is done in the other test.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When serializing a model, the checkpoint files (i.e. the original
weights formatted for neuron) and the compiled artifacts are stored
under two folders in the model directory.
This modifies the
NeuronForCausalLM.push_to_hubmethod to excludethe folder containing the checkpoint files if they can be instead
reconstructed from a model on the hub.
Note that it is still possible to upload checkpoint files by using the
huggingface_clidirectly on the saved model folder instead ofpush_to_hub.
To allow
optimum-neuronto fetch and reconstruct the checkpoint whenfetchin a neuron model from the hub, the original repository that was
used to export the model and its revision are stored in the neuron config.
Example model here:
https://huggingface.co/dacorvo/llama-2-7b-chat-hf-neuronx-bs1-seq2048-no-checkpoint/tree/main