Skip to content

Commit

Permalink
FIX Correctly call element_size (huggingface#1635)
Browse files Browse the repository at this point in the history
Should fix the error introduced by huggingface#1630.

AFAICT, element_size should be called on the parameter, not the dtype.
Unfortunately, I had issues getting older PyTorch versions to work with
bnb, so I haven't tested the initial issue.

To be safe, I also re-added the previous code path using itemsize,
although it might be unnecessary (we would have to check the PyTorch
code to verify when the different attributes/methods were added).
  • Loading branch information
BenjaminBossan authored and DTennant committed Apr 19, 2024
1 parent 198a5f9 commit c2674ef
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion src/peft/peft_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -542,7 +542,12 @@ def get_nb_trainable_parameters(self) -> tuple[int, int]:
# one needs to multiply the number of parameters by 2 to get
# the correct number of parameters
if param.__class__.__name__ == "Params4bit":
num_bytes = param.quant_storage.element_size() if hasattr(param, "quant_storage") else 1
if hasattr(param, "element_size"):
num_bytes = param.element_size()
elif not hasattr(param, "quant_storage"):
num_bytes = 1
else:
num_bytes = param.quant_storage.itemsize
num_params = num_params * 2 * num_bytes

all_param += num_params
Expand Down

0 comments on commit c2674ef

Please sign in to comment.