-
Notifications
You must be signed in to change notification settings - Fork 99
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug]: Dear teachers, when i try to use spu on torch, i found a problem #771
Comments
Your code and errors seem to be that you are using the |
sorry, i use the following code to run gpt2 on spu, but it output "I0000 00:00:1721093883.848833 2880 cpu_client.cc:405] TfrtCpuClient created" and stuck for a long time, so i hope to get some advice on how to transfer the params, thank you very much!! import torch |
replace |
thanks very much, but i have a question that the pre_model can load params through " |
a possible reason is |
Currently, SPU for PyTorch is experimental and in the early stage, using some dirty hacks to support torch.nn.Module inference. As a result, for a PyTorch model (no matter from Hugginface or somewhere else), running it on SPU doesn't fully align with its plaintext version. |
|
Issue Type
Others
Modules Involved
Others
Have you reproduced the bug with SPU HEAD?
Yes
Have you searched existing issues?
Yes
SPU Version
spu0.9.1b0
OS Platform and Distribution
ubuntu18.04
Python Version
3.10.14
Compiler Version
GCC11.2.1
Current Behavior?
Dear teachers, when i try to use gpt2 on spu, i found a problem that i cannot transfer the params successfully, could you please give me some advice on how to transfer it, thanks very much!!
Traceback (most recent call last):
File "/a-tmp/g/a.py", line 32, in
File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/transformers/models/gpt2/modeling_gpt2.py", line 1302, in forward
transformer_outputs = self.transformer(
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/transformers/models/gpt2/modeling_gpt2.py", line 1002, in forward
input_shape = input_ids.size()
^^^^^^^^^^^^^^
AttributeError: 'collections.OrderedDict' object has no attribute 'size'
import torch
from transformers import AutoTokenizer, GPT2LMHeadModel, GPT2Config
import json
import urllib
from collections import OrderedDict
from jax.tree_util import tree_map
configuration = GPT2Config()
model =GPT2LMHeadModel(configuration)
pre_model = GPT2LMHeadModel.from_pretrained("gpt2")
tokenizer=AutoTokenizer.from_pretrained("gpt2")
inputs = tokenizer.encode("Hello, my dog is cute", return_tensors="pt")
params=pre_model.state_dict()
res = model(params, inputs)
print(res)
Standalone code to reproduce the issue
Relevant log output
No response
The text was updated successfully, but these errors were encountered: