Inference Problem

[HenryHe0123]

Hi, I'd like to express my sincere gratitude to the project. Your efforts are greatly appreciated.

However, after completing the model training, I encountered a problem: I am unable to load it using either the transformers or vllm for inference.

Can you provide a sample inference code for reference? Thank you in advance for your time and support.

[2U1]

I'm not saving the preprocessor config and other things, so I think you should load it download from the huggingface hub and put it into the folder.
Or you could just load the preprocessor from the hub.

[HenryHe0123]

I see. I tried adding the preprocessor config and other things to the fine-tuned model folder, but still encountered issues. However, it's now resolved. Thank you for your response.

[2U1]

@HenryHe0123 May I ask you what was the problem? And How did you solve it?