Support exporting text-generation models for sequence classification to ONNX

[dwyatte]

Feature request

Exporting text-generation models (e.g., LlamaForSequenceClassification) was disabled in #1308

Given https://arxiv.org/abs/2310.01208, these models can outperform typical encoder models for sequence classification (I can confirm this on my own datasets)

What would it take to support this in transformers? CC @fxmarty since it was mentioned specifically in the PR above

Motivation

I would like to export fine-tuned sequence classification models that use a decoder-only model as their base architecture to ONNX

Your contribution

Happy to submit a PR to transformers with guidance on what would be needed

[dwyatte]

Ah, I see what's happening -- ONNX doesn't support int64 inputs to argmax which is how these models are computing the sequence lengths for pooling. Will open a PR over in transformers and leave this open until we can enable export in this repo

[fxmarty]

Thank you for the fix!

[dwyatte]

Requires one more fix in transformers to get output validation passing with the current dummy inputs: huggingface/transformers#28144