adds sentiment example for a 20b model

[edbeeching]

This PR adds a sentiment example for a 20b model.
There are 3 scripts:

1. s01_cm_finetune_peft_imdb.py - Fine tuning a Low Rank Adapter on a frozen 8-bit model for text generation on the imdb dataset.
2. s02_merge_peft_adapter.py - Merging of the adapter layers into the base model’s weights and storing these on the hub.
3. s03_gpt-neo-20b_sentiment_peft.py - Sentiment fine-tuning of a Low Rank Adapter to create positive reviews.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[younesbelkada]

Thanks a lot for this! Left few nits!
Let's also add a line about it on the docs 💪 Let me know if you want me to do that!

[lvwerra]

In general happy to merge. Two comments

• left a nit about the naming
• we should add it to the list of examples in the docs

[younesbelkada]

Thanks a lot for this!

[Kororinpas]

Hi，there. I am a beginner in the field of NLP and I have been working with the GPT-J model recently. I came across your code for merging adapter layers into the base model's weights in s02_merge_peft_adapter.py , and I have some questions regarding the merging process.

From my understanding, after fine-tuning the model with the LORA layer and running this merging code, the LORA layer is replaced with a new randomly initialized linear layer. However, I did not see any indication in the code that the parameters of the LORA layer were inherited by this new linear layer.If this is the case, then it would mean that my previous training of only the LORA layer was pointless.

I would be grateful if you could provide me with some clarification on this matter. Thank you very much for your time and help.

[younesbelkada]

Hi @Kororinpas
Please have a look at this nice thread: #250 that goes over the details of merging LoRA layers!
Also now you can directly merge the lora layers using peft with the model.merge_and_unload() utility function, see huggingface/peft#227
Let us know if you have more questions after that

[Kororinpas]

@younesbelkada Thank you for sharing this information with me. I have already checked out the thread you suggested and the original code. Now, my problem is solved. Thanks again!