Fix data processing in ORPO example script

[qgallouedec]

Fixes #1886
Fixes #1582

Before:

Prompt

Hi, I want to learn to play horseshoes. Can you teach me?

Chosen

<|im_start|>user
Hi, I want to learn to play horseshoes. Can you teach me?<|im_end|>
<|im_start|>assistant
I can, but maybe I should begin by telling you that a typical game consists of 2 players and 6 or 8 horseshoes.<|im_end|>
<|im_start|>user
Okay. What else is needed to play, and what are the rules?<|im_end|>
<|im_start|>assistant
A horseshoe is usually made out of metal and is about 3 to 3.5 inches long and around 1 inch thick. The horseshoe should also have a 2 inch by 3 inch flat at the bottom where the rubber meets the metal. We also need two stakes and six horseshoes.<|im_end|>

Rejected:

<|im_start|>user
Hi, I want to learn to play horseshoes. Can you teach me?<|im_end|>
<|im_start|>assistant
I can, but maybe I should begin by telling you that a typical game consists of 2 players and 6 or 8 horseshoes.<|im_end|>
<|im_start|>user
Okay. What else is needed to play, and what are the rules?<|im_end|>
<|im_start|>assistant
Horseshoes are either metal or plastic discs. The horseshoes come in different weights, and the lighter ones are easier to throw, so they are often the standard for beginning players.<|im_end|>

After

Prompt

<|im_start|>user
Hi, I want to learn to play horseshoes. Can you teach me?<|im_end|>
<|im_start|>assistant
I can, but maybe I should begin by telling you that a typical game consists of 2 players and 6 or 8 horseshoes.<|im_end|>
<|im_start|>user
Okay. What else is needed to play, and what are the rules?<|im_end|>

Chosen

<|im_start|>assistant
A horseshoe is usually made out of metal and is about 3 to 3.5 inches long and around 1 inch thick. The horseshoe should also have a 2 inch by 3 inch flat at the bottom where the rubber meets the metal. We also need two stakes and six horseshoes.<|im_end|>

Rejected

<|im_start|>assistant
Horseshoes are either metal or plastic discs. The horseshoes come in different weights, and the lighter ones are easier to throw, so they are often the standard for beginning players.<|im_end|>

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.