Fix nit in LoRA doc

[awni]

Very tiny fix, closes #1053

[madroidmaq]

hi @awni , According to OpenAI's official documentation, this may not be a bug, as arguments is not an object but a "string" type "object" that needs to be deserialized into a real object.

For details, see: https://platform.openai.com/docs/guides/fine-tuning/fine-tuning-examples

[hansvdam]

Yes it is true that it should be like that according to the openai docs, but MLX does not fine-tune well if you adhere to that...
I guess there is something wrong under the hood then with the interpretation of the training data in that format...
I used it for fine-tuning meta-llama/Llama-3.1-8B-Instruct

[madroidmaq]

If you check the chat_temlpate configuration in the meta-llama/Llama-3.1-8B-Instruct repository, you may find that the reason could be the requirement for the function return format, which is inconsistent with the OpenAI format. Llama-3.1-8B-Instruct requires a dictionary to be returned.

You have access to the following functions. To call a function, please respond with JSON for a function call.
'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.'

So if you return according to the document's format, some issues may arise (the format in the base model is inconsistent with the fine-tuning data format). When I reviewed the HuggingFace chat_template document again, I found it also highlighted this part:

If you’re familiar with the OpenAI API, you should pay attention to an important difference here - the tool_call is a dict, but in the OpenAI API it’s a JSON string. Passing a string may cause errors or strange model behaviour!

When I checked the mistral-finetune project again, I found that its data usage is consistent with the OpenAI format.

So I think there might not be a strictly correct format here. The key point is to ensure that the format of your fine-tuning dataset needs to be consistent with the base model's format, otherwise problems will arise. I think this part can be explained in the documentation.

[awni]

I think the right call for this is to leave a pointer in the doc that the data format that works well for tool calling can be different for different models and linking to the page about tool use in HG chat templates that you found @madroidmaq. Thanks for digging into that. I will update the doc and leave the current example as is.

[madroidmaq]

@awni I submitted a PR #1063, could you help check if any adjustments are needed?