getting error while running the sft_llama2.py

[PRASANTH-1427]

when I'm running this part I get an error in the 58th command line:
[ 57th line in sft_llama2.py ]
#57 parser = HfArgumentParser(ScriptArguments)
#58 script_args = parser.parse_args_into_dataclasses()[0]
#59
#60 if script_args.group_by_length and script_args.packing:
#61 raise ValueError("Cannot use both packing and group by length")

error:
ValueError Traceback (most recent call last)
in <cell line: 3>()
1 parser = HfArgumentParser(ScriptArguments)
2 print('abc ',parser.parse_args_into_dataclasses)
----> 3 script_args = parser.parse_args_into_dataclasses()[0]
4
5 if script_args.group_by_length and script_args.packing:

/usr/local/lib/python3.10/dist-packages/transformers/hf_argparser.py in parse_args_into_dataclasses(self, args, return_remaining_strings, look_for_args_file, args_filename, args_file_flag)
345 else:
346 if remaining_args:
--> 347 raise ValueError(f"Some specified arguments are not used by the HfArgumentParser: {remaining_args}")
348
349 return (*outputs,)

ValueError: Some specified arguments are not used by the HfArgumentParser: ['-f', '/root/.local/share/jupyter/runtime/kernel-b7bcb4cc-5414-43c1-a131-7f5468dae325.json']

HOW TO SOLVE THIS?

[lvwerra]

Are you running the script inside a Jupyter notebook?

[Badrul-QAI]

@lvwerra I'm getting the same error and I am using Google Colabs. Is it not meant to be run in a notebook?

[lvwerra]

You need to modify the code since the HfArgumentParser expects the arguments from the command line. You can for example just have the script_args as a dictionary.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.