[Bug] ValueError: Model is not multi-lingual but language is provided.

[vitaliy-sharandin]

Describe the bug

I encounter error while trying to use xtts_v2 multilingual model in loop to generate several audio clips. This happens only for this model, VITS works fine while looping over existing tts object.

To Reproduce

Reproduction code:

tts = TTS('tts_models/multilingual/multi-dataset/xtts_v2', gpu=True)
for i in range(5):
    tts.tts_with_vc_to_file(text=f"Как оно {i}?", speaker_wav='/content/drive/MyDrive/Data/SPEAKER_00_voice_clips.wav', language='ru', file_path=f'/content/drive/MyDrive/Data/test{i}.wav')

Error:

/usr/local/lib/python3.10/dist-packages/TTS/api.py in tts_with_vc_to_file(self, text, language, speaker_wav, file_path)
    486                 Output file path. Defaults to "output.wav".
    487         """
--> 488         wav = self.tts_with_vc(text=text, language=language, speaker_wav=speaker_wav)
    489         save_wav(wav=wav, path=file_path, sample_rate=self.voice_converter.vc_config.audio.output_sample_rate)

/usr/local/lib/python3.10/dist-packages/TTS/api.py in tts_with_vc(self, text, language, speaker_wav)
    461         with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as fp:
    462             # Lazy code... save it to a temp file to resample it while reading it for VC
--> 463             self.tts_to_file(text=text, speaker=None, language=language, file_path=fp.name, speaker_wav=speaker_wav)
    464         if self.voice_converter is None:
    465             self.load_vc_model_by_name("voice_conversion_models/multilingual/vctk/freevc24")

/usr/local/lib/python3.10/dist-packages/TTS/api.py in tts_to_file(self, text, speaker, language, speaker_wav, emotion, speed, pipe_out, file_path, **kwargs)
    389                 Additional arguments for the model.
    390         """
--> 391         self._check_arguments(speaker=speaker, language=language, speaker_wav=speaker_wav, **kwargs)
    392 
    393         if self.csapi is not None:

/usr/local/lib/python3.10/dist-packages/TTS/api.py in _check_arguments(self, speaker, language, speaker_wav, emotion, speed, **kwargs)
    240                 raise ValueError("Model is not multi-speaker but `speaker` is provided.")
    241             if not self.is_multi_lingual and language is not None:
--> 242                 raise ValueError("Model is not multi-lingual but `language` is provided.")
    243             if not emotion is None and not speed is None:
    244                 raise ValueError("Emotion and speed can only be used with Coqui Studio models.")

ValueError: Model is not multi-lingual but `language` is provided.

Then, if you remove parameter, the launch will trigger exception requiring you to return language parameter back.

/usr/local/lib/python3.10/dist-packages/TTS/api.py in tts_with_vc_to_file(self, text, language, speaker_wav, file_path)
    486                 Output file path. Defaults to "output.wav".
    487         """
--> 488         wav = self.tts_with_vc(text=text, language=language, speaker_wav=speaker_wav)
    489         save_wav(wav=wav, path=file_path, sample_rate=self.voice_converter.vc_config.audio.output_sample_rate)

/usr/local/lib/python3.10/dist-packages/TTS/api.py in tts_with_vc(self, text, language, speaker_wav)
    461         with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as fp:
    462             # Lazy code... save it to a temp file to resample it while reading it for VC
--> 463             self.tts_to_file(text=text, speaker=None, language=language, file_path=fp.name, speaker_wav=speaker_wav)
    464         if self.voice_converter is None:
    465             self.load_vc_model_by_name("voice_conversion_models/multilingual/vctk/freevc24")

/usr/local/lib/python3.10/dist-packages/TTS/api.py in tts_to_file(self, text, speaker, language, speaker_wav, emotion, speed, pipe_out, file_path, **kwargs)
    389                 Additional arguments for the model.
    390         """
--> 391         self._check_arguments(speaker=speaker, language=language, speaker_wav=speaker_wav, **kwargs)
    392 
    393         if self.csapi is not None:

/usr/local/lib/python3.10/dist-packages/TTS/api.py in _check_arguments(self, speaker, language, speaker_wav, emotion, speed, **kwargs)
    236                 raise ValueError("Model is multi-speaker but no `speaker` is provided.")
    237             if self.is_multi_lingual and language is None:
--> 238                 raise ValueError("Model is multi-lingual but no `language` is provided.")
    239             if not self.is_multi_speaker and speaker is not None and "voice_dir" not in kwargs:
    240                 raise ValueError("Model is not multi-speaker but `speaker` is provided.")

ValueError: Model is multi-lingual but no `language` is provided.

Temporary fix:
As a workaround I have put tts object within loop and for some reason it works with constant reinitialization. Although it's eating away RAM(up to 9gb and then stops), the error disappears and the generation works.

for i in range(5):
    tts = TTS('tts_models/multilingual/multi-dataset/xtts_v2', gpu=True)
    tts.tts_with_vc_to_file(text=f"Как оно {i}?", speaker_wav='/content/drive/MyDrive/Data/SPEAKER_00_voice_clips.wav', language='ru', file_path=f'/content/drive/MyDrive/Data/test{i}.wav')
    del tts; import gc; gc.collect(); torch.cuda.empty_cache()

Expected behavior

No error while generating using same xtts_v2 object

Environment

- TTS built from main
- Google Colab environment

[WeberJulian]

Indeed, looks like a bug. Looking into it.

[vitaliy-sharandin]

Did you get a chance to look into it? Anything I can do to help with resolving?

[Kaszanas]

Indeed, looks like a bug. Looking into it.

I have the same issue.

[sunnnnnnnny]

it's a brute solution is that comment _check_arguments funciton;

[erogol]

@WeberJulian any updates?

[Edresson]

It was already fixed. If anyone is currently facing this issue please upgrade your 🐸 TTS version.