ElevenLabs Support

[Roctose]

Sorry, posted this as an issue only to come back and notice there's a protocol for this that I missed. Sorry.

ElevenLabs is most likely the most high quality accessible TTS out there. As it has API hooks, I think it'd be an incredibly strong candidate for this plugin.

beta.elevenlabs.io/

[karashiiro]

I'll have a look, I was under the impression that ElevenLabs had an upfront cost, but it looks like their free tier has API access, so it shouldn't be hard to test this for development purposes.

[Roctose]

Oh! That's super exciting! If you'd like to use mine for testing purposes, I'd be happy to help should that prove necessary.

[Soleman1989]

Yes, please make this a possibility. The quality ElevenLabs provides does not even compare with all of the other services currently used in the plugin,

[apowel]

I made a voice clone for Y'shtola and have been fiddling with an ElevenLabs backend. The voice can be scarily good with the right settings, but there's definitely a bit of lag generating the audio in real time even with a creator tier sub. I haven't worked on dalamud plugins before so the ui stuff is taking me a bit to learn.

[Roctose]

You already got this mocked up? Wow!! I'd prolly only turn it on for cutscenes, myself. But damn if it isn't exciting! Perhaps not cost effective, no, but well worth it if you do have access.

I do agree though on your individual file point. Is there a way I could achieve that? I'd very much like to try.

[albeva]

Perhaps an option would be to record the voice lines and distribute them with the add-on? This is what the wow VoiceOver addon does (they use ElevenLabs as well - https://github.com/mrthinger/wow-voiceover) - they generate voice lines for all quests in the game, and the addon includes all sounds as mp3 files when you download it. This also opens the way to give different characters different voices and dramatically improves user experience when downloading & using the addon. It might be possible to crowdsource the funding to generate the sound files.

[Roctose]

As cool as that would be, it would be very out of scope for this project at least.

[ryankhart]

I'm joining this discussion late after being on break from FFXIV at the moment, but I share the desire to have ElevenLabs voices in FFXIV. However, the cost is a concern for me. Their website estimates it would cost almost $99 for the length of an average audiobook. Given the amount of text dialogue in FFXIV, it could potentially cost up to $1000, but I don't have any exact figures. Additionally, with the advancements in AI, there is a possibility that cheaper or even free AI-based text-to-speech tools may become available in the future.

Regardless, one idea I had, inspired by @apowel, was to have an option that allows users to automatically upload their generated ElevenLabs audio to share with other TextToTalk users. However, this could create complications such as only allowing it for NPC dialogue and not player chat. It could also be confusing for users not knowing when their ElevenLabs account will be charged unless there is some sort of UI element to indicate this.

[albeva]

One idea I had was to do a proxy server for caching and generating the voice-over lines for quests. The proxy server could be run by crowdsourcing the financing - add a donate button, and the money generated would be used for new voice lines and paying for hosting.