This is an AI narrator for the SpaceBlaster game. It uses a large language model to generate text based on the game state. The game state is sent to the AI narrator as a text prompt, and the AI narrator generates a response based on the prompt. Then the text received from the language model and sent to text-to-speech model for generating audio.
The final audio track is then sewn together from the audio clips generated by the text-to-speech model. The output of this process is a full length audio file of the event file, that was provided as input to the AI narrator.
At the moment it remains for the user to manually combine the audio track with the video track of the game.
Note that it's not a real time AI narrator, but rather a post-processing tool for generating interesting commentary for the game, as a proof of concept and interesting use case for AI tools.
The AI narrator is implemented in Java and uses the OpenAI API for the language model and the ElevenLabs API for the text-to-speech model.
The game is available at: https://github.com/JoonasVali/SpaceBlaster
- Java 21+
- Maven 3.9+
- OpenAI token (paid subscription)
- ElevenLabs API key (some free usage possible, but generally paid subscription is required)
- Save OpenAI token as
OPENAI_TOKEN
environment variable. - Save ElevenLabs api key as
ELEVENLABS_API_KEY
environment variable. - Clone the SpaceBlaster repository and
mvn clean install
it to get the spaceblasterevent
library. - You also need to
mvn install
the customization of elevenlabs-api - Clone the SpaceBlaster-ai repository
- In
Launch
class change the constantsEVENT_DATA_PATH
andSOUND_OUTPUT_DIRECTORY_ROOT
to point to the generated event data file and to a directory which serves as a workspace for the output of this project. - Run the main class
Launch
to generate the audio file. It's a long process to avoid rate limiting. Adjust or remove the sleeps in case your accounts have higher rate limits.
Follow instructions here.
Note that generally the flow should be:
- Play the game with event system activated and record the video of it.
- Enter the event data file to the AI narrator.
- Run the script
- Combine the audio track with the video track.
- Nature documentary style, Narrated by Fin, GPT-4o
- Roasting the player, Narrated by Liam, GPT-4o
- Roasting the player, Narrated by Liam, GPT-4o-mini
- Roasting the player (pre-refactor attempt), Narrated by Liam, GPT-4o
Since this project is using external services (with paid subscriptions), it's important to note that the project author does not take any responsibility for the usage of the services or for the used tokens. The user is responsible for the usage of the services and for the costs that may arise from the usage of the services. It's possible that the project contains bugs or other issues that may cause the services to be used in an unintended way or which might cause unintended costs. The project author does not take any responsibility for these issues or costs. Before launching, set reasonable limits in your accounts to avoid accidental overspending. The user activating this program will take full responsibility of any negative effects it might have in spending the credits in external services.