Batch Inference

[ppmzhang2]

What are the best practices for efficiently handling batch inference using fish-speech? e.g. generating tokens for an entire book in sections

[Maharshi222]

Try Preprocess the data. Tokenize book into chapters beforehand to avoid repeated tokenization during inference.