FAQ: How to Improve Training Speed

[polm]

When training in spaCy takes too long, there are a few things you can do to speed it up.

Change the evaluation frequency. It's not in the config the widget generates, but there's an eval_frequency option - it should be filled in if you use fill-config as recommended. The default value is relatively low, and evaluation is slow. You should increase this value a lot if you have a large amount of training data.

Reduce the size of your dev data. This is related to the evaluation frequency - while a larger dev set can help more fairly evaluate your model, it also makes evaluation slower. When reducing the size of dev data, be sure that the data you keep is still representative of your training set.

Use init labels. By default, spaCy infers the labels your model needs from training data, but it's also possible to provide the labels directly. If you provide labels directly then spaCy doesn't have to do an initial scan of your full training data every time you run spacy train, so it saves time on each training startup.

Use the efficiency presets instead of accuracy. The exact changes depend on what components you use, but one main difference between these settings is the width and depth of the tok2vec. Note this alone won't affect speed that much, but because it definitely reduces memory usage it can be usefully combined with the next option.

Increase batch size. In training the time to process a batch is relatively constant, so larger batches means fewer batches for the same data, which means faster training. How large a batch you can handle depends on the size of your documents and your hardware.

Use CPU models instead of Transformers. It's tempting to start with Transformers because they usually have the highest accuracy, but early in development it's often better to use CPU based models for faster turnaround time while you validate your problem. You might even find you get good enough performance without using Transformers at all!

Use less training data. This is very rarely something that we'd recommend, but if you have a large number of records you may not need them all to get a good model. Try 10,000 to start with and see how your model performs, and scale up until you get the accuracy/speed tradeoff you want. This will also help you validate your training data more quickly.

For tips on faster inference (not training), see the spaCy speed FAQ.