Supervised Learning

What You Will be Able to do

After reading this article, you should be able to take a collection of PGN's and train a net using lczero-training.

Getting Ready

[Note: this article assumes you are using Linux. Performing the same on Windows is possible, but as I don't use Windows, documenting the details will have to be left to someone else.]

You will need the following software:

1. pgn-extract - the supervised learning pgn parser is very brittle. As a starting point, I would run your pgn file through pgn-extract with pgn-extract -7 -C < input.pgn > output.pgn. See here for details.
2. The "supervise" branch of my fork of lczero. Yes lczero is the old engine for the nets, but it also has the supervised training code in it, which I fixed and reenabled. This should also be merged into the master branch of the original repo. I don't control that, so can only be confident that my fork has the right code.
3. The master branch of lczero-training. There's a fair bit of fiddling with setup here. You'll need CUDA-9.0 for tensorflow, et al, which is different than the CUDA-9.2, et al you got for lc0. I'll eventually add a section on configuration of this beast.

The Process

The high level supervised learning process runs as follows:

1. Make sure the individual pgn files you will be converting to training data have less than 500k games in them. The training software expects files -- called "chunks" -- with one game per chunk. So training data directories will be created with a potentially large number of files, which can become unwieldy. You can use pgn-extract to break a pgn file into equal sized files with N games. See documentation.
2. Clean up the pgn files with pgn-extract -7 -C < input.pgn > output.pgn. Change the filenames to reflect your naming scheme.
3. Run lczero to generate the training data. Note that lczero requires a weights file for this step. The weights file is loaded but ignored. This is an artifact of the all in one nature of lczero.

./lczero -w weights_useless.txt.gz --supervise my_pgn_file.pgn

4. Clean up the mess and edit your pgn file when it dumps core because of some minor pgn issue.
5. Finally you get a clean run. You should have a directory called supervise-my_pgn_file with files of the form training.XXXXX.gz where the X's are digits (there could be 1 or a dozen digits, depending on home many games you had). There should be one file for each game in your pgn.
6. If you've converted several pgn's, put all the various "supervise" directory in a common subdirectory. This will make it easier to process them in the training step.
7. Edit a config file.