Skip to content

Technical details

Jump to bottom Edit New page
Fabian Fichter edited this page Mar 3, 2023 · 11 revisions

Training data format

The bin training data format used for this trainer is a modified version of the bin training data format used by the official Stockfish team. The original format uses a block of 256 bit to store a position. However, for most variants with large boards and/or many piece types 256 bit are not sufficient to store a position, or it would at least require a very specialized format for each variant, which is undesirable in this context where a generic solution/format is more suitable.

In principle with up to 26 different piece types and up to 120 squares (12x10 board) plus arbitrarily many pieces in hand, a training data format being able to store positions of any configurable variant in Fairy-Stockfish would need to be very big. However, this is not very practical as >95% of variants in practice fit into 512 bit. Therefore as a pragmatic decision the training data format was decided to use 512 bit. The current code from the training data generator that packs a position into 512 bit is at https://github.com/fairy-stockfish/variant-nnue-tools/blob/a7f2df622b1e5719307265f492786d7709dd8085/src/tools/sfen_packer.cpp#L169-L230. It is structured as follows:

As you might see from this there is no easy rule when this is below 512 bit. You need to consider the board size, number of piece types, and number of pieces on board to calculate whether this is the case. However, the format is designed in a way that e.g. shogi, despite its large board, large number of pieces and piece types, as well as pieces in hand, still fits. The "worst case" example outlined in https://github.com/fairy-stockfish/variant-nnue-tools/blob/a7f2df622b1e5719307265f492786d7709dd8085/src/tools/sfen_packer.cpp#L127-L136 would be shogi on a 12x10 board.

Variant NNUE HalfKAv2 architecture

Input features

The Half(K)Av2 architecture in Fairy-Stockfish is based on HalfKAv2 from official Stockfish with several generalizations in the feature transformer as described below.

As explained in the aforementioned link, the HalfKAv2 architecture has 64 * 11 input features per king square, i.e., one per piece type (6), color (2), and square (64), but with the two kings merged into one (therefore only 2*6-1 = 11). In Fairy-Stockfish this is generalized for arbitrary board sizes and number of piece types, so the number of features per king square is RANKS * FILES * (2 * PIECE_TYPES - 1).

On top of that there can be additional features in case a variant uses pieces in hand. If this is the case, for each color and piece type except kings there are an additional 2 * FILES number of features representing the count of pieces in hand. This choice was made since most drop variants have one pawn-like piece on each file, so even when one side has all pawns in hand (both his and his opponents), then the number equals twice the number of files. E.g., in shogi one can have at most 18 (2*9) pawns, or in crazyhouse at most 16 (2*8). So in the example of crazyhouse this adds 2 (colors) * 5 (non-king piece types) * 16 (max hand count) number of features.

Since not only the number of features for a given king square, but also the number of king squares can vary, this is also taken account of. Usually the number of king squares simply equals the number of squares on the board. However, for variants where the king piece is limited to a certain area of the board, these accessible squares are mapped to a range corresponding to their number. E.g., for Xiangqi the palace has 9 squares, so the number of king squares is 9.

For variants that do not have any king-like piece type that has a constant count of 1, the number of king squares is set to 1. Therefore, this is referred to as Half(K)Av2, since in such a case the overparametrization with king squares disappears. One additional modification is required in this scenario, because the optimization of merging the two king feature planes is no longer possible, since they would otherwise be indistinguishable due to the missing overparametrization. Therefore the RANKS * FILES * (2 * PIECE_TYPES - 1) features are extended to RANKS * FILES * (2 * PIECE_TYPES).

So overall the number of input features for a given variant is:

INPUT_FEATURES = KING_SQUARES * [RANKS * FILES * (KING_SQUARES != 1 ? 2 * PIECE_TYPES - 1 : 2 * PIECE_TYPES) + (DROPS ? (2 * FILES) * (2 * NON_KING_PIECE_TYPES) : 0)]

Also see the same calculation in https://github.com/fairy-stockfish/variant-nnue-pytorch/blob/master/halfka_v2.py.

File size

Since the feature transformer weights consume a large part of the size of an NNUE file, the file size can be roughly estimated as the number of input features times the size of the first layer, which is 512 + 8 (for the PSQT part), times the size of one weight, which is 2 Byte (16 bit integer). Therefore, multiplying the above number of input features by 520 and by 2 Byte per weight gives a lower bound of the file size which should be close to the actual size. Neglecting the features of pieces in hand and the minor adjustments due to the merging of king features, this can be simplified to SIZE >= SQUARES * KING_SQUARES * PIECE_TYPES * 2080 as the size in bytes.

Due to the effect of the overparametrization, variants without royal kings (e.g., antichess) or variants with very constrained kings (e.g., xiangqi) have particularly small NNUE networks in this architecture.

  • For variants without kings the size commonly is around 1 MB or below.
  • For Xiangqi/Janggi with only 9 king squares the size is ~11 MB.
  • For chess-like variants the size is around 50 MB.
  • For variants with large board, many piece types, and/or drops, the size can be up to around 200 MB. E.g., for shogi it is ~150 MB.
Add a custom footer
Add a custom sidebar
Clone this wiki locally