Skip to content

New Neural Net Architecture!
Compare
Choose a tag to compare
@lightvector lightvector released this 08 Jan 04:43
· 585 commits to master since this release
v1.12.0
d93629e

This release is not the latest release, see newer release v1.12.4!

For TensorRT users only, this release contains a bug, the new net may compute incorrect values. Additionally, the OpenCL version may tune poorly and experience some errors or performance issues, upgrading to the newest release is recommended.

If you're a new user, don't forget to check out this section for getting started and basic usage! If you don't know which version to choose (OpenCL, CUDA, TensorRT, Eigen, Eigen AVX2), read this: https://github.com/lightvector/KataGo#opencl-vs-cuda-vs-tensorrt-vs-eigen

Also, KataGo is continuing to improve at https://katagotraining.org/ and if you'd like to donate your spare GPU cycles and support it, it could use your help there!

As before, attached here are "bs29" versions of KataGo. These are just for fun, and don't support distributed training but DO support board sizes up to 29x29. They may also be slower and will use much more memory, even when only playing on 19x19, so you should use them only when you really want to try large boards.

The Linux executables were compiled on an old 18.04 Ubuntu machine. As with older releases, they might not work, and it may be more reliable to build KataGo from source yourself, which fortunately is usually not so hard on Linux (https://github.com/lightvector/KataGo/blob/master/Compiling.md).

New Neural Net Architecture

This version of KataGo adds support for a new and improved neural net architecture!

The new neural nets use a new nested residual bottleneck structure, along with other major improvements in training. They train faster than KataGo's old nets and learn more effectively. Attached to this release is a one-off net b18c384nbt-uec.bin.gz that was trained for a tournament in 2022, which should be of similar strength to the 60-block nets on http://katagotraining.org/, but on many machines will run much faster, on some machines between 40-block and 60-block speed, but on some machines even as fast as or faster than 40-block.

The training code has been all rewritten to use pytorch instead of tensorflow. The training scripts and self-play scripts should be updated to account for the new implementation, but feel free to open an issue if something was overlooked.

Many thanks to "lionfenfen" and YiXiaoTian, and "Medwin", for contributing ideas and discussions and testing for improving the training, and "inbae" for the initial work and catalyst of the new pytorch implementation. Many thanks also to those on the discord server who helped with testing.

Once enough contributors have switched to this release, the new architecture will also be integrated into KataGo's main public run, where hopefully it can drive future improvement. If you are a contributor to http://katagotraining.org/, please upgrade if you can. Thanks again to everyone!

Other Notable Changes

Analysis Engine (doc)

  • Added "terminate_all" command. #727
  • Analysis engine now echos errors and warnings for bad queries to stderr by default, which can be optionally disabled. #728
  • A few additional values are now reported, including "weight", and two parameters rawStWrError rawStScoreError that measure the neural net's estimation of its own uncertainty about a position.
  • Fixed minor weighting oddity in calculation of pruned root values.

Self-play training and Contribution

  • Self-play data generation and/or contribution to the public run are changed to calculate and record slightly less-noisy values now for auxiliary value training.
  • Slightly better automated error checking was added for contributing to the public run.
  • Added some parameters to support more flexible komi initialization in selfplay. These will likely also be used for KataGo's public run soon after enough people upgrade.
  • Fixed bug in turn number computation for SGFs that specify board positions when generating sgfposes/hintposes for self-play training.

Algorithm and performance improvements

  • Improved LCB implementation to be smoother and also handle lower visits, giving a mild strength improvement for low-visits.
  • Fixed possible bug with extreme komi inputs to the neural net.
  • Improved OpenCL performance tuning logic.

Other

  • Cleaned up and clarified gtp_example.cfg. #714
  • Fixed several more bugs in the handling of non-ascii file paths.
  • Cleanup and much greater flexibility added to KataGo's config system and logging #630
  • Fixed bug where KataGo would not handle some SGF placements correctly if those placements edited and replaced many different stones on the board in the middle of a game, where all edits together would be legal but subsets of those edits might be illegal.
Assets 23
Loading