KataGo has multiple components, the two main ones being MCTS, and the policy network. The policy network looks at the board, and gives probabilities for which moves are likely to win. (If that network had “solved” the game, all winning moves would have equal probability, and all others would be 0%.)
But the game is not solved, which is where MCTS comes in. It picks a move, and plays through it, selecting moves that are good but varied, until it learns more about which side would likely win (typically by expanding a leaf of the search tree). Each playthough is a visit. Each visit allows updating its estimate of the win probability for that move, and thus, changes the policy probabilities.
More visits typically anchor the policy estimates into reality further, yielding better play. In fact, this is how the policy network learns.
Thanks! So, is the 100k visits "a lot" for KataGo, or not so much? I understand that without the special anti AI tactics described in the article this would result in "superhuman" play, but what is the reasonable practical limit for the number of such visits?
100K visits is a reasonably large amount. ELF OpenGo beat professional players 20:0 with 80K visits per move[0]. The AlphaGo version that beat Ke Jie likely did 200K visits per move, since the paper indicated that their 1600 simulations per move during training corresponds to 0.4s of think time (with 4 TPUs).
On a good consumer GPU (RTX 3060), I get 1.6K visits per second, so 100K takes about a minute. That makes a game last three to five hours typically.
But the paper emphasizes that even with 10M visits per move (>1h per move), KataGo loses 72% of games against this exploit[1].
