Other Safari - Battle Bot using Oak Neural Network Search

[opencover]

​

Safari​

Source code

Almost a year ago I introduced Oak, a reinforcement learning framework for perfect information RBY battling. The information constraint is necessary to apply the Alpha Zero methodology, including Monte-Carlo Tree Search to RBY OU. I hinted that the network and search could be applied to the full game in the same manner as Foul-Play, using Information Set MCTS.

This work has been recently completed and now Safari can be considered an iteration upon Foul Play with a focus on generation one.

The improvements include:

Accurate protocol log parsing and battle state inference​

As you know, generation 1 has many mechanical differences from later generation and many bugs which are relevant to competitive play, such as the Stat Modification glitch. Foul-Play's support for these mechanics is limited and, currently, broken. I have written tests to aide the development of the log parser and its complete enough that the parsed battle will remain in sync with the 'truth' over thousands of random seeds and games.

Bluffing and Bayesian Games​

​

After the various searches, when it is time to select a move, Foul-Play currently takes the average policy over all the searches and picks the move with the most visits. In particular, all the determinizations of the public battle state (i.e. the partial battle with unrevealed species and moves filled in) all use the bots actual team. As far as its search is concerned, the opponent always has your team-sheet. Because of this the bot is categorically incapable of bluffing, a very important strategy in imperfect information games.
Safari applies its team-guessing code to its own partially observed team and thus formalizes the battle as a full Bayesian Game

Improved Tooling​

All search information is displayed on a custom HTML dashboard and can be saved as 'replays' for later review.

A brief explanation of the dashboard is spoilerd

VISUALIZATION EXPLAINED

* Top left: determinization (team guesses) for both players with their associated probabilities. Pictured, I set the number of guesses for the player's team to 1, like Foul Play. Of course this can be increased to simulate uncertainty for the opponent.
*Bottom left: Display for the battle state. 'Sprite' is WIP but it looks like the PS client. 'Text' is a verbose description of the entire battle. "Public" is the same as the previous but with the guessed information omitted. "Request" is just the JSON from the Showdown protocol. "Log" is just the usual battle log e.g. |switch|p2a: Chansey|Chansey|703/703|move|p1a: Starmie|Thunderbolt|p2a: Chansey|-damage|p2a: Chansey|630/703
* Middle: This is the real meat of the display. It is the empirical value matrix at the root of the search. The color coded values lie in the range [0, 1], with black entries being unexplored. There is a button to change this to a visit count display
* Right: These are the final strategy outputs of the search. The top 2 correspond to the bot and the bottom to the opponent. The very top applies to the selected determinzation (pair of teams, we can see that in the top left each player has a highlighted team). The bottom applies the the full matrix game, which is why we have aforementioned 'Bayes' option. 'Prior' is the raw network policy prior at the root.

PokeAgent Challenge​

The PAC is a battle bot and cartridge speedrunning competition that was held late last year (and the Gen 9 battle portion was won by FoulPlay!)

The competition has concluded but the organizers are still running the models on the custom showdown server. This is wonderful opportunity for me and other developers to test Safari and other agents. It is actually the sole reason I started working on Safari :)

The Smogon RBY OU ladder was actually topped by one of their models(!) so it's a serious benchmark. As you probably know, the official ladder is inactive and players are generally (and reasonably) not enthused to play vs bots. If you've ever matched vs black_swan_23, you probably share this feeling. The difference in activity means that trying to test bots on PS official is a waste of time and a nuisance to players.

I am keen to work with anyone willing to run Safari on the PAC server. If you are serious, I am happy to provide my own trained network weights and instructions/assistance running the software. There are many unexplored search and move selection settings that I need help with. Below is an example:

python run.py \
--websocket-uri wss://battling.pokeagentchallenge.com/showdown/websocket \
--login-uri https://battling.pokeagentchallenge.com/action.php? \
--ps-username safaritest2 \
--ps-password OMITTED \
--bot-mode search_ladder \
--format 'gen1ou' \
--teams teams1 \
--predictor-teams teams1500 \
--run-count 5 \
--p1-types 1 \
--p2-types 4 \
--threads 16 \
--budget 3000ms \
--eval /home/user/pacnet \
--bandit pucb-1.0 \
--policy empirical \
--selection argmax \
--save-replay on_loss \
--log-level DEBUG

Thanks for reading!