Jaxcalibur
Hi everyone! I am excited to share a new Pokémon bot named Jaxcalibur. It only plays Gen 9 Random Battles, but it plays them extremely well. Over the past three weeks, it has spent time at #1 on the ladder and reached a peak Elo of 2557 and peak GXE of 95.3. As far as I know, it is the first AI to reach the level of top human players at Pokémon battling, albeit in a narrow format. For comparison, the strongest previous bot I'm aware of, FoulPlay, peaked around 2341 Elo and 88 GXE, which is roughly top-50 rather than top-1.
The ladder on August 10. This was Jaxcalibur's peak Elo (but not GXE)
That said, I can't say Jaxcalibur is the world's best random battler because one human outperformed it over this time period! MichaelderBeste2 held #1 on the ladder about 50% of the time compared to Jaxcalibur's 20%, and reached a slightly higher GXE of 95.5. I am really impressed that anyone can play better than a bot trained over millions of games and simulating thousands of possibilities each turn; that is an incredible feat!
How it works
Jaxcalibur uses a neural net trained with self-play reinforcement learning combined with pUCT search, similar to AlphaGo Zero. It is built on poke-env and Jax (which is where the name comes from). There is much more on how it works in the technical write-up here.
Some observations on how it plays: it is fairly conservative, preferring safe lines over hard reads. It is especially good at endgames because it can calculate several turns ahead. I think its clearest weaknesses are an imperfect understanding of PP stalling and a tendency to become predictable in some situations.
Current status
As of August 28, Jaxcalibur has stopped searching for ladder games and will let its rating decay. However, it will still accept randbats challenges if you'd like to play against it. It can only play one battle at a time, so it might not accept your challenge right away. Jaxcalibur currently plays random battles only, but I think it could be extended to other singles formats and, with more work, to doubles battles. It can't build teams itself, but you could use self-play with different teams to evaluate which ones are strongest
The open-sourcing question
When I started this project, I was intending to fully open-source Jaxcalibur, but it's strong enough I'm now a bit worried about how this could affect the community.
The case for releasing it: for me, the biggest benefit is that others could learn from it and build on it. Strong bots could also make single-player Pokémon more challenging, serve as practice partners, and be powerful tools for team building or balancing formats.
The case against: cheating is the big one. Someone could use Jaxcalibur to climb the ladder, obtain suspect reqs, or potentially even win tournaments in a way that's hard to detect. This feels especially relevant given the recent discussion about bots and suspect reqs. I also wonder whether easily accessible information about which teams, leads, and openings perform best would make team building less interesting.
One possible middle ground would be to release the code but not the model weights. However, for now I am holding off releasing either. I'd love to hear what people here think: should Jaxcalibur be open-sourced? If so, what would be a responsible way to release it? Strong, widely available Pokémon bots are probably coming sooner or later. This may be a useful chance to establish some norms around how they should be released and used.
Links
Update (September 12)
Thanks everyone for all the thoughtful discussion around Jaxcalibur and the open-sourcing question. After reading the responses here and talking with members of the community, I've decided not to release Jaxcalibur's code or model weights. My main concern is preserving the integrity of the ladder and the broader Showdown/Smogon competitive scene.
I'm also taking Jaxcalibur offline from public challenges for at least the next week while I work on some fixes and improvements. I may make it available to challenge again afterward, but it will not be laddering.
Hi everyone! I am excited to share a new Pokémon bot named Jaxcalibur. It only plays Gen 9 Random Battles, but it plays them extremely well. Over the past three weeks, it has spent time at #1 on the ladder and reached a peak Elo of 2557 and peak GXE of 95.3. As far as I know, it is the first AI to reach the level of top human players at Pokémon battling, albeit in a narrow format. For comparison, the strongest previous bot I'm aware of, FoulPlay, peaked around 2341 Elo and 88 GXE, which is roughly top-50 rather than top-1.
The ladder on August 10. This was Jaxcalibur's peak Elo (but not GXE)
That said, I can't say Jaxcalibur is the world's best random battler because one human outperformed it over this time period! MichaelderBeste2 held #1 on the ladder about 50% of the time compared to Jaxcalibur's 20%, and reached a slightly higher GXE of 95.5. I am really impressed that anyone can play better than a bot trained over millions of games and simulating thousands of possibilities each turn; that is an incredible feat!
How it works
Jaxcalibur uses a neural net trained with self-play reinforcement learning combined with pUCT search, similar to AlphaGo Zero. It is built on poke-env and Jax (which is where the name comes from). There is much more on how it works in the technical write-up here.
Some observations on how it plays: it is fairly conservative, preferring safe lines over hard reads. It is especially good at endgames because it can calculate several turns ahead. I think its clearest weaknesses are an imperfect understanding of PP stalling and a tendency to become predictable in some situations.
Current status
As of August 28, Jaxcalibur has stopped searching for ladder games and will let its rating decay. However, it will still accept randbats challenges if you'd like to play against it. It can only play one battle at a time, so it might not accept your challenge right away. Jaxcalibur currently plays random battles only, but I think it could be extended to other singles formats and, with more work, to doubles battles. It can't build teams itself, but you could use self-play with different teams to evaluate which ones are strongest
The open-sourcing question
When I started this project, I was intending to fully open-source Jaxcalibur, but it's strong enough I'm now a bit worried about how this could affect the community.
The case for releasing it: for me, the biggest benefit is that others could learn from it and build on it. Strong bots could also make single-player Pokémon more challenging, serve as practice partners, and be powerful tools for team building or balancing formats.
The case against: cheating is the big one. Someone could use Jaxcalibur to climb the ladder, obtain suspect reqs, or potentially even win tournaments in a way that's hard to detect. This feels especially relevant given the recent discussion about bots and suspect reqs. I also wonder whether easily accessible information about which teams, leads, and openings perform best would make team building less interesting.
One possible middle ground would be to release the code but not the model weights. However, for now I am holding off releasing either. I'd love to hear what people here think: should Jaxcalibur be open-sourced? If so, what would be a responsible way to release it? Strong, widely available Pokémon bots are probably coming sooner or later. This may be a useful chance to establish some norms around how they should be released and used.
Links
- Showdown Profile: Challenge it to a random battle! Note it only plays one game at a time so it may not accept your challenge immediately.
- Technical Write-Up: More details on how it works.
- Self-Play Pokémon Win Rates: Which randbats Pokémon are strongest and weakest in top-level play?
- Replays: See Jaxcalibur's games alongside what it was thinking.
Update (September 12)
Thanks everyone for all the thoughtful discussion around Jaxcalibur and the open-sourcing question. After reading the responses here and talking with members of the community, I've decided not to release Jaxcalibur's code or model weights. My main concern is preserving the integrity of the ladder and the broader Showdown/Smogon competitive scene.
I'm also taking Jaxcalibur offline from public challenges for at least the next week while I work on some fixes and improvements. I may make it available to challenge again afterward, but it will not be laddering.
Last edited:
