Ruffles said:Am I the only one that is not that worried about AI in competitive Pokemon? (...)
Now, I am not sure how AI is topping ladder in the first place, but my understanding is that its looking at pure data at what most players do in this scenario or lead in this MU. You can even take advantage of this if you know you are facing a bot.
no there are probably other people that don't understand it too, and as a result are more worried about bots making usage stats or suspect tests worse, or playing lots of bots on the ladder being an annoyance instead of being worried about the death of the game.
The bots do not work the way you suggest they do, where would they even find this data? To give a very simplified explanation they work first by training, during which it plays against itself millions of times and learns what leads to winning games more often. In this step it figures out that having your hazards out and the opponent not having theirs is good and that knocking out the opponent's pokemon is good, basic things like that. This gets it to the level of an average player.
Then once it's playing for real in any given turn it'll analyze its chance of winning right now, then its chance of winning if they click move a, b, c or d, with or without tera (and if it's clicking something like Blizzard, it's chance of winning in case it misses, or freezes, or crits, or just does regular damage) or switch to mons 1, 2, 3, 4, 5, and how each of these choices interact with the expected potential choices the opponent might pick. Then it'll play out the game against itself for a few turns for each possible choice, with each turn branching out further and then finally pick one of the options that increase its chance of winning the most, keeping in mind that it has to use a mixed strategy to achieve equilibrium play*.
Sidebar: Nash
Consider you have a Great Tusk that's faster than the opponent's Gholdengo, and they have a Landorus-T in the back. If you always Headlong Rush, they'd always go Landorus, so you should always Ice Spinner, but then they would always stay in, so you should pick Headlong Rush sometimes and Ice Spinner sometimes, and likewise for them between staying in and switching. This is an incredibly simple example to demonstrate equilibrium play, the real game is complicated by a trillion factors (hazards, pp, damage taken, tera used or not, teammates' hp etc) that the bot is better equipped to analyze than we are. Also, individual player psychology factors, which the bots don't have a way to interpret yet. It's not hard to imagine a bot in the future that you can feed every single game ABR ever played into and then have it be able to very accurately predict what ABR will do in a given situation, though that's not the problem we're facing rn.
The way this works for the bot is in a given position it'll decide that move a is the best move, move b is the second best move and switch to pokemon 3 is the third best move, then it'll assign a weight to all these options, based on how much they help improve its chance of winning. So consider a complicated turn where it thinks it should pick move a 50% of the time, move b 30% of the time and switch to mon 3 20% of the time. It'll then randomly pick one of these based on those odds. If it thinks move a is waaaaay stronger than anything else (like last mon Landorus-T vs last mon Heatran) then it'll assign a very high value to that and pick that every time.
You (and I, and everyone that played the game at a high level) do this too, except we just think "if I go ferro and he goes heatran that's bad, maybe i should go gliscor then i can set up rocks as he switches fearing eq, but if he goes weavile i'm better off going ferro hm..." while the bot which thinks a trillion times faster than us can go out and simulate each possible world and make the decision from there. Will it win 100% of the time like it does in chess? No, you can crit it and you can't crit Stockfish's queen, sometimes you'll have a great matchup, sometimes you'll outplay regardless, just like how sometimes a weaker player outplays ABR etc. But it will win a lot of the time, and anyone with access to it will win a lot against people that should be better than them.
The shitty triple weather bot that people used to face in the OU ladder is fundamentally different from this. You can't exploit the bot people are talking about now by making the same moves every turn like you could that one. The best VGC player in the world is probably already a bot, and the same being true for OU is only a matter of time (or more accurately, a matter of someone interested in building that coming along). Once that happens we can't hold meaningful online tournaments anymore.
So to address the point of the thread, I already said it on discord but I think that:
1) No price is too great if it means preventing people from using bots to cheat in tournaments
2) Making the timer shorter does nothing to prevent bots from being used in tournaments, and as such is not worth doing
I'm willing to accept very draconian measures that would have been considered absurd last year if it means halting AI, such as making playing tournament playoff games on a PS client that has some kind of anti-cheat built-in, having an evaluator that determines if people are using AI and tourbanning anyone caught by it with no possibility of appeal, webcams etc, because the consequence to having AI that is better than humans is literally the end of the game. But the measure suggested in this thread would only work if the bottleneck to the bot seeing the game-state is manual input from the user. The only publicly available tool has this constraint, but the closed models other people are using don't, and there's no reason to think that the first person to open their bot (or to make a strong bot and use it to cheat themslves) will have this constraint. If we can somehow stop the bot from reading PS! directly (which I think is impossible?) We can revisit this, there's no point in doing it until then.