Curious about what we're currently doing about bots? Check out the latest updates from staff in this thread.
Welcome to Smogon! Take a moment to read the Introduction to Smogon for a run-down on everything Smogon, and make sure you take some time to read the global rules.
Caveat: this is not the only anti bot measure being worked on nor has it been discussed outside my own head, just an idea I had.
How do we feel about reducing the timer for our battles? One consistent thing among both regular ghosting and (hopefully currently theoretical) ghosting using AI tools / bots is that it takes time to collude with them. With regular ghosting you're talking, debating, going back and forth. With an AI ghosting tool you need to put data in, wait for the output, consider the output vs your own logical thought process then making a decision.
Therefore, if we reduced the timer, it becomes harder for these things to be done. Of course this doesn't solve every flavour of these cheating actions, but it does make it harder to do.
If anyone knows how our current timer rules were decided that would be interesting as well
Posted on discord but I’ll copy my very unorganized thoughts here
Console timer is 45 seconds per turn or something like that with a 7 minute chess timer. I think it would be a good idea to standardize that moving forward, or at least standardize the timer somewhat. This would require a rework of the timeout system imo since currently PS will just allow someone to run down their entire chess timer. In game it times you out if you auto select (letting the timer run down so the game picks the first valid move) 3 turns in a row.
Idk how feasible that is but I think if that system was implemented and maybe adjusted to have a slightly longer chess timer for 6v6 formats so people aren’t scrambling to play the endgame it would be nice.
I think this is a very short sighted solution that wouldn't even necessarily help with actually stopping AI usage.
If someone wants to use AI to analyze their gamestate and determine the optimal turns, A. in longer games (eg more ai inputs required) it is easier to regen timer by making midgrounds that are harder to punish, thus giving you more timer to input commands to the ai (which, from my shitass 0 cs courses understanding of ai) would basically be a battle log reader//screenshot upload hotkey, and B. in shorter games (ho v ho) fewer turns are required in total, and so ai usage would be limited to basically one pivotal turn (assuming they are going to gpt astra saying help me win this game here is my team xyz). this also does not account for the fact that it is likely that bot makers would have local models obviously available that could simulate their optimal click almost instantly, making decreasing timer obsolete. I have my own takes for what to do regarding bots/ai but to me it feels like this type of action is not going to help anyone and only serve to make things marginally more irritating for actual players/timer addicts (me)
This “solution” has 0 actual benefit in the near term (right now), and exceptional harm to people who take time to think about what they want to do. It’s designed for a long term issue that we don’t fully understand yet. If there was ever proposal that looked highly knee jerk with no real immediate benefit, this would be the one. It can maybe be revisited in the future but there’s zero reason to do it right now.
quick post to say that i dislike this and think it harms players both in the current & maybe future.
1) on making timer shorter each turn/overall, you're pushing advantage in the direction of using tools like showdex vs just playing regularly with the ps calc in another tab. if time crunch becomes a decision between reasonably thinking over ur turn and calcing things out, having the calcs preset be available is a big advantage over someone just playing in the "traditional" ps+manually checking calcs. i think this is a bad thing.
2) forgive my ignorance of understanding ai's potency, but it isn't hard for me to imagine a future where what I described in 1 is available in a more advanced capacity with the same quickness to a broader extent of advantage. this would completely eliminate the effectiveness of the change to begin with.
3) punishing current active players in a trade-off for integrity with a guaranteed result of improvement is bad enough but nonetheless a warranted discussion to have. punishing current active players in a trade-off for integrity in a new & emerging problem for all of us that has no guarantee of being effective in the present/future feels fruitless and not worth it to me.
4) (very) long-standing status quo that effects literally everyone who plays showdown from mcmeghan to 1100 elo joe being uprooted is a hard sell in its own right. this is independent of point number 3.
last time I checked AI works faster than humans. If the issue is timer, making AI generate its output faster should be a small issue. You're only making the problem worse and also decreasing quality of overall human performance in tournaments by giving them less time to think.
This seems like it is unnecessarily detrimental to the people who use the fuller extent of their timer, while not actually provably reducing the effectiveness of ghosting or undisclosed bot usage. Even just in theory, there is nothing preventing ghosters from simply adapting to a reduced timer, or for bots to simply get better and reduce the amount of time they spend searching for optimal lines. At the end of the day this only really hurts the humans that play the game in good faith.
Reducing our timer is only relevant if bot ghosting tools need to be manually fed with information which doesn't seem likely. Someone with more technical knowledge can correct me but I don't imagine we can prevent a tool/extension relaying the battle information to a bot and automatically presenting its output to streamline the process and invalidating the point of this change.
Thinking about it there's some assumptions that go along with my thinking here that help elaborate why I think this helps:
1. Tools which instantly read game state and require no input from the user would be detectable + preventable. Something like showdex as a browser extension could be blocked. Other tools which read from ps would leave some trace and be detectable. If that's not the case then yes of course this wouldn't work (but also if that's not the case then we are cooked more generally). The use case is preventing tools which are external websites and require data input.
2. Making any AI tool work faster is going to cost more development and money. Maybe there's a limit where it stops becoming worth it but we are still happy with the amount of timer set
adjusting the timer completely goes against what has made the game so skill based for years.
One of the KEY skills of any good pokemon player is to use the timer. This doesnt sound like much on paper but In practice the timer usage has always worked for people to play at their best and genuinely think and improve at the game. BSS has lower timer sure but planning lines in 3v3 is far less time consuming because games are far shorter and endgames happen quicker, and likely that youve already planned the endgame usually. I say this even not being a bss player because it would NOT work in 6v6, where games are more methodical / longer and take far more in game planning over the course of a game, because an endgame doesnt come that quickly.
Lowering fhe timer would only hurt people’s chances to improve at the game. If we lower timer, the new generation of players is incentivized to rush through lines at the cost of… other people ghosting, which should already be punished by the staff of the site? Sarcasm is used here because i think it is personally just going to harm others and new talent and that is going to spring up.
Of course i completely get the concern for ai and why people want to limit it. Lowering timer is a reasonable solution to limit all types of ghosting in theory, but this should be punished by staff themselves instead of limiting the most important tool in pokemon that makes or breaks a good player. Im not even kidding, if u look at any official game 99% of the time outside of the unusual timeouts you will see incredible usage of timer from top players. As someone who is improving at the game whenever i lose i dont like to focus on what the factors that werent in my control were but the factors that WERE. Timer usage is a major factor in if someone plays the game well and something i have always criticized of my play. The people who have taught me the game as close friends have always taught me “ALWAYS USE TIMER” whenever i have asked for advice and it is something i will die on a hill for.
I address the counterargument that, while lowering timer DOES limit the amount of ghosting / ai / bot usage from a player, this procedure is listed in tournament guidelines as ghosting already. In addition, using that much timer already leads to panics, misplays, even timeouts. Take it from someone who has lost many games because of overuse of timer on preview.
Tl:dr - any adjustment of the timer destroys the integrity of the game and what has built its high level scene for years for an issue that can be solved with usual moderating and is not something that will bring good effects to the community and rising talent in the future.
And to all players who want to improve; USE. THE. TIMER. It will majorly help you think critically and give you that super important skill to think through lines. I dont want to give that up, even if its just by a second.
I don't dispute that timer usage is healthy and it's good to use it as much as possible. The point of the post is to question whether the current timer limits we have set are effective - there's a possibility that the current limit is too high and there is no meaningful difference in reducing it by e.g. 10s. Taken to the logical extreme, external influence is impossible in Blitz mode, so where is the right balance? If we cannot conceive of reducing the timer by even a single second, then that indicates to me that it's actually not long enough and we should be advocating for increasing the timer?
This is also why I'm interested if anyone knows how the current limits were set, because if they're arbitrary then it's valid to investigate changing them one way or another
edit: ghosting is of course punished every time it comes up, but cannot be detected in all its forms. so making it harder in some way is a valid alternative when detection is not possible
Not to beat a dead horse since I think the general consensus in this thread is very heavily against this but for the sake of future debate this is just flatly not true. Pokemon is a game that is astoundingly easier to instantly analyze since it is technically playable only through text that is given to the user instantly. An ai tool doesn’t need to be on its own account, it just literally can ctrl c ctrl v the battle log over and over to analyze game state. I think if we are to do anything about human assisted AI usage, a transition to a downloaded client with some type of anticheat (we’ve been working on this new client anyway eh?) for tournaments may just be the only realistic path forward. There is also the reality of if we want to counter ai usage, comparing turns to the outputs of common models à la gptzero (with all the faults that can have) is probably the best way forward. If someone is making plays that 99.9% line up with ai’s common plays across all their games across an entire tour run, this seems pretty reasonable as something for investigation. For team tours, I think the usage of a downloaded client is not out of the question, as these are players with much more dedication and probably won’t be necessarily mobile gaming all the time. This is a pretty tall ask though and I don’t host tours so I’ll let people more involved in the hosting scene make their statements before me.
If upper staff wants to do something about bots, it should be something drastic and not a shortsighted half-measure that hurts real players more than it does bots. A shorter timer doesn't make ghosting/bot usage harder through some random other mechanism; it makes that stuff harder because it makes playing the game coherently more difficult in the first place (which, incidentally, would motivate an unassisted human to find illegal aid via ghosts/bots MORE rather than less). I would hazard a guess to say that a bot optimized for it could produce better-quality turns in Blitz formats than most humans anyways. What Goldilocks zone could possibly exist out there that is worth forcing a very large population of human players to relearn the constraints of the game they play? Bots are a difficult issue, but "make the game harder for humans" is not going to help at all.
PS regardless of how the current timer limits were set, considering it is the status quo on the sim, it is not valid to change them sans a significant burden of proof against them (which does not exist now, and probably won't exist ever)
I don't dispute that timer usage is healthy and it's good to use it as much as possible. The point of the post is to question whether the current timer limits we have set are effective - there's a possibility that the current limit is too high and there is no meaningful difference in reducing it by e.g. 10s. Taken to the logical extreme, external influence is impossible in Blitz mode, so where is the right balance? If we cannot conceive of reducing the timer by even a single second, then that indicates to me that it's actually not long enough and we should be advocating for increasing the timer?
This is also why I'm interested if anyone knows how the current limits were set, because if they're arbitrary then it's valid to investigate changing them one way or another
edit: ghosting is of course punished every time it comes up, but cannot be detected in all its forms. so making it harder in some way is a valid alternative when detection is not possible
I think the pushback is moreso that how can we even possibly measure if a policy like this is effective, while the negatives are very clear. If reducing timer by X % gives us a highly noticable decrease in player experience because of less time to think, and an unknowable positive effect on bot usage, is this a policy actually worth pursuing? Personally, I dont think so. It's just cutting your nose to spite your face, I think this policy would not leave anyone satisfied.
Edit:
If working with bot developers lets us come to some conclusion that an actual specific time reduction could tangibly reduce the effectiveness then I do think this is worth pursuing, it's just with the current information it feels a shot in the dark without a benefit people would feel is worth it.
fully agree that this is a very short-sighted solution that is going to hurt more than it helps, a major skill that goes into high level play is using your timer and we frequently see major tournament games where players run their clock down to 30 seconds or less. while i agree that the history behind how the numbers for our current timers were set is something worth looking into, and we should strive to make the least arbitrary decision possible, that feels like an entirely separate discussion than reducing timer values to take a swing at "bot collusion", which as mentioned by larry and earthflax, will be at best moving the goalpost and at worst totally ineffective due to the primary advantage of AI tools being that they can process information faster than any human. switching tabs to access your cheating bot should realistically take no more time than switching tabs to use the damage calculator or jot down notes, something that nearly every single player on this site does actively. the only point in favor of this that really makes sense to me is derpysux bringing up the idea of cartridge accurate chess timers, but again, separate point than what we're actually tackling here. totally not worth it when we're unsure if the effects would work as intended while there's a very strong and clear negative.
The primary advantage computational systems have over humans is speed. AI is not and likely never will be smarter than us because they aren't actually that intelligent, it's just that they are capable of making data comparisons so unfathomably faster than us, that their capacity to reach a regional optimum cancels out their inability to quickly intuit a solution. You could easily sort the numbers 1-10,000 in ascending value because you know how numbers work and can simply look at each individual one and place them accordingly, even if it would take days to actually accomplish. Most sorting algorithms are actually fucking bad at sorting numbers but when you can process all 10,000 numbers in less than a minute it doesn't matter, you've already blown way past what we are capable of. This was happening before the advent of modern AIs like LMMs, and in current days that gap between human intelligence and artificial intelligence has shrunk significantly.
Reducing the timer is not making AI tools worse, it's making humans worse. This does end up including humans that would use AI, since they are still bound to all the same physical constraints we are as humans, but the AI component they are leaning on is not. The ball is still in their court, especially now that the task of making an informed decision is harder on a non-AI user. If we were playing on Blitz rules, both a regular user and an AI-assisted user are going to be far more prone to missclicks or timing out compared to a regular timer setting, but the AI-assisted user can just defer any actual decision-making to the AI and never deal with the mental constraint of needing to plan out a long-term strategy or dwell over 50/50 scenarios. And a fully-automated AI doesn't missclick or run out of time at all as long as it continues to run properly.
If you wanted to reduce external influence with the timer, increasing it is far better than reducing it, as people manually running through calculations or future sequences have more time to catch up to the extreme advantage someone using AI has. This doesn't make for a good solution by any means, but when its competition actively makes regular users worse in exchange for maybe possibly slightly inconveniencing AIs that haven't yet adapted to the new constraints, what can you do.
proposal to lower timer is devastating news for players who have to take a piss mid-game and also want to wash their hands (optional)
on a serious note I think this isn’t really a real solution because bots can make direct inputs faster than a human can, and I think taking away from real players to stifle people sub-optimally cheating is a step in the wrong direction.
I think the point about matching the in-game timer is valid but not really the point of the thread, and it’s a QOL feature like the HP percentage mod that purely exists to make the competitive game more playable.
For team tours, I think the usage of a downloaded client is not out of the question, as these are players with much more dedication and probably won’t be necessarily mobile gaming all the time. This is a pretty tall ask though and I don’t host tours so I’ll let people more involved in the hosting scene make their statements before me.
Totally agreed that "just detect AI Showdex" is technically infeasible. Someone WILL make it, regardless of what we do, and it's not really detectable from a website's POV.
The idea of a downloaded client with anticheat is really interesting to me. If there's will and ability to do it, I think this might be a viable path. Multi-device cheating is still going to be a big issue, though, and that might be a situation where stricter time controls are warranted. I am concerned that this would be a big undertaking for a volunteer team, and effective anticheat methods are both controversial and cumbersome. Is this something sim developers are interested in working on?
Totally agreed that "just detect AI Showdex" is technically infeasible. Someone WILL make it, regardless of what we do, and it's not really detectable from a website's POV.
The idea of a downloaded client with anticheat is really interesting to me. If there's will and ability to do it, I think this might be a viable path. Multi-device cheating is still going to be a big issue, though, and that might be a situation where stricter time controls are warranted. I am concerned that this would be a big undertaking for a volunteer team, and effective anticheat methods are both controversial and cumbersome. Is this something sim developers are interested in working on?
The short answer is no, and the people who do the heavy lifting on the backend for development have no interest in this because any method is either wholly ineffective, not worth the amount of effort (especially considering showdown client is largely maintained via volunteer work), or both. you can read more in depth from them directly here, here, and here.
a shorter timer might lessen the use of human ghosting because it gives them less time to think and respond, but it's also detrimental to the humans actually playing the game for the same reason, and any ai worth its salt could spit out an answer in enough time that we'd only be hurting human play with this. we've got ais that can play champions now, which are all the "input data, wait for output" type of bot by necessity, and champions has a much shorter timer than we do. if they can do it over there, implementing the in-game timer won't be a preventative measure at all, and implementing anything shorter would severely harm the quality of gameplay
Am I the only one that is not that worried about AI in competitive Pokemon?
I cannot imagine a world where a computer can beat a Pokemon player that knows they're facing a computer, consistently. Pokemon is not like chess; it's not a game of pure mathematical calculation. There are genuine pure 50/50s, predictions, gut / instinctual calls, and assumptions, all happening in every turn. There are further variances from one opponent to another even if using the same team, and then there's the limitation of MU. Now, I am not sure how AI is topping ladder in the first place, but my understanding is that its looking at pure data at what most players do in this scenario or lead in this MU. You can even take advantage of this if you know you are facing a bot.
Furthermore, I am not sure if the use or the upside of these tools are being overblown. I mean, we already use plenty of tools. The damage calc is a good example because many VGC players consider it a form of cheating if you pull up your phone mid-VGC game to calc, but we don't. Now we have replay banks and detailed scouting tools that can explore the forums for previous instances of the team loaded with potential revealed sets. I've only seen some of these new AI tools, like the one about matchup and leading, which I think is a recommendation based on data aggregation, or like an AI tool that tells you the most common sets ran on this pokemon; it's just cutting down the work you have to put in to understand the tier enough to make assumptions mid battle.
It is not direct analysis or recommendations based on math, it is based on probable assumptions. It's basically just providing more readily available information to the player (possible sets, what tends to be a good lead in this matchup, what people tend to do in this scenario). Essentially, it is just doing what an intermediate player already does. It's not cutting through 50/50s, it's not going to account for niche sets or unique scenarios/matchups, and it's certainly not giving an experienced tour player an edge in a battle against peers, except in making calculations and assumptions faster.
Maybe someone could educate me on why I'm wrong, because maybe I'm not understanding or fully grasping this issue. Even though for general life and political policy I'm very against AI, It's difficult for me to see these tools as being really that dramatically influential on Pokemon.
I feel strongly that we should not reduce the timer. If anything, I would love to see more timer, not less, but I realize that's probably an unpopular opinion. Timer usage is something we should actively encourage, and shortening its duration devalues it as a tool to improve in-game play. While I understand the reasoning behind this discussion, I fundamentally believe that we should not worsen the experience for human players to combat the presence of bots in our game.
Am I the only one that is not that worried about AI in competitive Pokemon?
I cannot imagine a world where a computer can beat a Pokemon player that knows they're facing a computer, consistently. Pokemon is not like chess; it's not a game of pure mathematical calculation. There are genuine pure 50/50s, predictions, gut / instinctual calls, and assumptions, all happening in every turn. There are further variances from one opponent to another even if using the same team, and then there's the limitation of MU. Now, I am not sure how AI is topping ladder in the first place, but my understanding is that its looking at pure data at what most players do in this scenario or lead in this MU. You can even take advantage of this if you know you are facing a bot.
this is the main issue here. in the moment, you cannot know that you're facing a computer. if you only match up with a bot once on ladder and it doesn't disclose itself, there's no real way to distinguish it from a particularly bot-like human player. even in a bo3 setting it's hard to tell except in hindsight, and that's just the autoclicker bots; there are more sophisticated methods of bot usage that are impossible to detect. the strength of these bots is also increasing rapidly, in some cases approaching or even eclipsing top players in their formats, and they become harder to exploit the more advanced they get
now, even in the scenario where we do end up being able to know at a glance when we're facing bots and know exactly the lines to exploit them, the tour cheating problem goes away but a new problem emerges for ladder. a bot that you can just get free wins against by knowing "oh i just do x y and z thing and the bot falls apart" is a free stepping stone for anyone who knows the secret inputs, so people who figure them out can be rewarded with elo regardless of how well or poorly they play the game. a bunch of easily beatable bots on ladder would artificially inflate elo (which could affect usage stats in icky ways), possibly harm olt, and definitely harm suspects
Am I the only one that is not that worried about AI in competitive Pokemon? (...)
Now, I am not sure how AI is topping ladder in the first place, but my understanding is that its looking at pure data at what most players do in this scenario or lead in this MU. You can even take advantage of this if you know you are facing a bot.
no there are probably other people that don't understand it too, and as a result are more worried about bots making usage stats or suspect tests worse, or playing lots of bots on the ladder being an annoyance instead of being worried about the death of the game.
The bots do not work the way you suggest they do, where would they even find this data? To give a very simplified explanation they work first by training, during which it plays against itself millions of times and learns what leads to winning games more often. In this step it figures out that having your hazards out and the opponent not having theirs is good and that knocking out the opponent's pokemon is good, basic things like that. This gets it to the level of an average player.
Then once it's playing for real in any given turn it'll analyze its chance of winning right now, then its chance of winning if they click move a, b, c or d, with or without tera (and if it's clicking something like Blizzard, it's chance of winning in case it misses, or freezes, or crits, or just does regular damage) or switch to mons 1, 2, 3, 4, 5, and how each of these choices interact with the expected potential choices the opponent might pick. Then it'll play out the game against itself for a few turns for each possible choice, with each turn branching out further and then finally pick one of the options that increase its chance of winning the most, keeping in mind that it has to use a mixed strategy to achieve equilibrium play*.
Consider you have a Great Tusk that's faster than the opponent's Gholdengo, and they have a Landorus-T in the back. If you always Headlong Rush, they'd always go Landorus, so you should always Ice Spinner, but then they would always stay in, so you should pick Headlong Rush sometimes and Ice Spinner sometimes, and likewise for them between staying in and switching. This is an incredibly simple example to demonstrate equilibrium play, the real game is complicated by a trillion factors (hazards, pp, damage taken, tera used or not, teammates' hp etc) that the bot is better equipped to analyze than we are. Also, individual player psychology factors, which the bots don't have a way to interpret yet. It's not hard to imagine a bot in the future that you can feed every single game ABR ever played into and then have it be able to very accurately predict what ABR will do in a given situation, though that's not the problem we're facing rn.
The way this works for the bot is in a given position it'll decide that move a is the best move, move b is the second best move and switch to pokemon 3 is the third best move, then it'll assign a weight to all these options, based on how much they help improve its chance of winning. So consider a complicated turn where it thinks it should pick move a 50% of the time, move b 30% of the time and switch to mon 3 20% of the time. It'll then randomly pick one of these based on those odds. If it thinks move a is waaaaay stronger than anything else (like last mon Landorus-T vs last mon Heatran) then it'll assign a very high value to that and pick that every time.
You (and I, and everyone that played the game at a high level) do this too, except we just think "if I go ferro and he goes heatran that's bad, maybe i should go gliscor then i can set up rocks as he switches fearing eq, but if he goes weavile i'm better off going ferro hm..." while the bot which thinks a trillion times faster than us can go out and simulate each possible world and make the decision from there. Will it win 100% of the time like it does in chess? No, you can crit it and you can't crit Stockfish's queen, sometimes you'll have a great matchup, sometimes you'll outplay regardless, just like how sometimes a weaker player outplays ABR etc. But it will win a lot of the time, and anyone with access to it will win a lot against people that should be better than them.
The shitty triple weather bot that people used to face in the OU ladder is fundamentally different from this. You can't exploit the bot people are talking about now by making the same moves every turn like you could that one. The best VGC player in the world is probably already a bot, and the same being true for OU is only a matter of time (or more accurately, a matter of someone interested in building that coming along). Once that happens we can't hold meaningful online tournaments anymore.
So to address the point of the thread, I already said it on discord but I think that:
1) No price is too great if it means preventing people from using bots to cheat in tournaments
2) Making the timer shorter does nothing to prevent bots from being used in tournaments, and as such is not worth doing
I'm willing to accept very draconian measures that would have been considered absurd last year if it means halting AI, such as making playing tournament playoff games on a PS client that has some kind of anti-cheat built-in, having an evaluator that determines if people are using AI and tourbanning anyone caught by it with no possibility of appeal, webcams etc, because the consequence to having AI that is better than humans is literally the end of the game. But the measure suggested in this thread would only work if the bottleneck to the bot seeing the game-state is manual input from the user. The only publicly available tool has this constraint, but the closed models other people are using don't, and there's no reason to think that the first person to open their bot (or to make a strong bot and use it to cheat themslves) will have this constraint. If we can somehow stop the bot from reading PS! directly (which I think is impossible?) We can revisit this, there's no point in doing it until then.
no there are probably other people that don't understand it too, and as a result are more worried about bots making usage stats or suspect tests worse, instead of being worried about the death of the game.
The bots do not work the way you suggest they do, where would they even find this data? To give a very simplified explanation they work first by training, during which it plays against itself millions of times and learns what leads to winning games more often. In this step it figures out that having your hazards out and the opponent not having theirs is good and that knocking out the opponent's pokemon is good, basic things like that. This gets it to the level of an average player.
Then once it's playing for real in any given turn it'll analyze its chance of winning right now, then its chance of winning if they click move a, b, c or d, with or without tera (and if it's clicking something like Blizzard, it's chance of winning in case it misses, or freezes, or crits, or just does regular damage) or switch to mons 1, 2, 3, 4, 5, and how each of these choices interact with the expected potential choices the opponent might pick. Then it'll play out the game against itself for a few turns for each possible choice, with each turn branching out further and then finally pick one of the options that increase its chance of winning the most, keeping in mind that it has to use a mixed strategy to achieve equilibrium play*.
Consider you have a Great Tusk that's faster than the opponent's Gholdengo, and they have a Landorus-T in the back. If you always Headlong Rush, they'd always go Landorus, so you should always Ice Spinner, but then they would always stay in, so you should pick Headlong Rush sometimes and Ice Spinner sometimes, and likewise for them between staying in and switching. This is an incredibly simple example to demonstrate equilibrium play, the real game is complicated by a trillion factors (hazards, pp, damage taken, tera used or not, teammates' hp etc) that the bot is better equipped to analyze than we are. Also, individual player psychology factors, which the bots don't have a way to interpret yet. It's not hard to imagine a bot in the future that you can feed every single game ABR ever played into and then have it be able to very accurately predict what ABR will do in a given situation, though that's not the problem we're facing rn.
The way this works for the bot is in a given position it'll decide that move a is the best move, move b is the second best move and switch to pokemon 3 is the third best move, then it'll assign a weight to all these options, based on how much they help improve it's chance of winning. So consider a complicated turn where it thinks it should pick move a 50% of the time, move b 30% of the time and switch to mon 3 20% of the time. It'll then randomly pick one of these based on those odds. If it thinks move a is waaaaay stronger than anything else (like last mon Landorus-T vs last mon Heatran) then it'll assign a very high value to that and pick that every time.
You (and I, and everyone that played the game at a high level) does this too, except we just think in our head "if I go ferro and he goes heatran that's bad, maybe i should go gliscor then i can set up rocks as he switches fearing eq, but if he goes weavile i'm better off going ferro hm..." while the bot which thinks a trillion times faster than us can go out and simulate each possible world and make the decision from there. Will it win 100% of the time like it does in chess? No, you can crit it and you can't crit Stockfish's queen, sometimes you'll have a great matchup, sometimes you'll outplay regardless, just like how sometimes a weaker player outplays ABR etc. But it will win a lot of the time, and anyone with access to it will win a lot against people that should be better than them.
The shitty triple weather bot that people used to face in the OU ladder is fundamentally different from this. You can't exploit the bot people are talking about now by making the same moves every turn like you could that one. The best VGC player in the world is probably already a bot, and the same being true for OU is only a matter of time (or more accurately, a matter of someone interested in building that coming along). Once that happens we can't hold meaningful online tournaments anymore.
So to address the point of the thread, I already said it on discord but I think that:
1) No price is too great if it means preventing people from using bots to cheat in tournaments
2) Making the timer shorter does nothing to prevent bots from being used in tournaments, and as such is not worth doing
I'm willing to accept very draconian measures that would have been considered absurd last year if it means halting AI, such as making playing tournament games on a PS client that has some kind of anti-cheat built-in, having an evaluator that determines if people are using AI and tourbanning anyone caught by it with no possibility of appeal, webcams etc, because the consequence to having AI that is better than humans is literally the end of the game. But the measure suggested in this thread would only work if the bottleneck to the bot seeing the game-state is manual input from the user. The only publicly available tool has this constraint, but the closed models other people are using don't, and there's no reason to think that the first person to open their bot (or to make a strong bot and use it to cheat themslves) will have this constraint. If we can somehow stop the bot from reading PS! directly (which I think is impossible?) We can revisit this, there's no point in doing it until then.
Here to say that i agree the most with this post. And i think we have to look arround other communities that run into this problem before us, like speedrunning or nohit. Using a proper client with anti-cheat while cool i think it could take a lot of time to develop before we solve the problem. (im just comming back so i dont really know how bad it is) So i would like to propose a solution similar to the one these communities found. (that also is really simple) Recording your games from your pov. And i know this could be seen as invasive or too much for some tours, but i dont think its a horrible idea on finals, semi finals or big stake games. You dont need to test everyone just the ones that win the most (Powerlifting, MMA, Weightlifting do this, and even if does not work perfectly it does make it much harder to do.)
