it would be impossible to use any such tool in a vgc event (or any in-person battle). they don't even let you listen to music during your matches. AI is a non-issue outside of showdown.
-
Follow our Instagram!
-
Curious about what we're currently doing about bots? Check out the latest updates from staff in this thread.
-
Welcome to Smogon! Take a moment to read the Introduction to Smogon for a run-down on everything Smogon, and make sure you take some time to read the global rules.
Discussion Bots are now capable of suspect reqs, do we care?
- Thread starter freezai
- Start date
There have been perhaps ~10 games in the history of our tournaments where one player played perfect the entire game. Have a few examples here. Any game that goes longer than this/one player doesn't have a perfect matchup will have several mistakes by both players. Even at the highest level people blunder all the time. I got SM reqs yesterday and my one loss was because I Mega Evolved my Alakazam under Psychic Terrain to get a kill on a Tapu Bulu. Yeah. There's plenty of room for improvement in play from an AI that makes the optimal move every turn, especially if one side has it and the other doesn't, the difference is if both sides have it, then good players will know when to deviate from the optimal line to get an advantage.
The objective of what I'm proposing isn't to have the AI play the game for you, it's to have so no one has an advantage by virtue of having an AI other people don't have access to.
And for the people saying that the bot is very far away from being superhuman at SV OU, consider RBY OU. There are roughly ~13 viable pokémon, half of them use the same exact 4 moves every time, the other half has small variations*. There are no EVs, items or abilities. There's even less hidden information in RBY OU than in randbats and we use that tier for several trophy tournaments. A bot that can become superhuman at randbats can become superhuman at RBY. (I recognize no bot is superhuman at randbats yet). I'm willing to bet jaxcalibur could top the RBY ladder and do pretty well in RBY tournaments already.
Ideally we never get there and all the people that can build such bots are as principled as jaxcalibur_author, or the technology hits a ceiling but I find that unlikely. Compare what ChatGPT could do in 2023 and what it can do now. It's not hard to see the direction the wind is blowing. At some point you're gonna play someone in OST using an AI to play better, the difference is whether you have a comparable AI on your side or not.
*this is a simplification but mostly true.
Just for the record, Metamon's Tauros engine has already topped RBY's ladder.
This is just not true at all lol there are many many short games where one side did not make mistakes. I'm not sure what you mean by "perfect" (those are some good games in the link that I suppose fall within your definition) but many moves are simply a product of probability + prediction, combined with not making an obvious misplay. I've seen definitely more than 10 games this year so far where one player is not making any mistakes - that's playing optimally. The "perfect" move (probability + prediction) oftentimes loses because you made a good read. Bots can make reads too, but it's based on probability factors rooted in training. It's not definitively "worse", I just fail to see how it can be definitively better when a great mons player is already making these levels of decisions. If you argument is "a bot won't forget that Alakazam will Trace Bulu's Terrain on Mega Evolution" then yes you are correct, it will not forget that. But if your argument is that a bot-assisted SPL player
This isn't to say that having bots play with humans in tournament is "bad" - if anything we should be training off of them to get better ourselves. Banning them and pretending they don't exist is definitely not going to be the way to go. We don't want to ignore the issue and get swallowed by the eventual bot invasion - we can start improving now so the machines do not claim Smogon dot com. If a bot has tendencies and patterns, it can be learned by a human and defeated. I for one will not be used as an energy source battery for the machines.
yeah lets not do anything to disincentivize the creation and use of potential ghosting machines and lets instead use them as an excuse to make 50 pr threads about them
This is true but also the bigger point is your original post said "top players mostly do perfect plays". They do. Mostly. No word in this line indicates that you would mean they play perfectly all the time. Quite the contrary, "mostly" indicates the exact opposite.
Bots are going to be an absolute cancer to this website in the near future. Right now they can already be used to give you an advantage in tournaments and they infest the ladder. There is no more reasoning needed to do something about them. But i understand even coming up with something that works is really difficult, i have no solution myself bc im not a techbro. I just trust the greater minds of the community will come up with a solution to let us play in peace before Pokémon Champions VGC and the bots kill this website
Reminder that the peak ladder bot peaked randbats ladder, which has a lot more games to work off AND also has all the sets public (at least to my knowledge). The lc ladder bot did also do decently well but 77 gxe isn't exactly great (for reference my chopped ass main is at 78.9 gxe, and I use a lot of meme shit like cetitan and yanmega on there. 77 is if the bit didn't dc a few games btw, it's at 73 or something so even worse).
People doomering about the end of Smogon is silly, the future of bots being better then players is far off and not an issue currently.
However, this will become a problem in the future and thus should be addressed as soon as possible, but we should not do drastic shit like only have one account per person. I don't have a surefire solution because I'm not a techwiz but doing stuff like banning accounts that clearly have way more games then humanly possible is at least a start.
Every time somebody tries to win a war before it starts, shit hits the fan. Let's not make this something that will ultimately backfire in our faces.
People doomering about the end of Smogon is silly, the future of bots being better then players is far off and not an issue currently.
However, this will become a problem in the future and thus should be addressed as soon as possible, but we should not do drastic shit like only have one account per person. I don't have a surefire solution because I'm not a techwiz but doing stuff like banning accounts that clearly have way more games then humanly possible is at least a start.
Every time somebody tries to win a war before it starts, shit hits the fan. Let's not make this something that will ultimately backfire in our faces.
"lmao why are you so worried? it's only ruined one tier so far. it probably won't even be that good in OU."
Motherfuckers I play the randombattles circuit religiously. I care about it more than any official, and let me tell you, the moment this gets out we are already fried. It will not be very long until it comes for you too, maybe it will take another breakthrough and some time but I am not doomposting about a hypothetical future.
I am doomposting about a PRESENT issue - the legitimacy of any and all randombattles tournaments is in danger, including the world cup which I helped build from the ground up and have helped run for 5 years, the circuit which I have played for the same timeframe, and RBTT - possibly the biggest teamtour on the website, and to my knowledge the only non-official teamtour with custom avatars for all participants. This community might not mean anything to you but it means a lot to me.
I am ALSO doomposting about a very near future issue - as has been pointed out, RBY OU is very light on the hidden information and if a bot can hardcode and train on all possible gen9randombattle matches (500+ viable pokemon with multiple viable sets) it sure as fuck can hardcode and train gen1ou (like 25 viable pokemon if you're being very very generous, realistically something closer to 15, many of them with not much variance in sets at all). GSC isn't a big jump in complexity either, once you add EV spreads it's a different ballgame but this tool can definitely already ruin many important formats.
Obviously the problem is more complex in CG OU, but are we fine if the bot ruins a lot of our formats just as long as it doesn't touch CG OU? Come on lmao. TO ME, if Random Battles and RBY are ruined (and they sure look like they are!), I might as well be done with Smogon. It's genuinely a VGC angle. Your glorious formats with variable EV spreads will undoubtedly last a bit longer, but if I was in your shoes I'd at least be a little concerned.
And re: "well it's not like it can be that much better than humans anyway", two replies:
(1) Yes the fuck it can bro. You're sounding like a guy watching the 1996 match Kasparov 4-2 Deep Blue and going "oh well Kasparov's still a bit better, no worries then."
You can have all your technical speculation for why Pokemon is different (tm) and definitely not going to be solvable (tm) but that just sounds extremely extremely careless to me. We have seen this song and dance countless times, "oh how can it win at Poker when it doesn't understand bluffing and reads?", "oh how can it solve a search space as big as Go?", and so on. Scrabble is going through an eerily similar situation right now as well (the best bots are better than every human on earth other than One Guy. i've been saying michaelderbeste truly is the nigel richards of pokemon)
I find everyone going "we're different bro our game is different don't even worry this shit ain't solvable bro the indomitable human spirit will never be topped by mere bots" to be very naively optimistic. Like, yea, it is in fact a little different, and there is a chance bots don't develop further than Jaxcalibur for a while longer or maybe ever (technical challenges + lack of resources invested in development), but simply ignoring the threat and confidently claiming that it is impossible for it to materialize seems extremely shortsighted to me.
I could go into details of why I believe the bots have real potential to outclass and outperform humans in meaningful ways, but it's really not the point. The point is that nobody knows for certain until it happens, but I believe we have reached a stage where it would be fucking stupid to ignore what is currently available, let alone what might be available in the near future.
(2) Even if it never gets beyond "slightly worse than the top players", are we supposed to take that with a smile on our faces??? Brother I've been playing RBY OU for 8 years, any charizardfan I run into might be able to tap into an engine that gives him 40-60 odds against me, and I'm supposed to say "ah yea no problems there I've still got a positive expected winrate"? What the fuck are we talking about man?
Some of you guys are talking as if the only problem scenario is the bot becoming literally Stockfish-level unbeatable. It does not need to be that good to be an existential threat. If Jaxcalibur releases TODAY, my teammate playing a Gen9 Random battle tiebreaker in a couple hours from making this post cannot trust that his opponent is authentically playing the game. We would not be able to finish the tournament in good faith knowing that everybody (derbeste's team got eliminated r1) could very easily gain an unfair advantage.
I've introduced the RBY Ladder Tour to the circuit a couple years ago, now as fiish pointed out above, in May 2026 a bot topped our ladder. Am I supposed to close my fucking eyes and just go "yeah bro no big deal man I still beat this bot 1v1 >50%"? Really? Be serious man. I don't think I can run this tour again in 2027. The idea that bots able to damage our ecosystem are still very far from reality / never going to happen is so fucking foolish, it is happening RIGHT NOW to my favorite formats, it may be only a matter of time until they come to all the others too. Smogon has a problem even if they don't make that jump.
Finding solutions is another can of worms, I'm entirely unconvinced by the "if you get caught it's lifetime ban" stuff - obviously we can put that rule in writing very easily, but it's not a deterrent as the risk of getting caught is zero. But let's at least all get on the same page and agree that we have a huge fucking problem on our hands first? That seems wise to me.
I want to develop on Luigi's point that games are rarely played perfectly even at the top level, as it is a truth essential to understand the threat bots are to online competition. One may laugh at his anecdote with M-Alakazam on the SM Ladder, but such errors happen all the time even at the highest level. There are tons of gifs of high level players making oopsies that get regularly posted on Discord (Sahki) and most of them come from important tournaments.
SPL Week 9: https://replay.pokemonshowdown.com/smogtours-gen6ou-824956 - turn 28, both players seem to have forgot about Tracing Contrary, despite it poping up in chat on the previous turn.
Classic Finals g1: https://replay.pokemonshowdown.com/smogtours-gen1ou-660803 - turn 4, SoulWind didn't check his HP before clicking Soft-Boiled
Slam Finals g2: https://replay.pokemonshowdown.com/smogtours-gen9ru-714068 - turn 16, mncmt tera his Pokémon and it still fainted
Classic Semifinals g3: https://replay.pokemonshowdown.com/smogtours-gen1ou-800547 - turn 71, mind gaming let Wrap PPs overflow
SPL Week 1: https://replay.pokemonshowdown.com/smogtours-gen7ou-670237 - turn 40, Ojama forgot M-Alakazam could Trace Storm Drain (he admited live as he was playing the game).
Now that this is out of the way, I believe it's safe to say that top players aren't immune to making bad plays and they surely make more little suboptimal choices that are harder to detect for non-top player like me. Even across top players, there is not always a consensus on what was the best play in any given situation, so talking about playing perfectly makes no sense. Pokémon is a mentally challenging game. There are a lot of things you constantly have to consider and remember; and it's only normal for humans to fail sometimes. Bots don't and are superior to humans in this aspect.
Bots will one day play Pokémon better than humans. Nodoby knows when that day is, but it is certain it will come like it did for chess and go. As Amaranth rightfully pointed out in his post just above, tournaments competitivity is already jeopardized in many prominent formats. RBY and Randbats have their own ribbons, and RBY is also featured in both SPL and Classic. There are some questions I would like to read users who have been working on bots answer so we can have some expectations for the near future.
- Is there any reason to believe GSC won't be solved as easily as RBY and Randbats, given it also has fixed EV spreads?
- How much of an obstacle to bots EV spreads are?
- Are bots close to being able to react to sets it hasn't been trained on?
- Are bots able to reuse data from different metagames? Are bots able to keep up with metagames that keep evolving through banlist changes like CG Lower Tiers?
The answers to these questions are necessary to evaluate how much Smogon has to react. However, regardless of these answers, immediate solutions are necessary for RBY and Randbats.
SPL Week 9: https://replay.pokemonshowdown.com/smogtours-gen6ou-824956 - turn 28, both players seem to have forgot about Tracing Contrary, despite it poping up in chat on the previous turn.
Classic Finals g1: https://replay.pokemonshowdown.com/smogtours-gen1ou-660803 - turn 4, SoulWind didn't check his HP before clicking Soft-Boiled
Slam Finals g2: https://replay.pokemonshowdown.com/smogtours-gen9ru-714068 - turn 16, mncmt tera his Pokémon and it still fainted
Classic Semifinals g3: https://replay.pokemonshowdown.com/smogtours-gen1ou-800547 - turn 71, mind gaming let Wrap PPs overflow
SPL Week 1: https://replay.pokemonshowdown.com/smogtours-gen7ou-670237 - turn 40, Ojama forgot M-Alakazam could Trace Storm Drain (he admited live as he was playing the game).
Now that this is out of the way, I believe it's safe to say that top players aren't immune to making bad plays and they surely make more little suboptimal choices that are harder to detect for non-top player like me. Even across top players, there is not always a consensus on what was the best play in any given situation, so talking about playing perfectly makes no sense. Pokémon is a mentally challenging game. There are a lot of things you constantly have to consider and remember; and it's only normal for humans to fail sometimes. Bots don't and are superior to humans in this aspect.
Bots will one day play Pokémon better than humans. Nodoby knows when that day is, but it is certain it will come like it did for chess and go. As Amaranth rightfully pointed out in his post just above, tournaments competitivity is already jeopardized in many prominent formats. RBY and Randbats have their own ribbons, and RBY is also featured in both SPL and Classic. There are some questions I would like to read users who have been working on bots answer so we can have some expectations for the near future.
- Is there any reason to believe GSC won't be solved as easily as RBY and Randbats, given it also has fixed EV spreads?
- How much of an obstacle to bots EV spreads are?
- Are bots close to being able to react to sets it hasn't been trained on?
- Are bots able to reuse data from different metagames? Are bots able to keep up with metagames that keep evolving through banlist changes like CG Lower Tiers?
The answers to these questions are necessary to evaluate how much Smogon has to react. However, regardless of these answers, immediate solutions are necessary for RBY and Randbats.
Yes, randbats it definitely is more of an issue, but also notice how I never said "well we should ignore it" (and IDT anybody else has), I just said that doing drastic measures is going to make shit hit the fan. We have dealt with stuff like mazaar before, we can deal with this by addressing it early but also not making drastic measures that just blow up in our face.
Again, didn't say to ignore it, so stop the crap about me not being a little concerned. Gsc and rby are much less complex, but with the bot not being publicly released, we do have time to formulate things to help deal with this issue. My current point is basically that going full force and implementing drastic changes will cause other issues (like for instance the solution of only one account but Smogon person, which yeah gl testing any teams on ladder which is a big way even top players test teams AND also makes it way more restrictive for people who may not want to create a Smogon account and just play mons at school of some shit).
It's not CURRENTLY better then humans, and probably not for a good few years, of which if we haven't figured anything out about this that sounds like just site dwarnism or some shit like that. Nobody is saying to ignore it, and if they are, they are stupid.
Ultimately, my major point is, address this, get it early, BUT do not do stuff that we will regret in the future. Making it seem like the end of the site is just unhelpful and at times actively detrimental. Did we die when mazaar or the kyurem cheating suspect occured. Well me typing this should tell you, no! We learnt from it and improved our processes/systems to make sure it doesn't happen again. We can do this for bots, and ultimately it is going to benefit the site as a whole.
I think it's pretty notable that Mazar got caught because a) the method he was using to cheat was really obvious, and b) he happened to be relying on another human to run the bot for him. Neither of those necessarily apply here.
How familiar are you with Poker? This is a big thing there, and there are two significant factors that you seem to be discounting:
1) Determining the optimal probability spread can be difficult. The game theory might say that you should click Hurricane 65% of the time, and Thunderbolt 35%; if you only click Thunderbolt 10% of the time, you're not playing optimally (in the long run).
2) Actually playing according to the probability is not trivial for humans, unless they know what probability they're aiming for. The only reliable way to do something randomly is to rely on an external source of randomness, but you can't really do that if you don't know what odds you want.
I don't know how big of a difference these make in Smogon tournaments. A lot of the tours are low sample size, and you have to play aggressively or you get knocked out. In Poker, you're playing in a setting that's like, bo1000. Also, Poker engines calculate the game-theoretical best play, whereas neural networks are... doing a lot of linear algebra stuff. In theory they should mooooostly end up in the same spot, but in practice, it seems like neural nets sometimes develop exploitable strategies, at least in perfect-info games (see KataGo, for instance).
In case the additional context helps anyone: the value of playing what the engine suggests in Poker isn't that it tells you exactly what your best option is. Basically, it tells you something like: "in this situation, you should raise 10% of the time, call 20% of the time, fold 70% of the time." The benefit of this is that, if you play the strategy it suggests, you will be "non-exploitable." Basically, nobody will be able to play a strategy that counters your strategy. When you deviate from the engine-optimal strategy, you might gain some advantage against an imperfect opponent. But an opponent who recognizes that you've deviated may be able to counter your strategy, taking advantage of you to gain an edge. Since humans can't play optimally, there are two big implications of this:
1) You want to know your theory as much as possible, so that you can play in a non-exploitable (ie, engine-optimal) way.
2) You want to recognize when your opponent is deviating from the theory, so that you can deviate from the theory in a way that takes advantage of them.
This is basically where the strategy in high-level Poker comes from, these days. This is also where a lot of Pokemon strategy comes from, by the way. The difference is that, when you're playing a bo3, the pressure to deviate for a short-term advantage is much higher.
Last edited:
A few scattered thoughts (mine, not as td):
- Let's put aside the technological consideration of whether a software-side anticheat solution is possible, because that's out of policymakers' hands entirely. If it did exist, I would be curious on what would happen if we mandated it for official tournament games (the rules already state that you "should" be using the Smogtours server, but it's not required, so it'd be one degree of separation removed from the present to fully mandate Smogtours, another to mandate whatever anticheat on top of that). If this is engaging a bit too far in hypotheticals, imagining a rule that mandates streaming your games live for host review is the same idea; that one doesn't require any technological innovation at all. My gut is that it would pretty substantially slash participant numbers in any open bracket tournament; like, the whole appeal of STour is that you can jump in and play games immediately, having to make any additional downloads on top of that might be hugely detrimental. Entirely a hypothetical at this point, and others may disagree, both on whether such measures would substantially cut participation and on whether that's worth it
- Main server ladder is another story entirely. The scope of this thread has kind of expanded pretty dramatically beyond the original, which was focused on suspect tests. Makes sense, because we know that we have options to rework suspect tests to minimize the potential impact of Showdown bot-users with no Smogon account on official tiering (VR tiering, tournament usage tiering, etc), so if there's nothing that can be done about the botting issue there, there are still obvious policy solutions with pros and cons. Ladder tournaments are tournaments so any participants would be bound by any potential streaming/anticheat download mandate were they to exist, but the issue of contestants repeatedly farming exploitable bots for ELO is messier. I didn't like it in Cycle 3 of OLT and it gets worse for non-official tournament ladder tours, where getting a quick response from Showdown staff to do something about it is probably harder
- I'm reminded of the short-lived thread on RNG manipulation in the early generations. IIRC the RNG formula was then changed on Smogon to be less exploitable in large part because if we kept it exploitable, then competitive Pokemon becomes a game of having a calculator open on a side tab to optimize your ability to set up and exploit certain RNG outcomes, and we don't want that to be the nature of competitive Pokemon. All of which is to say, while I'm empathetic to the idea that making bot-assisted gameplay the standard for competitive Pokemon (not too different from calc-assisted gameplay in principle) is the best way to remove the possibility of any player getting an unjust advantage over the competition, I don't know. Acting with some goal of what playing this game should look like is important
- Clarifying rules about using bots in tournaments and enforcing harsher punishments is within our power. Yeah, we're probably unable to socially police botting out of tournaments in the same way as ghosting (because ghosting requires a participant tapped into tournaments and that's just not the case for ghosting if strong bots become widely available), but policy can skew the risk-reward of trying to use one in tournaments to make it much less worth it. Adequately so? Not sure
- Let's put aside the technological consideration of whether a software-side anticheat solution is possible, because that's out of policymakers' hands entirely. If it did exist, I would be curious on what would happen if we mandated it for official tournament games (the rules already state that you "should" be using the Smogtours server, but it's not required, so it'd be one degree of separation removed from the present to fully mandate Smogtours, another to mandate whatever anticheat on top of that). If this is engaging a bit too far in hypotheticals, imagining a rule that mandates streaming your games live for host review is the same idea; that one doesn't require any technological innovation at all. My gut is that it would pretty substantially slash participant numbers in any open bracket tournament; like, the whole appeal of STour is that you can jump in and play games immediately, having to make any additional downloads on top of that might be hugely detrimental. Entirely a hypothetical at this point, and others may disagree, both on whether such measures would substantially cut participation and on whether that's worth it
- Main server ladder is another story entirely. The scope of this thread has kind of expanded pretty dramatically beyond the original, which was focused on suspect tests. Makes sense, because we know that we have options to rework suspect tests to minimize the potential impact of Showdown bot-users with no Smogon account on official tiering (VR tiering, tournament usage tiering, etc), so if there's nothing that can be done about the botting issue there, there are still obvious policy solutions with pros and cons. Ladder tournaments are tournaments so any participants would be bound by any potential streaming/anticheat download mandate were they to exist, but the issue of contestants repeatedly farming exploitable bots for ELO is messier. I didn't like it in Cycle 3 of OLT and it gets worse for non-official tournament ladder tours, where getting a quick response from Showdown staff to do something about it is probably harder
- I'm reminded of the short-lived thread on RNG manipulation in the early generations. IIRC the RNG formula was then changed on Smogon to be less exploitable in large part because if we kept it exploitable, then competitive Pokemon becomes a game of having a calculator open on a side tab to optimize your ability to set up and exploit certain RNG outcomes, and we don't want that to be the nature of competitive Pokemon. All of which is to say, while I'm empathetic to the idea that making bot-assisted gameplay the standard for competitive Pokemon (not too different from calc-assisted gameplay in principle) is the best way to remove the possibility of any player getting an unjust advantage over the competition, I don't know. Acting with some goal of what playing this game should look like is important
- Clarifying rules about using bots in tournaments and enforcing harsher punishments is within our power. Yeah, we're probably unable to socially police botting out of tournaments in the same way as ghosting (because ghosting requires a participant tapped into tournaments and that's just not the case for ghosting if strong bots become widely available), but policy can skew the risk-reward of trying to use one in tournaments to make it much less worth it. Adequately so? Not sure
If this is on the table, I imagine it would have to be past a certain round only. If it's not a requirement for the early rounds, I don't think it would impact sign-ups THAT much. I certainly would not want to be the TD whose job it is to review 500 round 1 vods to make sure neither player is looking at their phone.
I don't play serious tours, and don't really have a place in those policy/culture considerations. But as a lay person, I think top cut streaming requirements should at least be considered. Privacy concerns are real, but allowing facemasks would be enough to alleviate a lot of that. Cost concerns are also real, but I think webcams (inck phones, laptops, etc) are cheap and accessible enough that it wouldn't be a major problem. I will say, I deal with that latter issue regularly at work, and even my poorest students have never had a problem accessing a webcam of some variety.
I'm not getting a degree or money for playing Smogon tours 99% of the time so the comparison here is bad because your average Smogon user has little incentive to do this unless they are straight up forced, at which point we go back to ego's point about signups taking a hit. I should not have to pay use this site in any capacity, otherwise you might as well just do VGC events IRL and play Champions than worry about this. A big selling point to PS and Smogon is that it has an extremely low cost of entry. Paying for a Webcam and a facemask (??????) might be a like a 20-30 dollar budget but that is not nothing for a lot of younger people who might not even have jobs or would rather save for much more important matters. Your students might not have a problem because their moms and pops might foot the bill for whatever they need at the end of the day, but how understanding are their parents going to be when they ask them to support Smogon activities?
Also how practical is a just webcam going to be when a lot of this cheating concern is done so on a computer? You would need something that actively reads a client's computer to truly check for cheating (again really concerning privacy issues involved).
Well, I said I don't really have a place in hammering out the exact policy goals and parameters, but if you're going to grill me on the specifics, I guess I will. It's hard to not feel a little attacked when you hit a guy with the sextuple question mark parenthetical, y'know. I would forgive you for glossing over the hidebox, so I just want to state for the record: I know all of this sounds like a miserable time for tournament directors, who already have the most thankless job on the thankless job website. I really have nothing but respect for people who volunteer to keep this machine running, which I understand includes you. I'm not saying this stuff because I'm a naive kid who thinks TDs should just try harder. I'm saying it because I think it presents the most viable solution to having tournaments not be overrun by bots in the next few years.
Everybody playing in a Pokemon Showdown tournament has, at minimum, access to a computer capable of connecting to the internet. They also, probably at some point, had access to some kind of video game console. I am sure there are people on this site who have been truly without a dollar to their name before -- we don't need to talk about the circumstances of my childhood, but they weren't much better.
As you said earlier in the thread though: if you care about competing at a high level, shouldn't you be expected to make sacrifices to do it? If you're okay with someone being forced to devote time and energy to get suspect reqs, I don't see why you wouldn't be okay with someone being forced to find access to a piece of cloth, and a computer with a camera attached. This is something that is within access of any child who is regularly using the site, I can assure you. It might be annoying for them to access, but compared to the amount of effort it would take to actually make top cut in a Smogon official, that effort may as well be nothing. I certainly went through more effort than that to access some of my hobbies when I was a teenager.
You do not have to pay for a webcam (and incidentally, if you do, we're talking more like a $10 budget). If you have access to a smartphone, that will do. If you have access to a chromebook, that will do. If you have access to a public (or school!) library, that will very likely do. If you don't have access to any of those things, well, that sucks I guess. I mean, I genuinely feel bad thinking about it! We're talking about a child who plays Pokemon -- probably a child I would relate to a lot -- who cannot scrap together a few bucks for a one-time purchase. That just sucks. But I don't really feel bad thinking about how that kid won't be able to face off against ABR in the ADV Cup finals. It's just not really their biggest problem, at that point.
You mention privacy concerns about watching someone's entire computer. They have free software that will actually record your computer's entire screen, in real time, and share it live on the internet. As the NPC in Palette Town says, technology is incredible!
As for the facemask thing -- I was specifically talking about a cloth facemask, of the kind you would have seen during the COVID pandemic. My rationale for mentioning that was specifically to address your earlier point about safety/privacy concerns. I certainly do not want to obligate any children to share personally-identifying information to play in a Pokemon tournament -- though for the record, participating in the higher levels of VGC does that, and they have kids younger than Smogon. Still, I think it is easily dealt with. We REALLY cannot be talking about children who don't have access to a single scrap of cloth. If you think that isn't enough to alleviate the security concerns, then fine, that's your prerogative. I think it's a pretty reasonable step, though, in this imaginary scenario where a 13 year old is reaching the later stages of a Smogon official tournament. I will freely admit that you know infinitely more about tours than me, but is that really a thing that happens?
I brought up my job just to illustrate that I am not totally naive to the woes of being a poor child in 2026. For the record, I have never ONCE had a parent actually follow my suggestion to buy a $10 webcam at Best Buy. When the students don't have a webcam, they universally end up coming to the next session with a cell phone camera. A lot of those students get access to our services 100% for free, either through their school system or as a job benefit that their parent has access to. To be frank: these kids are actively solving these exact tech/logistics problems, on their own, to come to DO SCHOOLWORK, for classes that their parents usually don't even remember they have! So yes, I think a child who makes a cinderella run in a weeks-long online Pokemon tournament can probably figure it out.
I recognize that none of this sounds like fun, and none of it sounds like it would be a foolproof method to prevent cheating. But that's just kind of the world we live in. You can do nothing, or you can do something.
As you said earlier in the thread though: if you care about competing at a high level, shouldn't you be expected to make sacrifices to do it? If you're okay with someone being forced to devote time and energy to get suspect reqs, I don't see why you wouldn't be okay with someone being forced to find access to a piece of cloth, and a computer with a camera attached. This is something that is within access of any child who is regularly using the site, I can assure you. It might be annoying for them to access, but compared to the amount of effort it would take to actually make top cut in a Smogon official, that effort may as well be nothing. I certainly went through more effort than that to access some of my hobbies when I was a teenager.
You do not have to pay for a webcam (and incidentally, if you do, we're talking more like a $10 budget). If you have access to a smartphone, that will do. If you have access to a chromebook, that will do. If you have access to a public (or school!) library, that will very likely do. If you don't have access to any of those things, well, that sucks I guess. I mean, I genuinely feel bad thinking about it! We're talking about a child who plays Pokemon -- probably a child I would relate to a lot -- who cannot scrap together a few bucks for a one-time purchase. That just sucks. But I don't really feel bad thinking about how that kid won't be able to face off against ABR in the ADV Cup finals. It's just not really their biggest problem, at that point.
You mention privacy concerns about watching someone's entire computer. They have free software that will actually record your computer's entire screen, in real time, and share it live on the internet. As the NPC in Palette Town says, technology is incredible!
As for the facemask thing -- I was specifically talking about a cloth facemask, of the kind you would have seen during the COVID pandemic. My rationale for mentioning that was specifically to address your earlier point about safety/privacy concerns. I certainly do not want to obligate any children to share personally-identifying information to play in a Pokemon tournament -- though for the record, participating in the higher levels of VGC does that, and they have kids younger than Smogon. Still, I think it is easily dealt with. We REALLY cannot be talking about children who don't have access to a single scrap of cloth. If you think that isn't enough to alleviate the security concerns, then fine, that's your prerogative. I think it's a pretty reasonable step, though, in this imaginary scenario where a 13 year old is reaching the later stages of a Smogon official tournament. I will freely admit that you know infinitely more about tours than me, but is that really a thing that happens?
I brought up my job just to illustrate that I am not totally naive to the woes of being a poor child in 2026. For the record, I have never ONCE had a parent actually follow my suggestion to buy a $10 webcam at Best Buy. When the students don't have a webcam, they universally end up coming to the next session with a cell phone camera. A lot of those students get access to our services 100% for free, either through their school system or as a job benefit that their parent has access to. To be frank: these kids are actively solving these exact tech/logistics problems, on their own, to come to DO SCHOOLWORK, for classes that their parents usually don't even remember they have! So yes, I think a child who makes a cinderella run in a weeks-long online Pokemon tournament can probably figure it out.
I recognize that none of this sounds like fun, and none of it sounds like it would be a foolproof method to prevent cheating. But that's just kind of the world we live in. You can do nothing, or you can do something.
Anyway, just to reiterate: this is all something I said should be considered as one potential option, if the details work out. If you're serious about Smogon official tournaments having any kind of future, I think it behooves you to either figure out those details, or figure out how to make Smogon Official LAN Tournament happen. I have no idea what other option there even is. Do you have another idea? I mean, genuinely -- without a shred of sarcasm -- do you? I would love to hear someone else propose a system that would do anything about people cheating with analysis engines. All of this stuff sounds like a huge pain in the ass, I don't want anyone to be subjected to it. It just seems like the thread is mostly stuck between "I welcome our new AI overlords" and "welp, we had a good run."
Last edited:
this thread has largely gone askew and we're all reacting in different ways to a pretty major development that upends what we know about a longstanding hobby. I've appreciated the good-faith posting and care apparent in most of these responses. I think we can all agree it sucks.
general responses:
general responses:
- smogon and PS's semi-anonymity, accessibility, and cross-section of ages has always been part of what drives this community. I met a plethora of thinkers, writers, and players when I was young on this website who, to this day, have shaped how I see the world, my politics, and my interests. that's the beauty of this small-but-not-small community. damn near one of a kind. forcing webcams, especially when children are involved, cannot happen. it is prohibitive and enables harm on a website that has already historically attracted and fostered predators. this really cannot continue being a point of contention. I don't think the people recommending it are posting in bad faith, but I've lived on this website for far too long and seen far too much childism to even consider it.
- screen recording at best, but we can already see the problems with that. truly nefarious actors are going to be difficult, if not impossible, to stop. and yea, at some point, why not just play Champions or VGC?
- feel similarly about one ladder alt per person, attached to a Smogon account. our community already has growth issues. making participation more annoying and less accessible seems like a bad response.
- we also have to keep in mind that there is very little inherent incentive for a new player to join us instead of playing tournaments sponsored by the developers of the actual game, which now has a phone app. we probably need to reconsider suspect requirements and eventually divest from ladder as the primary means of establishing them.
- I am very uncomfortable with the idea of normalizing AI in a user hub or giving everybody equal access to it. I hate the technology as a politic, so I'm obviously biased there, but man. it also just seems like it would strip so much personality out of the game and sanitize a lot of what I love about it. really, really don't like this as a kneejerk response. that doesn't mean it's necessarily a bad idea. again, we're all just reacting in real time to an imminent disruption of a game we love and hold dear.
- anyone denying the inevitability of this feels very myopic to me. AI technology has advanced an absurd amount in only a few years, and it is only going to become easier to build bots capable of playing these games at an increasingly high level. whatever technical barrier exists today is not a serious argument for what will or won't be possible tomorrow.
- Blizihguh did a wonderful job articulating how poker solvers actually work for a lay audience. their posts are mostly good and worth reading
- also note Blizihguh: august won OST at age 13.
I think time will prove that dismissing the implications of AI and bots for competitive Pokémon this confidently was profoundly naive.
I give this suggestion because, unlike the current proposal you advocate for, it moves the burden away from the player and onto the devs instead. Forcing people to buy or figure out how to obtain things to solve a crisis is needless headache if it can be solved on a technical level. Which I believe can be done. It would also be the least antagonizing route taken in regards to upsetting the tours scene and helps preserve suspect tests to an extent instead of leaving it to essentially die.
Since we're dishing strays ig, I never dismissed the implication of AI and bots, I specifically said the opposite if you don't cherry pick one line from my post. It's not something to dismiss or ignore, it's real and we have to deal with it. But I think dismissing our own abilities and tendencies to play at the highest level is patently more naive. The human element of competitive pokemon is immense when it comes to high variance metagames. Instead of throwing our arms up in the air about how all hope is lost, maybe we should try to understand how the bots work more, use them to get better ourselves, and see how they can benefit certain aspects of the website. If you truly want to integrate them into battling all you really need to do is have a standard PS bot available to literally everyone that can be toggled that highlights selections in various color shades in game based on % of how optimal that play is. Sure someone can make their own private bot to be slightly better, but at that point everyone has similar tools regardless and we can keep playing the game. That's not even something needed at the moment, and it may be too technically cumbersome to incorporate directly into PS without plug-ins etc.
Example of what I meant by above btw:
Honestly, while I still don't think it would be a privacy problem for tournaments per se, you are probably right about this. No aspect of this site should encourage children to figure out how to access a webcam if they don't already have one. Or like -- I recognize that the vast majority of children could figure this out, but it's a slight push in the direction of normalizing the idea of being on webcam with people from Smogon. Even a slight push in that direction should probably be avoided.
I honestly think basic screen recording + some kinda way to prevent phone usage would do 95% of the work, so it's kind of a bummer to dismiss. I guess a handcam would even be enough for that; it still doesn't feel amazing but it's at least better. I dunno, maybe we can mandate that you have to have your mic pic up TikTok running in the background, that way we can at least verify that they have one phone not in use for cheating
Thanks for the info btw and retroactive congrats to august. When I was 13 I was running Farfetch'd gimmick teams on ShoddyBattle and geeking out because I skipped Gen 2 and hadn't heard of Smeargle before.
EDIT: You know, VGC has a children's division and an adult division. This honestly might be something worth thinking about? I also think it's really important that kids be included in the site as equal participants to the older users, and this does kinda cut against that a little. But if bot ghosting really does kill tours, it might be a viable path.
Respectfully, I don't really see how that's a solution. Just kinda seems like replacing human tournaments with de facto bot tournaments. I'm not aware of any other game where you could run a tournament like that without it instantly devolving into "the human follows what the computer tells it to do without fail," so I'd imagine Pokemon will end up in that realm
Last edited:
i would like to drop a couple thoughts, starting with the strength of jaxcalibur. PLEASE note that these initial thoughts do not reflect my stance on any potential solutions to or ramifications of future bot development.
Jaxcalibur is strong. This is undeniable. BUT i am wary that some people in this thread may be doomposting and overexaggerating its strength. i am not good at rands. i only peaked around 2200, couldnt even get into RBEL, and on my first two matches against jaxcalibur, i won both. this is not to see i can beat it consistently; quite on the contrary, i proceeded to get beat 4 games in a row, win another one, then lose some more. But therein lies the problem. Looking at my first 7 games, i only lost 3-4 - and i made quite a few mistakes that were just objectively incorrect and honestly just careless. now my question is this: if some random guy who hasnt even touched rands in like half a year+ can have an okay chance at beating jaxcalibur, will top players taking a bo3 or something seriously not have a favorable chance? i mean unless you think i'm somehow some secret overlooked talent (which i'm not, but you're free to think so), trying to say "oh yeah this bot is leagues ahead of everyone... with the sole exception of michaelderbeste" and acting like jaxcalibur is already on the level of deep blue in 1996 seems disingenuous to me. again, please note - these thoughts DO NOT have any specific implications yet, i am simply stating that jaxcalibur is not that strong YET - a statement that i think can be treated objectively.
now, why does this matter? there are many good points in this thread about how the development of bots COULD lead to bots being much better and to the level of top level players. while i disagree with amaranth's comparison to kasparov vs deep blue, his underlying point that bots can and likely will improve at a rapid rate is correct. however, this has the asterisk of even more money and even more time being poured into bots - jaxcalibur already took relatively significant investment. im inclined to agree with this
where bots are a problem, however, are ladder and as guides. clearly, as jaxcalibur as demonstrated, bots can take over the rands ladder, and i echo the sentiments that rby ladder and maybe gsc ladder are in danger. furthermore, for players who already good, a bot could assist in unclear situations during a match. now, i would say that i don't think the idea of being "unexploitable" really applies in a bo1 or a bo3 series. a 50/50 remains just that, a bot is never going to pick up useful frequencies in these sample sizes. especially when players lose a 50/50, the bot has no way of determining if the player goes "dang it, guess i shouldnt do that i'll play differently next time" or "it's ok the same play will work next time". the idea of clicking certain moves with x, y, and z frequencies doesn't really work in a "short series" against one player. sure, in the long run, it might help, but also, "in the long run, we are all dead". a bot is not figuring out individual player tendencies that easily, and even if it figured out one player, it still has to play the next. delving into a player's replays to figure out tendencies is a whole story of its own, and i think bots might be able to do that to some success in rby - that is a whole different concern and is one of the main issues in my eyes.
so we have problems. what are the solutions? quite frankly, i don't have the slightest idea. on ladder, botting or playing assisted by bots is nigh-undetectable. in tournaments, getting ghosted by a bot is just like getting ghosted by another player - both are really hard to detect, and getting ghosted by a bot even has the benefit of the ghoster always being active and never reporting you. then, in terms of play, there's not really an "inhuman" plays because most of the time, there is no objectively correct line - especially one that's hard to find. when there is a "better" line that you try to cover, even a regular player could read it. bots are 100% undetectable in terms of predictions. as for avoiding silly mistakes, the absence of silly mistakes should not be met with suspicion. thus, bot-detection issues are likely unsolvable. then what? many of the solutions suggested in this thread are too extreme. forcing every account on ladder to link to smogon kills much of the casual playerbase. i also specifically have grievances with bliziguh's idea.
webcam and screen recording for tournament play is honestly a terrible idea considering newcomers might not want to go out of their way to do all of that, especially considering some people play on mobile which honestly makes it even more annoying. this is not to mention potential privacy concerns, as now any time-sensitive changes will be part of the recording whether it be a notification recorded on the computer or having to move locations in real life. i'm assuming we have to keep a continuous feed of both, otherwise such restrictions would be ineffective, so would i just have to reveal some of the things in the location around me? not to mention if somebody was playing in a location outside of their house, we know how good geoguessr players are and unless the camera was constantly kept facing the sky, some parts of the location would inevitably be recorded, and great, now people have a general idea of where you live. also yes, would just like to reiterate the issue of younger people of this site having large privacy issues, although as i am writing this, this has been addressed.
given all of this, what was even the purpose of my post? to just say, ok we have problems and can't solve them? no. This is where i might get flamed, but I just don't think bots are a problem in the short-term. Yes, bots will get better. Yes, these will be increasingly relevant problems. And yes, while I'm not entirely convinced of the inevitability of progression to a much more threatening level, we can assume that will happen at some point in time. But the whole point of this post was to say, we don't need to hit the nuclear button yet. We don't need to do all of these things that might kill decent portions of our playerbase yet. Of course, while I say this, I know that something has to be done. This thread has derailed from the original idea of just "hey do we care about suspect reqs" which of course we do, but beyond that, I think we should start smaller rather than just full sending a radical shift. This is not to take away from any suggestions in this thread. At the end of the day, everyone here is just trying to help keep competitive Pokemon fun and fair, and i believe everybody is acting in good faith. I think that having these ideas is good - I 100% agree that something needs to be done at some point. I just think that as of right now (RIGHT NOW), with the main issue being the thankfully non-open sourced jaxcalibur (which i believe to have an overexaggerated skill level), it is important to not be hasty and let the community as a whole gather more ideas and feedback before doing something that might just end up killing the community before the bots even get to it.
Jaxcalibur is strong. This is undeniable. BUT i am wary that some people in this thread may be doomposting and overexaggerating its strength. i am not good at rands. i only peaked around 2200, couldnt even get into RBEL, and on my first two matches against jaxcalibur, i won both. this is not to see i can beat it consistently; quite on the contrary, i proceeded to get beat 4 games in a row, win another one, then lose some more. But therein lies the problem. Looking at my first 7 games, i only lost 3-4 - and i made quite a few mistakes that were just objectively incorrect and honestly just careless. now my question is this: if some random guy who hasnt even touched rands in like half a year+ can have an okay chance at beating jaxcalibur, will top players taking a bo3 or something seriously not have a favorable chance? i mean unless you think i'm somehow some secret overlooked talent (which i'm not, but you're free to think so), trying to say "oh yeah this bot is leagues ahead of everyone... with the sole exception of michaelderbeste" and acting like jaxcalibur is already on the level of deep blue in 1996 seems disingenuous to me. again, please note - these thoughts DO NOT have any specific implications yet, i am simply stating that jaxcalibur is not that strong YET - a statement that i think can be treated objectively.
now, why does this matter? there are many good points in this thread about how the development of bots COULD lead to bots being much better and to the level of top level players. while i disagree with amaranth's comparison to kasparov vs deep blue, his underlying point that bots can and likely will improve at a rapid rate is correct. however, this has the asterisk of even more money and even more time being poured into bots - jaxcalibur already took relatively significant investment. im inclined to agree with this
where bots are a problem, however, are ladder and as guides. clearly, as jaxcalibur as demonstrated, bots can take over the rands ladder, and i echo the sentiments that rby ladder and maybe gsc ladder are in danger. furthermore, for players who already good, a bot could assist in unclear situations during a match. now, i would say that i don't think the idea of being "unexploitable" really applies in a bo1 or a bo3 series. a 50/50 remains just that, a bot is never going to pick up useful frequencies in these sample sizes. especially when players lose a 50/50, the bot has no way of determining if the player goes "dang it, guess i shouldnt do that i'll play differently next time" or "it's ok the same play will work next time". the idea of clicking certain moves with x, y, and z frequencies doesn't really work in a "short series" against one player. sure, in the long run, it might help, but also, "in the long run, we are all dead". a bot is not figuring out individual player tendencies that easily, and even if it figured out one player, it still has to play the next. delving into a player's replays to figure out tendencies is a whole story of its own, and i think bots might be able to do that to some success in rby - that is a whole different concern and is one of the main issues in my eyes.
so we have problems. what are the solutions? quite frankly, i don't have the slightest idea. on ladder, botting or playing assisted by bots is nigh-undetectable. in tournaments, getting ghosted by a bot is just like getting ghosted by another player - both are really hard to detect, and getting ghosted by a bot even has the benefit of the ghoster always being active and never reporting you. then, in terms of play, there's not really an "inhuman" plays because most of the time, there is no objectively correct line - especially one that's hard to find. when there is a "better" line that you try to cover, even a regular player could read it. bots are 100% undetectable in terms of predictions. as for avoiding silly mistakes, the absence of silly mistakes should not be met with suspicion. thus, bot-detection issues are likely unsolvable. then what? many of the solutions suggested in this thread are too extreme. forcing every account on ladder to link to smogon kills much of the casual playerbase. i also specifically have grievances with bliziguh's idea.
webcam and screen recording for tournament play is honestly a terrible idea considering newcomers might not want to go out of their way to do all of that, especially considering some people play on mobile which honestly makes it even more annoying. this is not to mention potential privacy concerns, as now any time-sensitive changes will be part of the recording whether it be a notification recorded on the computer or having to move locations in real life. i'm assuming we have to keep a continuous feed of both, otherwise such restrictions would be ineffective, so would i just have to reveal some of the things in the location around me? not to mention if somebody was playing in a location outside of their house, we know how good geoguessr players are and unless the camera was constantly kept facing the sky, some parts of the location would inevitably be recorded, and great, now people have a general idea of where you live. also yes, would just like to reiterate the issue of younger people of this site having large privacy issues, although as i am writing this, this has been addressed.
given all of this, what was even the purpose of my post? to just say, ok we have problems and can't solve them? no. This is where i might get flamed, but I just don't think bots are a problem in the short-term. Yes, bots will get better. Yes, these will be increasingly relevant problems. And yes, while I'm not entirely convinced of the inevitability of progression to a much more threatening level, we can assume that will happen at some point in time. But the whole point of this post was to say, we don't need to hit the nuclear button yet. We don't need to do all of these things that might kill decent portions of our playerbase yet. Of course, while I say this, I know that something has to be done. This thread has derailed from the original idea of just "hey do we care about suspect reqs" which of course we do, but beyond that, I think we should start smaller rather than just full sending a radical shift. This is not to take away from any suggestions in this thread. At the end of the day, everyone here is just trying to help keep competitive Pokemon fun and fair, and i believe everybody is acting in good faith. I think that having these ideas is good - I 100% agree that something needs to be done at some point. I just think that as of right now (RIGHT NOW), with the main issue being the thankfully non-open sourced jaxcalibur (which i believe to have an overexaggerated skill level), it is important to not be hasty and let the community as a whole gather more ideas and feedback before doing something that might just end up killing the community before the bots even get to it.
I wanna take a step back on this entire discussion and talk about something that is essential based on some of the comments I’ve read: what do we want out of competitive Pokemon for the future? Pokemon is a game with hidden information, as a lot of people have mentioned. This means that the knowledge of the player and their familiarity with their metagames is essential in the way in which they approach actions sequentially. The inclusions of AI that could be able to be fed and learn from resources is already a difficult and expensive process, but let’s think of what this means for the state of the game in a very nebulous scenario:
Metas already provide resources, calc and ladder for learning their metagames and allow players to acquire information through them. This is an essential part of the experience of playing mons that is external to your level of skill in a vacuum. Let’s just say for a second that creating a bot that processes all of this information is feasible and cheap, or that someone has a connection who is able to provide it for them: Do we want Pokemon to be a game where a lot of the information is already processed for you before you even load into a match, so your plays have a higher probability of being correct based on data? I doubt that this is the case for most people since this is an inherently human process that is vital to the competitive aspect of the game. This is why I think that the idea of “MichaelderBeste2 + AI vs. SoulWind + AI” is absurd as an ideal future (does anyone want to see this???). The second point in relation to this is that it is way more feasible for AI to gather more accurate information from bigger tiers with more resources and ladder. I am a PU player and am not personally afraid of this threat for lower tiers yet due to this, but I do feel like there is an infrastructure problem within Smogon overall if people cannot trust that the players playing at top level in the most played metagames aren’t using AI to cheat in a not-so-distant future. This sounds like a huge turnoff for anyone who lacks the resources or the knowledge to be on an even field and would make Smogon even less attractive in the face of Champions and VGC as a whole. An ecosystem that does not even the field in the future due to simply how difficult it is to do what the jaxcalibur guy did in randbats, which is already a tier with the easiest entry for information. With that being said, I think that even remotely considering player + AI variables of the game leads to the dissolution of the competitive aspects of the game as we know them. Do not seriously consider this option, it is not only incredibly damaging to the reputation of our game and the site but also not what a lot of people want the game to become.
A lot of people have also compared the idea of chess bots to what is going on on ladder or possibly in tour games in the future, and I wanna comment on something LBN said about chess bots being beatable. The beatable chess bots that exists are beatable because they are designed to be so. Magnus Carlsen will never be able to win a match against Stockfish. So, when people are discussing the subject of bots in tournaments, we first have to assume that someone has the strongest version of one and what effects this could have on play. First thing I will say here is that we are not only dealing with a layer of hidden information that is impossible to compare to that in chess, but that someone using a bot to cheat also would be harder to detect since our game does not have obvious winning moves in the same way that Chess.com or lichess detect cheaters. I think it is impossible to compare a game with all of the information in the world laid out to the player to the amount of variance that exists in this game. This is a good thing because it means that bots will be unable to be 100% accurate in Pokemon, since 100% accuracy doesn’t exist and it also means that the amount of resources necessary probably make it much less realistic in the present, but it also means that, if existing, we'd easily remove skill and expression in favor of a clear advantage. The nature of mons is also why engine analyses would not work for something like PS. The number of variants based on RNG would simply make it so you'd be able to see possibilities in your current position but not much ahead of that. I also fail to see the arguments that suggest that top players will simply be unable to lose to bots when the OP already showcases known players losing to them as well. I think it's important to recognize that players playing bots is different from players playing other players who are using bots. Bots are often exploitable and break at a certain point under certain circumstances, but a human that is able to discern between a human move and a bot mistake would simply be able to correct their play without ever hinting at the possibility of them utilizing AI. As it already stands, in a tier like LC, it does not seem like players vs bot matches are a 100-0 in favor of the player, so I can't imagine this will get any better for the humans, even if bots will never truly be as consistent as in a game like chess. I'm actually curious if there are other games with hidden information that have bots or AI of any sort that could more closely resemble what is going on in mons right now. I'd be interested in knowing how these situations are unfolding elsewhere, since I am ignorant to them. If you know, chime in!
Now obviously we’ve come to a circle here, and I still think we’re dealing with an issue that is just barely looming into the present into most tiers. The only concrete answer right now I would assume is that anyone using engines or bots to gain an advantage of any kind in a tournament setting should be immediately banned. But let's also look at the broader questions so we can figure out the rest hopefully. I wish I had a more definitive ending to this but I think that there’s 3 questions here that are essential and that I unfortunately am the wrong person to answer:
Metas already provide resources, calc and ladder for learning their metagames and allow players to acquire information through them. This is an essential part of the experience of playing mons that is external to your level of skill in a vacuum. Let’s just say for a second that creating a bot that processes all of this information is feasible and cheap, or that someone has a connection who is able to provide it for them: Do we want Pokemon to be a game where a lot of the information is already processed for you before you even load into a match, so your plays have a higher probability of being correct based on data? I doubt that this is the case for most people since this is an inherently human process that is vital to the competitive aspect of the game. This is why I think that the idea of “MichaelderBeste2 + AI vs. SoulWind + AI” is absurd as an ideal future (does anyone want to see this???). The second point in relation to this is that it is way more feasible for AI to gather more accurate information from bigger tiers with more resources and ladder. I am a PU player and am not personally afraid of this threat for lower tiers yet due to this, but I do feel like there is an infrastructure problem within Smogon overall if people cannot trust that the players playing at top level in the most played metagames aren’t using AI to cheat in a not-so-distant future. This sounds like a huge turnoff for anyone who lacks the resources or the knowledge to be on an even field and would make Smogon even less attractive in the face of Champions and VGC as a whole. An ecosystem that does not even the field in the future due to simply how difficult it is to do what the jaxcalibur guy did in randbats, which is already a tier with the easiest entry for information. With that being said, I think that even remotely considering player + AI variables of the game leads to the dissolution of the competitive aspects of the game as we know them. Do not seriously consider this option, it is not only incredibly damaging to the reputation of our game and the site but also not what a lot of people want the game to become.
A lot of people have also compared the idea of chess bots to what is going on on ladder or possibly in tour games in the future, and I wanna comment on something LBN said about chess bots being beatable. The beatable chess bots that exists are beatable because they are designed to be so. Magnus Carlsen will never be able to win a match against Stockfish. So, when people are discussing the subject of bots in tournaments, we first have to assume that someone has the strongest version of one and what effects this could have on play. First thing I will say here is that we are not only dealing with a layer of hidden information that is impossible to compare to that in chess, but that someone using a bot to cheat also would be harder to detect since our game does not have obvious winning moves in the same way that Chess.com or lichess detect cheaters. I think it is impossible to compare a game with all of the information in the world laid out to the player to the amount of variance that exists in this game. This is a good thing because it means that bots will be unable to be 100% accurate in Pokemon, since 100% accuracy doesn’t exist and it also means that the amount of resources necessary probably make it much less realistic in the present, but it also means that, if existing, we'd easily remove skill and expression in favor of a clear advantage. The nature of mons is also why engine analyses would not work for something like PS. The number of variants based on RNG would simply make it so you'd be able to see possibilities in your current position but not much ahead of that. I also fail to see the arguments that suggest that top players will simply be unable to lose to bots when the OP already showcases known players losing to them as well. I think it's important to recognize that players playing bots is different from players playing other players who are using bots. Bots are often exploitable and break at a certain point under certain circumstances, but a human that is able to discern between a human move and a bot mistake would simply be able to correct their play without ever hinting at the possibility of them utilizing AI. As it already stands, in a tier like LC, it does not seem like players vs bot matches are a 100-0 in favor of the player, so I can't imagine this will get any better for the humans, even if bots will never truly be as consistent as in a game like chess. I'm actually curious if there are other games with hidden information that have bots or AI of any sort that could more closely resemble what is going on in mons right now. I'd be interested in knowing how these situations are unfolding elsewhere, since I am ignorant to them. If you know, chime in!
Now obviously we’ve come to a circle here, and I still think we’re dealing with an issue that is just barely looming into the present into most tiers. The only concrete answer right now I would assume is that anyone using engines or bots to gain an advantage of any kind in a tournament setting should be immediately banned. But let's also look at the broader questions so we can figure out the rest hopefully. I wish I had a more definitive ending to this but I think that there’s 3 questions here that are essential and that I unfortunately am the wrong person to answer:
- To leadership, what do you want out of the games when it comes to the potential of bots processing information for the player and granting them best moves based on probability? Do you envision a world in which the fields are equalized by allowing players to utilize artificial intelligence (ignoring how monetarily and logistically impossible this is now)? This question is mainly asking if you'd consider this the good or the bad ending considering our available information.
- To tech people, what are our capabilities in detecting this threat in ladder and potentially outside of it? This is probably the most important question and also the one people are able to understand the least unless they're knowledgeable in the field.
- How should we move ahead with policy regarding this subject matter in the future based on the previous two answers? How do we prevent cheaters when the signs are a lot more ambiguous than in other games?
I don't have a specific solution in mind, but I know some things to be true.
1) Any attempt at personally "monitoring" the player – face, hand, browser, etc would instantly alienate and deter well over half of the playerbase and is a non-starter. Anything that requires "click yes to consent to xyz" would just mean most people clicking no. By extension, any true attempts at monitoring have to come from PS itself but people more technologically adept can speak on the specifics there.
2) Trust is not a sustainable system-wide solution. People who can cheat, especially without requiring collaboration, will cheat.
3) Technology advancing is inevitable, idk how fast the pace will be but "please don't build or open-source this" won't work either.
If it's not able to be deterred or prohibited via the above then yea that leaves "ok just allow it" indeed. In the meantime I think tours are okay but we can probably axe suspect ladders.
1) Any attempt at personally "monitoring" the player – face, hand, browser, etc would instantly alienate and deter well over half of the playerbase and is a non-starter. Anything that requires "click yes to consent to xyz" would just mean most people clicking no. By extension, any true attempts at monitoring have to come from PS itself but people more technologically adept can speak on the specifics there.
2) Trust is not a sustainable system-wide solution. People who can cheat, especially without requiring collaboration, will cheat.
3) Technology advancing is inevitable, idk how fast the pace will be but "please don't build or open-source this" won't work either.
If it's not able to be deterred or prohibited via the above then yea that leaves "ok just allow it" indeed. In the meantime I think tours are okay but we can probably axe suspect ladders.
I would just like to point out that, at least to my understanding after reading up on this thread, the title 'bots are now capable of suspect reqs' while technically true, may be a bit on the doomer side no?
There is currently only this randbats bot that is particularly powerful, and like was mentioned in the thread, it took a great deal of effort and money invested into it to reach that point, and only specifically in sv randbats, for now.
The little cup bot was able to achieve reqs playing over 150 games, should we not just be harsher with our requirements for people to achieve reqs? Unless im mistaken we have no reason to believe currently that the bot would be able to achieve reqs with a higher gxe than that, or a smaller number of games.
There is a whole other argument about whether we should make reqs harder or not(i think we should) but would this not, for the foreseeable future, alleviate this problem?
Reqs used to be harder back in the day with people needing to reset and try multiple alts after losing like 5 games in a row anyways
There is currently only this randbats bot that is particularly powerful, and like was mentioned in the thread, it took a great deal of effort and money invested into it to reach that point, and only specifically in sv randbats, for now.
The little cup bot was able to achieve reqs playing over 150 games, should we not just be harsher with our requirements for people to achieve reqs? Unless im mistaken we have no reason to believe currently that the bot would be able to achieve reqs with a higher gxe than that, or a smaller number of games.
There is a whole other argument about whether we should make reqs harder or not(i think we should) but would this not, for the foreseeable future, alleviate this problem?
Reqs used to be harder back in the day with people needing to reset and try multiple alts after losing like 5 games in a row anyways
ishtar already touched a bit on this point, but i just wanted to push back as well at the idea of "ai-assisted gameplay" that i've seen some people in the thread bring up and support. to be completely honest i'm a bit confused why that seems to be a train of thought here at all. i unfortunately do not have a solution to the botting problem but i just wanna say i think accepting "yeah just make it easier for everyone access to bots so the access to it isn't assymetrically unfair" is just, like, idk, why bother play pokémon (or any game for that matter) at that point? granted it's very clear we're not yet at the stage where these bots can give you an optimal probability spread of how often you should click each move, but imo if one turn alone of a battle has its outcome influenced by bot assistance i think that defeats the whole point of a competitive game like this. no one tunes in to the chess championships to watch magnus carlsen + ai vs fabiano caruana + ai, no one tunes to the poker championship to watch phil hellmuth + ai vs maria ho + ai. the whole interest of games like this is to use your own brain to find the best lines, do your best with luck management, and get calls right when it comes down to it, having an external tool help you just takes away from the actually fun, interesting and competitive part of competitive pokémon. as for spectating, people watch these games to vicariously enjoy the competition and root for someone, i feel like it's harder to root for someone when you know a considerable part of their decisions would be made by a bot exploring a search tree.
granted, there are out there tournaments for bots on these kinds of games, i think most famously chess, and i think that is extremely insteresting on its own, as especially on these games where they're better than humans, you get to learn aspects of high level play that go beyond human intuition as well as just take a new perspective on the game and find out how a computer tackles your favorite game. while i would 1000% tune in to watch a bot tournament for competitive pokémon out of curiosity for how they play and what lines they find and miss, i just feel like the idea of mixing that up with real players is just not really an interesting one at all, it takes the worst of both worlds. it's hard to attribute the good plays to the player, and you don't get to see at which point the bots are in terms of their skill, because maybe it's the player correcting a bad line from the bot, or not listening to the optimal move it suggested by disagreeing with it and throwing? idk. it just doesn't seem appealing to me on either aspect.
now, assuming these bots keep improving and at some point they surpass human ability, especially on tiers where it may be easier for them to do so like the aforementioned randbats, rby, gsc etc. i think that makes the "players + ai tag team" idea even worse. the better the bots become the less player the agency has, until you're just going through the motions of clicking the buttons for the ai, but what is the player even there for?
but again just to reiterate, even the bots on their current quasi-human level of play don't really add anything to the game either way. the most they do is patch up a weaker player's shortcomings or help remind you shouldn't calm mind with your mega alakazam after it's traced contrary from serperior or w/e. maybe they'll slightly (?) improve the level of competition at the level these bots are at by avoiding blunders like this, but again, is that really worth taking the agency out of the players and undermining the point of competition for? i've seen many comparison with other resources like damage calcs, but i think that's a really bad faith comparison. the damage calc does not think for you, it does not help you choose the moves, it just helps you inform and ground your OWN decisions, going to a math test with a calculator and going to a math test with an llm are two very different things.
anyway, i don't really know what else to say, i'm just so baffled that i even feel the need to write this out, i feel like it's just so clear that the idea of embracing the bots and encouraging won't really make the game better in any way? it seems to me like it's just weird cope with with the fact that this is a hard problem to solve. this feels like if we "solved" ghosting by "yeah just encourage everyone to play on call". i think the first step to resolving the threat these bots pose to online singles pokémon is to first of all address it as a threat, and not embrace it. that's my 2 cents i guess
granted, there are out there tournaments for bots on these kinds of games, i think most famously chess, and i think that is extremely insteresting on its own, as especially on these games where they're better than humans, you get to learn aspects of high level play that go beyond human intuition as well as just take a new perspective on the game and find out how a computer tackles your favorite game. while i would 1000% tune in to watch a bot tournament for competitive pokémon out of curiosity for how they play and what lines they find and miss, i just feel like the idea of mixing that up with real players is just not really an interesting one at all, it takes the worst of both worlds. it's hard to attribute the good plays to the player, and you don't get to see at which point the bots are in terms of their skill, because maybe it's the player correcting a bad line from the bot, or not listening to the optimal move it suggested by disagreeing with it and throwing? idk. it just doesn't seem appealing to me on either aspect.
now, assuming these bots keep improving and at some point they surpass human ability, especially on tiers where it may be easier for them to do so like the aforementioned randbats, rby, gsc etc. i think that makes the "players + ai tag team" idea even worse. the better the bots become the less player the agency has, until you're just going through the motions of clicking the buttons for the ai, but what is the player even there for?
but again just to reiterate, even the bots on their current quasi-human level of play don't really add anything to the game either way. the most they do is patch up a weaker player's shortcomings or help remind you shouldn't calm mind with your mega alakazam after it's traced contrary from serperior or w/e. maybe they'll slightly (?) improve the level of competition at the level these bots are at by avoiding blunders like this, but again, is that really worth taking the agency out of the players and undermining the point of competition for? i've seen many comparison with other resources like damage calcs, but i think that's a really bad faith comparison. the damage calc does not think for you, it does not help you choose the moves, it just helps you inform and ground your OWN decisions, going to a math test with a calculator and going to a math test with an llm are two very different things.
anyway, i don't really know what else to say, i'm just so baffled that i even feel the need to write this out, i feel like it's just so clear that the idea of embracing the bots and encouraging won't really make the game better in any way? it seems to me like it's just weird cope with with the fact that this is a hard problem to solve. this feels like if we "solved" ghosting by "yeah just encourage everyone to play on call". i think the first step to resolving the threat these bots pose to online singles pokémon is to first of all address it as a threat, and not embrace it. that's my 2 cents i guess
Ngl, if an open source tool gets released and everyone and their mom starts resorting to it for rands, it might just be preferable to play VGC because it's irl. It's not really fun when any chump can delegate decision making to a machine in a decision making game. For teambuilding formats bots are a bit more bearable.
Last edited:
