That's a good point, but the chess comparison is actually really apt here. Chess has a lot of serious online competition, but the most prestigious play, which people take the most seriously, takes place irl. With the prominence of VGC, you could argue that Smogon operates in a similar niche to something like LiChess (we don't have Chess.com money, sadly). Smogon formats are kind of a rough analogue to chess variants, which usually don't see "prestigious" irl play. That, to me, doesn't seem like a bad place to be.
Obviously there are some significant differences. Sites like LiChess generally aren't in charge of what the rules are, even for variants. But you could imagine a scenario where someone makes the first chess engine that works for Bughouse, for example, and the online Bughouse scene needs to figure out how to deal with it. Maybe that's already happened; I wouldn't know.
hi, you rang.
Bughouse engines exist and even despite the comparative lack of computing power/effort put into them as compared to regular chess engines and even considering the enormous complexities of bughouse, engines are pretty strong. certainly strong enough to be talking about them in the same sort of conversation as we are now regarding pokemon bot accounts getting reqs or being good enough to help a player cheat during a tournament, even if they aren't head and shoulders better than the best human players like they are in classical chess.
https://chess.stackexchange.com/questions/2590/are-there-any-bughouse-chess-engines
https://en.wikipedia.org/wiki/Sjeng_(software)
Engines struggled to dominate bughouse as easily as they did classical chess (or really how they could dominante any other single-board variant, including crazyhouse) predominantly because of attributes that arise from there being two boards and bughouse engines more so being built to play with human partners and attempting to make due with only looking at their own board, but even then they are still quite strong and humans did not like playing them.
suffice it to say really that any sufficiently complex game can probably be won by a computer over a human, given enough time and effort into making it happen. it was definitely only a matter of time before someone put their attention to pokemon in this way. even if we want to think about imperfect information and risk management as fundamentally human things that surely computers couldn't be good at, they are no obstacle. computers already mastered that in poker. with enough effort and resources, bots WILL be better players than humans at some point in the future.
I don't really know what the answer to this problem is, but chess is always a good analogy for how fair play can be ensured online given chess engines have been dramatically better than humans for decades. In tournaments with cash prizes etc, there's generally something like this going on:
https://www.chess.com/proctor
Doing something like this is at least technically speaking achievable for showdown for tournaments, certainly for trophy tournaments or others with cash prizes. Not saying it would be easy to implement or that it wouldn't require a major cultural shift. Just saying the general process for doing something like this technically has already been proven for us.
Ladder however... basically impossible. There have been steps taken to reactively try to limit the impact on usage stats, but in terms of proactively removing or even reactively "fixing" Elo won/lost from games against bots, it's just not doable. The only realistic way would be if it were impossible to run a bot locally and all bots only worked by being directly integrated into showdown itself using some process that would let showdown always be able to tell that it was a registered bot making a move. But how to actually get to that end state? You probably can't.
Assuming implementing something like that is impossible, well what else? Chess.com for example has an exceptionally complex and proprietary method for identifying cheaters. Lichess made theirs public here:
https://github.com/clarkerubber/irwin
But the basic method is to evaluate human moves against computer evaluations and this is only really doable because chess is deterministic, there's no RNG, and engines are fast and infinitely better than humans. You can quickly run a game through an engine that's orders of magnitude better than the humans who ostensibly played the game and identify every inhuman move that was made that sets off red flags for cheating. Above a certain threshold of certainty, act on that anti-cheating algorithm to remove or suspend bad actors from the site. Poker isn't that same type of game, and poker bots are way better than humans, but even in Poker there's just a lot fewer variables in play than Pokemon. I don't even want to begin to think about how you'd do something like this in Pokemon. So while I think there probably are methods to identify literal bots on the ladder by game volume, how long they take to make moves, etc. I really, really doubt there's ever going to be a way to identify a human playing with bot assistance on ladder, at least until pokemon bots are leagues ahead of humans. You kind of need there to be that gap to be able to identify that a human was playing like a computer. And of course there equally wouldn't be a way to identify bot usage for tournament games either, short of doing something like Proctor.
Btw I suspect that just like engines aren't as good at bughouse as they are at classical chess, they may be worse in doubles formats than singles formats (for now at least). Food for thought in the world of Champions and VGC being popular.
My take is that the real questions here go in the following order:
1) Is this existential enough to totally reshape formats the site wants to promote in the future? If bots can't get any good at doubles formats, is there a shift towards that? Would we want to just accept and in fact incentivize bot usage, integrate it into PS so that both players are getting recommendations and it's a level playing field?
2) How do you protect the prestige (and prize money) of tournaments? Which tours? Which methods?
and then way, way down the list
3) How do you handle the ladder/suspects? Gonna be real that this is the easy one. While there may be many people, myself included, who don't like using councils or tour reqs or whatever, these are all realistic options (assuming we can protect tours themselves).
