Battle Bot Nessie123: An OTS VGC bot that topped the Reg M-C Bo3 ladder

[Posho]

"upcoming SPL will be the last without AI"

if SPL players are willing and able to avoid using AI tools and ghosting currently they will remain willing and able to avoid AI tools in the future.

The barrier is not high.

Ridiculous fearmongering is not constructive.

Click to collapse...

You're gonna tell 100+ players to willingly not use a publicly available tool that tells you what the "optimal" move is for every possible turn. I'm fearmongering when the very people who are sharing these tools are saying they're aware of the adverse consequences they could have but they're willing to overlook that for the benefits.

I don't care about "constructive". This is a tool that endangers the state of a game I absolutely adore and I've spent years of my life polishing my skill in and I will speak out against it as much as possible. You guys take your AI tech somewhere they might appreciate it but not here.

 

[idkup]

You're gonna tell 100+ players to willingly not use a publicly available tool that tells you what the "optimal" move is for every possible turn. I'm fearmongering when the very people who are sharing these tools are saying they're aware of the adverse consequences they could have but they're willing to overlook that for the benefits.

I don't care about "constructive". This is a tool that endangers the state of a game I absolutely adore and I've spent years of my life polishing my skill in and I will speak out against it as much as possible. You guys take your AI tech somewhere they might appreciate it but not here.

Click to collapse...

If it were that simple the game would have died long ago and we would not be talking about vectorized simulations and search trees.

Due to the nature of the game, a strong human player who is on top of their game will always be ~= a flawless policy.

Discord is a free software that allows individuals to call their friends, some of which we may presume are stronger Pokemon players than they are.

Top-level tournament play is at relatively limited risk.

 

[Sqookblooks]

We're looking at potential solutions from the perspective of trying to stop the best possible play that could muck up elite level tournaments and top ladder but what about high ladder. Rn, I probably am roughly good enough to get to the 1700s-1800s on the main OU ladder which is not great but not terrible. Is there anything that could reasonably stop me from building "1700's the Bot" that can auto ladder me up to the 1700s without necessarily being the perfect StockFisk level bot. Seems like a recipe for fucking up ladder even if the Showdown Devs could make bot detection realistic for the best possible outcomes.

I'm not the biggest fan of VGC, but even then, I feel like there's still a lot of room for concern for that format, even in the physical in-person touraments. For one, VGC games tend to last around 7-8 turns, which is something I think a lot of people are forgetting when it comes to bot detection. Even winning turn one, or hell, just picking the best possible duo based off the opponent's team will have a massive impact on the game. I don't know the exact number, but chess takes roughly 40 moves for a game to be finished. I think it's unlikely that GameFreak is going to build their own anti-cheat detection software. Even for a potential Showdown anti-cheat, can we get so granular as to be able to detect when someone makes the perfect moves during certain states (i.e early, middle, end-game) without doing so on the other turns.

And with VGC going the way of OTS, are we going to get to a point where top players in VGC are expected to memorize as many possible outcomes as to not lose the early game? Some players already do this to an extent, but now they don't have to use their ingenuity or inductive reasoning. They can just plug in the opponent's OTS to the bot with their team and have the bot metagame the best possible choices.

And what about teambuilding? People love teambuilding, it's one of the big skills of Pokemon, but VGC is notorious for becoming extremely centralized around certain team comps as the season progresses. The bots seemingly aren't teambuilding right now, but whose to say they won't in a year or two from now? Are we going to get to a point where the mathematically optimal teams are all discovered within the first day of a new format?

There surely will be some positives, but we have to be realistic about the negatives that will come. One thing I do take issue with is the talk about chess. Chess didn't necessarily improve upon the intro of the bots, it just became a different sport. Same thing with Pokemon. The question is whether those changes are something we'll enjoy or not.

Also Showdown Devs, we need you to lock tf in.

 

[MyPokemonRBad12]

If it were that simple the game would have died long ago and we would not be talking about vectorized simulations and search trees.

Due to the nature of the game, a strong human player who is on top of their game will always be ~= a flawless policy.

Discord is a free software that allows individuals to call their friends, some of which we may presume are stronger Pokemon players than they are.

Top-level tournament play is at relatively limited risk.

Click to collapse...

There is no guarantee that any top player is legitimate, but at the same time that doesn't necessarily warrant the publicizing of mass machines to more people. There has always been nothing that we can do about cheaters who don't speak out about it, but does that mean that we should create potentially more? Also, newer players come into the scene at all times. Releasing a bot like this could drastically reduce the chance that they are real, and it isn't something that we necessarily want to see.
In the end, adding bots to competitive pokemon will just completely change how it works and because of the hobby being very much built upon human interaction UNLIKE chess which has historically been built on being as perfect as possible, the bots are far more likely to drive out the community than bring them together. It is just fundamentally not the thesis of pokemon to have external optimization (as it has become with chess) and that would require many people to put far more effort into playing competitively than they would want to. At the end of the day, this is a hobby and not a task where you are forced to spend hours to keep up. This isn't about doomposting, because bots are undermining the very fabric of what makes competitive Pokemon special.

 

[idkup]

There is no guarantee that any top player is legitimate, but at the same time that doesn't necessarily warrant the publicizing of mass machines to more people. There has always been nothing that we can do about cheaters who don't speak out about it, but does that mean that we should create potentially more? Also, newer players come into the scene at all times. Releasing a bot like this could drastically reduce the chance that they are real, and it isn't something that we necessarily want to see.
In the end, adding bots to competitive pokemon will just completely change how it works and because of the hobby being very much built upon human interaction UNLIKE chess which has historically been built on being as perfect as possible, the bots are far more likely to drive out the community than bring them together. It is just fundamentally not the thesis of pokemon to have external optimization (as it has become with chess) and that would require many people to put far more effort into playing competitively than they would want to. At the end of the day, this is a hobby and not a task where you are forced to spend hours to keep up. This isn't about doomposting, because bots are undermining the very fabric of what makes competitive Pokemon special.

Click to collapse...

Some element of how players are perceived has always come down to their communication skills. The advent of medium-effort ML tools that can play decent Pokemon only reinforces that. What someone says gives away a lot with regard to how they think - someone cheating with a bot will likely lack the knowledge to communicate about the metagame effectively. It will come down to captains in high level team tours doing their due diligence - avoiding drafting someone cheating with a bot is essentially the same to avoiding drafting someone cheating by ghosting in my eyes.

 

[MyPokemonRBad12]

Some element of how players are perceived has always come down to their communication skills. The advent of medium-effort ML tools that can play decent Pokemon only reinforces that. What someone says gives away a lot with regard to how they think - someone cheating with a bot will likely lack the knowledge to communicate about the metagame effectively. It will come down to captains in high level team tours doing their due diligence - avoiding drafting someone cheating with a bot is essentially the same to avoiding drafting someone cheating by ghosting in my eyes.

Click to collapse...

Again, is this something that we really want to devolve into? We are going to start tracking speech patterns on discord, we are going to track how close they play to the equilibrium, we are going to do all this instead of just not developing bots? If we don't make bots public, then it comes down to the fact that most people do not want to put in hours of effort to make, playtest, and use a bot to win tournaments only to gain false respect without monetary gain or anything of the kind. The reality is that not many people will want to keep up such a facade, and it eventually will get tiring. This is also why we have only had one Bloo in the history of Smogon. However, easy-to-access bots means that you have to put in far less effort but you still win all the big tournaments, and the tracking team has to put in 100s of hours of work to find out that you were maybe not even a cheater to begin with. Are we going to seriously ask this of unpaid people instead of tackling the issue at its core?

Let's look at it this way. If Stockfish wasn't public and nobody knew engines existed, people could in theory develop their own engines, make them play better, and use them at the Candidates and then Worlds to win. Is this very likely, considering that the whole thing is "fake"? No. But if the Candidates is hosted online and people have direct access to the engine, is it more likely that they use it to win? Yes.

None of us have an issue with how Smogon works at the present, we need to stop using arguments like "we can catch cheaters by doing XYZ and putting in a lot of time and effort and observing the people overtime" or "well people could be cheating right now" in order to snowball the bots and make it so that everyone has the best move at their fingertips.

 

[Quarante8]

Honestly the bots have never bothered me in any way or shape (we'll find ways to ban their usage as ghosting has been monitored more over the years, not really worried about that). What I dont really understand is the point of this project : VGC is at its core the IRL live events, and the bot serves 0 point for them. So you're essentially just showing you can make a tool to cheat in online events and even gloating about it saying its gonna be all around in a year or so anyway. All of this seems extremely callous to me. Your two main points are also inherently flawed : you did not prove anything as for furthering human understanding of high level VGC play. If anything, you're just holding someone's hand with the bot, if they had access to it and used it, how is that furthering our understanding our high level VGC play? The bot is telling them the plays, that is the complete opposite of learning. Second point is so moot I dont know if I even have to adress it. You're legitimately showing that anyone can get one of these bots up and running with literally minimal ressources. For VGC online events, that can award a large amount of money, the level of ressource investment for potential money you get back from going far in the event is ludicrous. You saying this is to help the community is laughable at best, you're either blind to what you're giving to people or straight up malicious. Refusing to answer anyone who disagrees with what you do is also quite incredulous. You are creating a problem to add to the pile that already exists and then asking people to find the solution for you as you have no idea of what you are talking about (just look at your replies with idkup which perfectly pinpoints the problem as that the best line is to expected at top level of pokemon).
You are not a fan of competitive pokemon of any kind or extremely misinformed (very easy to see by looking at your replies on game specific stuff), are very naive about all that you are doing or straight up malicious. To me whether it is malicious or not, you gave YOURSELF a challenge, managed to make it work and came here to gloat about it. Dont tell me otherwise, no one comes on this site specifically (look join date) to talk about their new bot and pretend its all in good faith. As if you had no idea this existed before and needed a place to put yourself in the light (when pretending you care about comp mons which is honestly icing on the cake). I can only hope that you are just a genuinely naive person, but from the OP and all your comments throughout the threat, seems to be like you're just some malicious user trying to gather some attention on your new toy under false pretense. Either way since you're apparently inevitable I hope the community comes up with solution for worthless stuff like what you're creating that are only made on a whim by a dev who has nothing to do with all of us.
Have a nice day

 

[idkup]

Again, is this something that we really want to devolve into? We are going to start tracking speech patterns on discord, we are going to track how close they play to the equilibrium, we are going to do all this instead of just not developing bots? If we don't make bots public, then it comes down to the fact that most people do not want to put in hours of effort to make, playtest, and use a bot to win tournaments only to gain false respect without monetary gain or anything of the kind. The reality is that not many people will want to keep up such a facade, and it eventually will get tiring. This is also why we have only had one Bloo in the history of Smogon. However, easy-to-access bots means that you have to put in far less effort but you still win all the big tournaments, and the tracking team has to put in 100s of hours of work to find out that you were maybe not even a cheater to begin with. Are we going to seriously ask this of unpaid people instead of tackling the issue at its core?

Let's look at it this way. If Stockfish wasn't public and nobody knew engines existed, people could in theory develop their own engines, make them play better, and use them at the Candidates and then Worlds to win. Is this very likely, considering that the whole thing is "fake"? No. But if the Candidates is hosted online and people have direct access to the engine, is it more likely that they use it to win? Yes.

None of us have an issue with how Smogon works at the present, we need to stop using arguments like "we can catch cheaters by doing XYZ and putting in a lot of time and effort and observing the people overtime" or "well people could be cheating right now" in order to snowball the bots and make it so that everyone has the best move at their fingertips.

Click to collapse...

this is a misunderstanding of how Stockfish was developed.

Stockfish uses hand-picked heuristics created by GMs who happen to also understand machine learning and a very small trained NNUE. Almost invariably, the Pokemon projects are 7-digit param transformers. (I have observed that blending value with certain heuristics results in a stronger policy, but that's a conversation for another day) If Stockfish weren't public, developing an analogue would be very difficult. Chess is deterministic, you can play millions of games against Magnus Carlsen and never win, as he always has perfect information.

Developing an 'adequate' Pokemon policy (one that can potentially beat top players), in my estimation currently requires the following:

- A basic familiarity with machine learning and neural networks (you should know what a transformer is, and what the LLM is talking about when it is talking about loss)

- A basic familiarity with competitive Pokemon (you should know when your engine is incorrect / your policy is doing something objectively wrong)

- $20 USD to either Anthropic, OpenAI, Moonshot, Deepseek...

- A working computer

None of these are nearly as rare as actually being good at chess.

There is no barrier. Someone will do it. Someone is almost certainly already doing it. As much as I would prefer tools not be released "because someone will do it eventually", we cannot be putting our heads in the sand. Just because it's not a good premise to open the floodgates completely doesn't make it any less true.

 

[nessie_dev]

Refusing to answer anyone who disagrees with what you do is also quite incredulous

Click to collapse...

I have replied to several people who are expressing their concerns (many of which I share) for what the potential future release of such tools could do to the integrity of online formats. I have made clear that I will, in general, not reply to people who argue in bad faith, or who assume bad faith on my part.

I'm willing to work with people who are making an effort to create bot-detection tools. How seriously was the community taking this 48 hours ago? How seriously is it taking it now? If you can answer these question, you might understand my intentions better, although I've been candidly, earnestly sharing them with you from my first post.

 

[Posho]

"I will not reply to people who argue in bad faith", so esentially you want an eco chamber.

We're at the mercy of people whose regdate is last week, have little participation in this community and want to show off their latest toy. If your intention was to help with bot detection tools, you'd be private messaging a Showdown dev with the intention to volunteer instead of posting this thread.

 

[monchooo]

VGC is at its core the IRL live events, and the bot serves 0 point for them.

Click to collapse...

VGC will definitely be affected by engine play. Worlds has a full day break before finals, if you wanna win you will be spending all night memorizing the top engine lines and on finals day seeing if you can remember them better than your opponent.

How long does it take to generate the top engine lines and learn them? You or a friend helping you might be able to do it in between rounds. The same way people memorize calcs vs popular mons/spreads they will check the top engine line vs Rinya Sun or Smurf HO and this would just be the legal stuff. All online competitions would be dead and every irl tournament is gonna have people claiming they’re opponent was using jaxcalibur anal beads telling them what to click.

Just because IRL guarantees a person is clicking the buttons doesn't mean that engines won’t fundamentally change the way mons is played and for most people it’s for the worse.

I'm willing to work with people who are making an effort to create bot-detection tools.

Click to collapse...

Hope you figure a way out. Don't know what the solution is but seems like a more important challenge to solve than seeing how you can optimize the bot even more.

 

[Schister]

"I will not reply to people who argue in bad faith", so esentially you want an eco chamber.

We're at the mercy of people whose regdate is last week, have little participation in this community and want to show off their latest toy. If your intention was to help with bot detection tools, you'd be private messaging a Showdown dev with the intention to volunteer instead of posting this thread.

Click to collapse...

Honestly you need to get off the computer man. It has been made abundantly clear that the barrier to bot creation is extremely low with modern publicly available AI capabilities, and this developer has stated that they will help with creating countermeasures if apporached for such a task. What more could you reasonably want?

You said in an earlier post that the bot is entire unhelpful to providing gameplay insights, but as someone who's watched how it plays a good bit now it's already doing things like bringing both mega slots on its team (and winning) far more often than human players do. This dev wants to engage with the community and help create tools and countermeasures and you and MyPokemonRBad12 as sitting here blowing a fuse over a doomed future and bad actors that almost certainly wouldn't be taking the time to discuss the logistics of the bots, analytical tools or offer to help create countermeasures.

Kindly shut the fuck up and go be productive somewhere else.

 

[MyPokemonRBad12]

as sitting here blowing a fuse over a doomed future and bad actors that almost certainly wouldn't be taking the time to discuss the logistics of the bots, analytical tools or offer to help create countermeasures.

Click to collapse...

I didnt expect to get pinged again but here we are

Firstly, my bad for making it seem like the future is doomed. Thats not really what I'm trying to get at even though it may come across that way.
I am putting forward concerns that I have over the future of Smogon in terms of integrity. I am saying that we have issues with the current framework and addressing the fact that we are not ready yet. I have also said that VGC is going to end up benefitting immensely from this change. Secondly, I had personally reached out to the dev and tried to understand their side of things and what direction they were looking towards to help catch cheaters. I was trying to understand why releasing the bot to the public is the choice we want, because that even further lowers the barrier to cheat.

For now, I'll leave it at this because this is kind of getting pointless. I'll continue discussion privately with the developer and see their side of things. I would request that I don't get pinged back to this thread anymore. Thank you.

 

[Posho]

Honestly you need to get off the computer man. It has been made abundantly clear that the barrier to bot creation is extremely low with modern publicly available AI capabilities, and this developer has stated that they will help with creating countermeasures if apporached for such a task. What more could you reasonably want?

You said in an earlier post that the bot is entire unhelpful to providing gameplay insights, but as someone who's watched how it plays a good bit now it's already doing things like bringing both mega slots on its team (and winning) far more often than human players do. This dev wants to engage with the community and help create tools and countermeasures and you and MyPokemonRBad12 as sitting here blowing a fuse over a doomed future and bad actors that almost certainly wouldn't be taking the time to discuss the logistics of the bots, analytical tools or offer to help create countermeasures.

Kindly shut the fuck up and go be productive somewhere else.

Click to collapse...

I'm aware of what the bot can do, but I literally don't care for it. I don't need a machine making decisions for me for the sake of "humanity understanding competitive pokemon on a deeper level". I love this game because of the intellectual challenge that it presents and it's because of developers like this parading their tools and people like you who openly support them that we're gonna lose that.

Yeah sure call me paranoid all you want while providing me the very reasons I should be paranoid about. What all you AI bros need to do is to stop being so damn invasive and spend your time in something more productive than trying to make competitive pokémon unplayable.

If you guys love competitive pokemon so much, use your energy actually trying to learn the game instead of cheating your way through it. I won't stop voicing my opposition.

 

[monchooo]

I greatly look forward to the prospects of the analysis tool you're working on here, I think it's a natural progression for the playerbase to understand the game at a certain level. As someone who's watched how it plays a good bit now it's already doing things like bringing both mega slots on its team (and winning) far more often than human players do

Click to collapse...

Words said by a man who's soul didn't quiver when pdc clicked z stealth rock

 

[idkup]

I'm aware of what the bot can do, but I literally don't care for it. I don't need a machine making decisions for me for the sake of "humanity understanding competitive pokemon on a deeper level". I love this game because of the intellectual challenge that it presents and it's because of developers like this parading their tools and people like you who openly support them that we're gonna lose that.

Yeah sure call me paranoid all you want while providing me the very reasons I should be paranoid about. What all you AI bros need to do is to stop being so damn invasive and spend your time in something more productive than trying to make competitive pokémon unplayable.

If you guys love competitive pokemon so much, use your energy actually trying to learn the game instead of cheating your way through it. I won't stop voicing my opposition.

Click to collapse...

"I love this game because of the intellectual challenge that it presents"

do you not realize how compatible that is with "I want to build a neural network that plays this game because of the intellectual challenge that it presents"

 

[PokeChess]

Whether people like it or not, I think there’s a good chance the use of bots in competitive Pokémon will increase. Trying to stop this feels like a losing battle.

Imagine a future where anyone can use botting tools. Even “worse”, imagine tools that simply show you what the bot would do, while you manually click and make the moves yourself. How exactly do you expect Smogon to stop that?

In that case, maybe we’d have to consider measures similar to those used in chess, like having cameras monitor players (although this wouldn’t prevent cheating 100% of the time, it would certainly help). The difference is that detecting suspicious play in chess is much easier, since it’s far more straightforward to identify a game where someone consistently plays the best possible moves. Pokémon is much more complex to analyze in that regard, making bot detection considerably harder.

The only real alternative for Smogon might be to have an in-person competitive category. Official in-person VGC would remain directly unaffected by all of this.

That said, I actually think it’s a good thing that people are openly reporting how successful these bots are. At least they’re being transparent about it and making the information available to the wider community. It could perhaps be worse if nobody talked about it and only a small group of people had access to this kind of “insider” knowledge while everyone else remained unaware of how effective these tools can be.

Finally, imagine if bots became so advanced that they could provide genuinely useful feedback on the plays we could have made better. Have you ever considered that this could actually be useful? Even professional VGC players could potentially benefit from something like that. Elite chess players already use engines to analyze their games, prepare for opponents, and find ideas they might have missed themselves. In that sense, couldn’t an equally powerful analytical tool for Pokémon also be used as a useful tool?

 

[idkup]

Whether people like it or not, I think there’s a good chance the use of bots in competitive Pokémon will increase. Trying to stop this feels like a losing battle.

Imagine a future where anyone can use botting tools. Even “worse”, imagine tools that simply show you what the bot would do, while you manually click and make the moves yourself. How exactly do you expect Smogon to stop that?

In that case, maybe we’d have to consider measures similar to those used in chess, like having cameras monitor players (although this wouldn’t prevent cheating 100% of the time, it would certainly help). The difference is that detecting suspicious play in chess is much easier, since it’s far more straightforward to identify a game where someone consistently plays the best possible moves. Pokémon is much more complex to analyze in that regard, making bot detection considerably harder.

The only real alternative for Smogon might be to have an in-person competitive category. Official in-person VGC would remain directly unaffected by all of this.

That said, I actually think it’s a good thing that people are openly reporting how successful these bots are. At least they’re being transparent about it and making the information available to the wider community. It could perhaps be worse if nobody talked about it and only a small group of people had access to this kind of “insider” knowledge while everyone else remained unaware of how effective these tools can be.

Finally, imagine if bots became so advanced that they could provide genuinely useful feedback on the plays we could have made better. Have you ever considered that this could actually be useful? Even professional VGC players could potentially benefit from something like that. Elite chess players already use engines to analyze their games, prepare for opponents, and find ideas they might have missed themselves. In that sense, couldn’t an equally powerful analytical tool for Pokémon also be used as a useful tool?

Click to collapse...

wrt "imagine if bots provide genuinely useful feedback"

In a situation where both teams are known and you have perfect information: don't have to imagine. Already exists. It's not a trivial build, but it isn't difficult either. I'd estimate it as a weekend project for a sufficiently motivated undergrad with a basic understanding of both worlds.

However, the utility of something like this isn't very high for strong players, as typically you realize you fucked up the turn immediately after, or failing that, at the end of the game. I'm sure we've all experienced that particular unpleasant feeling.

E: The threat presented by bots is that they will quantify the equilibrium and present it in a very accessible manner while searching deeper more quickly than any human can - not that they will play better than the strongest humans. Instead of "this looks winning", you get "this wins 80% of the time at equilibrium" without having to learn what "winning" looks like.

 

[Posho]

Finally, imagine if bots became so advanced that they could provide genuinely useful feedback on the plays we could have made better. Have you ever considered that this could actually be useful? Even professional VGC players could potentially benefit from something like that. Elite chess players already use engines to analyze their games, prepare for opponents, and find ideas they might have missed themselves. In that sense, couldn’t an equally powerful analytical tool for Pokémon also be used as a useful tool?

Click to collapse...

Pokemon is not chess. Pokemon does not have a finite number of options and is riddled with variables. If you lack the skill to reflect upon mistakes during your games and need a machine to tell you, it means that you're still not good enough at the game and probably won't ever be if you resort to such means to hack it lol

I just think, no offense, that most people who parrot these takes have never attempted serious runs in tournaments nor have they experienced actual high level play in high stakes situations. I don't care if the bot is good or better than the average player. It is the fact that bot play and player play has the potential to become indistinguishable that makes this game lose all its appeal.

 

[PokeChess]

Imagine the technology advances to the point where a bot can analyze YouTube videos in real time and tell me every Pokémon that has been used this year, every team it has appeared on, or mistakes players are making as they happen. A player could obviously do this themselves, but they probably aren’t going to manually analyze 30 one-hour-long videos in a short period of time.

Or imagine a bot analyzing over a thousand replays from a single player and identifying patterns in how they tend to play in very specific situations. There are a lot of things you could potentially do with that kind of technology. I’m just abstracting here and thinking about the possibilities.

Pokemon is not chess. Pokemon does not have a finite number of options and is riddled with variables. If you lack the skill to reflect upon mistakes during your games and need a machine to tell you, it means that you're still not good enough at the game and probably won't ever be if you resort to such means to hack it lol

I just think, no offense, that most people who parrot these takes have never attempted serious runs in tournaments nor have they experienced actual high level play in high stakes situations. I don't care if the bot is good or better than the average player. It is the fact that bot play and player play has the potential to become indistinguishable that makes this game lose all its appeal.

Click to collapse...

As for your take, that’s fair, and I can understand where you’re coming from. I just think it’s worth remembering that online games in general can run into engine-related issues, and Smogon isn’t necessarily an exception.

In-person VGC, as the official Pokémon competitive format, is naturally less affected by online bots than Smogon is. How exactly do you expect Smogon to stop that? Does anyone have a solution that truly prevents this?

 

[emonia]

If it were that simple the game would have died long ago and we would not be talking about vectorized simulations and search trees.

Due to the nature of the game, a strong human player who is on top of their game will always be ~= a flawless policy.

Discord is a free software that allows individuals to call their friends, some of which we may presume are stronger Pokemon players than they are.

Top-level tournament play is at relatively limited risk.

Click to collapse...

this is incredibly naive. most people who cheat in video games are already incredibly good (look at many of the famous examples of cheaters in speedrunning, for example). suggesting that no or even very few top players will use this to cheat in tournament play is just false, i'm sorry. mazar was already a great player. the convenience and availability of something like this makes it way easier to use than something like ghosting. ghosting also involves other people, and there's an inherent risk involved because the person doing the ghosting can come out with that information.

once this stuff becomes publicly available, competitive smogon will probably be ruined. i don't see how we can actually use anything to reliably detect cheating. i hope action gets taken before it gets to that point.

 

[Kau]

As these tools advance, their opinions will converge, and they'll have similar opinions of what equilibrium play is. Even elite players are not going to play inside-equilibrium-support more than ~80-90% of the time, and even when they play inside-equilibrium-support, they're not going to mix with the same frequencies as the bot suggests. I think looking at any one game and saying someone used outside assistance is never going to be feasible, but I think statistical analysis of many games of a player will be able to tell you, with high likelihood, who is consulting a bot. In endgames, my bot can often effectively solve the game, and even measure how large a blunder any off-equilibrium play is. High ladder opponents still regularly make endgame blunders, and a total lack of them is suspicious.

FWIW the same is true both in chess and poker. In all but the most outrageous cases, you cannot tell engines were used in a single chess game, but collections of games can give a statistical signature. It's even more true in poker.


I intentionally chose VGC for this particular reason. I don't think detection is as hopeless as you seem to think it is.


I'm happy to engage with you in good faith about how the community should react to the advent of these tools. I would ask that you lower the temperate a little bit with respect to statements like "In the creation of these bots, you do destroy an entire community of pretty much the entire 6v6 community". I think that statement is both inaccurate as to the outcome of the creation of these bots, and personalised/pointed towards me as an individual. I am doing what I can to help the community take the advent of these tools seriously.

People airing their legitimate concern about the integrity of the game is totally reasonable, but I'm trying to engage with the people who are looking for solutions.

Click to collapse...

Lines can still be memorised and be used without the ai present ingame, and vgc games can last as few as 6-8 turns, which is very easy for someone who has practised the branches against someone who has no idea about studying lines found by the ai. If and when this happens, I wonder how would one dictact whether this person is cheating or not. Is it cheating to memorise lines given by an ai, then play them out on ladder? How would it be any different practically if that person had the engine next to them on their phone? How could you differentiate the two? I understand the creation of these was inevitable, but I daresay you're trying to hide the destructive effects of this bot on online ladder and online tournaments behind your supposed good intentions to "improve" understanding. Perfect play in pokemon and chess work very differently, vgc only needs a few good turns to convert something into completely winning.

There's also the issue of people wrongly being accused of cheating, which would also be pretty horrible to them. I don't think there's a surefire way of telling when someone's cheating or not, these are just educated guesses. At the very least I hope what your saying is true in being able to detect cheating, but I don't think you've played vgc enough to truly understand the issue that the bot presents to online play.

 

[YoruNoxx]

"I love this game because of the intellectual challenge that it presents"

do you not realize how compatible that is with "I want to build a neural network that plays this game because of the intellectual challenge that it presents"

Click to collapse...

With all due respect, this is a ridiculous point. Playing a game because you are interested by the challenge it represents does not damage the integrity of said game, nor does it affect the enjoyment of other like-minded players. The same cannot be said for the second option.
This false equivalence is so ludicrous that it could be used to justify anything.

If it were that simple the game would have died long ago and we would not be talking about vectorized simulations and search trees.

Due to the nature of the game, a strong human player who is on top of their game will always be ~= a flawless policy.

Discord is a free software that allows individuals to call their friends, some of which we may presume are stronger Pokemon players than they are.

Top-level tournament play is at relatively limited risk.

Click to collapse...

Adding on to the above post, Top-level tournament play isn't safe at all, and any comparisons to ghosting aren't accurate. Ghosting requires the collaboration of another person, and therefore involves the risk of getting caught. How does one get caught when they are getting ghosted by AI?
As for tournament integrity, a player who is substantially better than their opponent should be able to benefit from that skill difference. If introducing an AI can systematically undermine that advantage, then the issue isn't merely that the top player might lose a few games here and there that they should have won. It directly affects the competitive value of getting better.

 

[idkup]

this is incredibly naive. most people who cheat in video games are already incredibly good (look at many of the famous examples of cheaters in speedrunning, for example). suggesting that no or even very few top players will use this to cheat in tournament play is just false, i'm sorry. mazar was already a great player. the convenience and availability of something like this makes it way easier to use than something like ghosting. ghosting also involves other people, and there's an inherent risk involved because the person doing the ghosting can come out with that information.

once this stuff becomes publicly available, competitive smogon will probably be ruined. i don't see how we can actually use anything to reliably detect cheating. i hope action gets taken before it gets to that point.

Click to collapse...

You are completely misunderstanding what I am saying.

There just isn't space for an agent to be better than a human player that is on top of their game. The game is largely solved already, the challenge is doing so under time pressure and doing so consistently.

Obviously malefactors will use it to cheat if it is available. Detection is difficult precisely because a perfect game is plausibly human.

 

[emonia]

There just isn't space for an agent to be better than a human player that is on top of their game.

Click to collapse...

how do you know this? if the game is solved then the ai is by definition it's at the very least as good as a perfect human player

regardless it's still going to be used to cheat in tournament play and will largely go undetected. any other reaction to this is complete cope