MyPokemonRBad12 said:Again, is this something that we really want to devolve into? We are going to start tracking speech patterns on discord, we are going to track how close they play to the equilibrium, we are going to do all this instead of just not developing bots? If we don't make bots public, then it comes down to the fact that most people do not want to put in hours of effort to make, playtest, and use a bot to win tournaments only to gain false respect without monetary gain or anything of the kind. The reality is that not many people will want to keep up such a facade, and it eventually will get tiring. This is also why we have only had one Bloo in the history of Smogon. However, easy-to-access bots means that you have to put in far less effort but you still win all the big tournaments, and the tracking team has to put in 100s of hours of work to find out that you were maybe not even a cheater to begin with. Are we going to seriously ask this of unpaid people instead of tackling the issue at its core?
Let's look at it this way. If Stockfish wasn't public and nobody knew engines existed, people could in theory develop their own engines, make them play better, and use them at the Candidates and then Worlds to win. Is this very likely, considering that the whole thing is "fake"? No. But if the Candidates is hosted online and people have direct access to the engine, is it more likely that they use it to win? Yes.
None of us have an issue with how Smogon works at the present, we need to stop using arguments like "we can catch cheaters by doing XYZ and putting in a lot of time and effort and observing the people overtime" or "well people could be cheating right now" in order to snowball the bots and make it so that everyone has the best move at their fingertips.
this is a misunderstanding of how Stockfish was developed.
Stockfish uses hand-picked heuristics created by GMs who happen to also understand machine learning and a very small trained NNUE. Almost invariably, the Pokemon projects are 7-digit param transformers. (I have observed that blending value with certain heuristics results in a stronger policy, but that's a conversation for another day) If Stockfish weren't public, developing an analogue would be very difficult. Chess is deterministic, you can play millions of games against Magnus Carlsen and never win, as he always has perfect information.
Developing an 'adequate' Pokemon policy (one that can potentially beat top players), in my estimation currently requires the following:
- A basic familiarity with machine learning and neural networks (you should know what a transformer is, and what the LLM is talking about when it is talking about loss)
- A basic familiarity with competitive Pokemon (you should know when your engine is incorrect / your policy is doing something objectively wrong)
- $20 USD to either Anthropic, OpenAI, Moonshot, Deepseek...
- A working computer
None of these are nearly as rare as actually being good at chess.
There is no barrier. Someone will do it. Someone is almost certainly already doing it. As much as I would prefer tools not be released "because someone will do it eventually", we cannot be putting our heads in the sand. Just because it's not a good premise to open the floodgates completely doesn't make it any less true.