I know this is probably a touchy subject, especially because suspect testing is such an important part of Smogon and Smogon culture, but I feel like it's time to take a second look at the process and see if we can improve it. The reason I think now is a good time is because all the singles metagames are as stable as they've been since the beginning of ORAS, and if we can figure out a way to improve the system now, it will pave the way for better suspect results leading into Gen VII (also this thread in IS started drifting towards this topic and I thought it would be a good idea to bring it up in here as well).
I think the main problem that people have with the current suspect testing process is that people feel like some voters in suspect tests, frankly aren't qualified enough to make an educated vote towards the future of a tier. Many people (myself included) feel that just "getting reqs" on the ladder, doesn't give a player who has had little to no experience in ___ tier, enough information in order to make an informed decision about the suspect at hand. This is also amplified when you have a test similar to the Victini re-test, where the metagame during the suspect test is different from the normal UU metagame, as an uninformed voter will have no idea whether the re-introduction of a Victini, or the lack of a Mega Pidgeot has improved the tier or made it worse. Ideally, a player will be well-versed in the conversation taking place on the forums as well as having made reqs, but in reality, a large number of the voters have little to no forum presence, and base their opinion solely on their laddering experience. To give an example, in the last UU suspect test, 33 of the 97 voters had under 50 posts on the forums, and if you bump it up to less than 100 posts, 48 of the 97 votes equaled that number. That equates to about 33 and 50 percent of voters, which is an astonishingly high number.
In my opinion, an ideal suspecting process would combine both of the following aspects:
Again, I feel like now is probably the best time to try and work out a solution to improving suspect testing, since it gives us enough time to properly discuss and sort everything any issues people may have so that we can be ready for the impending arrival of Gen VII with a stronger suspecting system than we had before.
I think the main problem that people have with the current suspect testing process is that people feel like some voters in suspect tests, frankly aren't qualified enough to make an educated vote towards the future of a tier. Many people (myself included) feel that just "getting reqs" on the ladder, doesn't give a player who has had little to no experience in ___ tier, enough information in order to make an informed decision about the suspect at hand. This is also amplified when you have a test similar to the Victini re-test, where the metagame during the suspect test is different from the normal UU metagame, as an uninformed voter will have no idea whether the re-introduction of a Victini, or the lack of a Mega Pidgeot has improved the tier or made it worse. Ideally, a player will be well-versed in the conversation taking place on the forums as well as having made reqs, but in reality, a large number of the voters have little to no forum presence, and base their opinion solely on their laddering experience. To give an example, in the last UU suspect test, 33 of the 97 voters had under 50 posts on the forums, and if you bump it up to less than 100 posts, 48 of the 97 votes equaled that number. That equates to about 33 and 50 percent of voters, which is an astonishingly high number.
In my opinion, an ideal suspecting process would combine both of the following aspects:
- All the voters in a suspect test know enough about a tier in order to not just have an opinion about whether to ban or keep a pokemon, but to be able to back up that opinion with knowledge of how suspect X affects the tier
- Everyone should feel like they are able to participate in a suspect test, and feel like they have an opportunity to voice their opinion, regardless of their standing in said tier. (this is one of the great things about smogon that we have to make sure is always applicable to any suspect test)
Again, I feel like now is probably the best time to try and work out a solution to improving suspect testing, since it gives us enough time to properly discuss and sort everything any issues people may have so that we can be ready for the impending arrival of Gen VII with a stronger suspecting system than we had before.