Proposal Suspect Test Qualification Reform

[Rigas]

This feels like we’re chasing solutions for a problem that doesn’t exist.

...

Every time this topic comes up, no one can ever really point to a good reason why we should raise/segregate requirements, other than “well I think so”.

I agree with these statements.

To address another thing in the OP and in some of the replies: those who believe 1750 elo + 80% GXE is "too easy" have a skewed perspective, likely as a result of engaging primarily with the top echelon of Pokemon Showdown players (such as active Smogon tournament players) and not fully grasping that there's a vast, vast playerbase below. I'm not saying this as a condemnation; it's understandable that it happens. But it needs to be kept in mind. I don't see any cause for concern that "unqualified voters" will create trouble. To be clear, I'm not saying that suspect reqs should be achieveable by every single one of the tens of thousands of people on the ladder; I'm saying that someone who reaches 1750 elo + 80% GXE is qualified to have a vote about the metagame.

Plus, while I don't agree with the entirety of the comment an excerpt from which I'm about to quote, I believe this passage is strong:

If someone can consistently hold a high enough elo to be well outside of low ladder, they should have some kind of voice in the tier's rules. You don't have to be a top player to have a stake in the game. Frankly, someone playing at that level has a decent chance of beating a top player in any given game, anyway. Pokemon is swingy like that.

As such, I am not convinced that anyone who can achieve 1750 elo + 80% GXE (I believe 80% GXE is the harder part) should get anything less than a "full" vote, which is what the OP is proposing with the tiered system.

 

[Tree69420]

To address another thing in the OP and in some of the replies: those who believe 1750 elo + 80% GXE is "too easy" have a skewed perspective, likely as a result of engaging primarily with the top echelon of Pokemon Showdown players (such as active Smogon tournament players) and not fully grasping that there's a vast, vast playerbase below. I'm not saying this as a condemnation; it's understandable that it happens. But it needs to be kept in mind. I don't see any cause for concern that "unqualified voters" will create trouble. To be clear, I'm not saying that suspect reqs should be achieveable by every single one of the tens of thousands of people on the ladder; I'm saying that someone who reaches 1750 elo + 80% GXE is qualified to have a vote about the metagame.

Even though someone who can reach 1750 + 80% GXE may be in the top 0.1% of all players who play, doesn't make them necessarily qualified to vote on the metagame. For example, someone who goes 1-2 at a major in smash is probably in the top 1% and would easily beat the vast majority of players, but they still don't understand the metagame in comparison to professional players who can consistently place top 64, and they clearly shouldn't have the same level of authority to vote.

However, I believe the current status quo is ultimately the best option, as increasing the difficulty more would hurt the time-sink more than it would filter out unqualified players imo. Giving reqs to top tournament players would be positive but not that impactful of a change.

Different tiers of votes doesn't feel like a good change to me because it just rewards people who have time to nolife ladder, and tournament reqs having more weight seems also unfair to people who simply didn't have the opportunity to get these reqs.

 

[Blizihguh]

I'm going to pitch the OP's proposal in a slightly different way. I hope this will make more sense to people.

We set reqs with the idea that, if someone does well consistently on ladder, they must understand the metagame. But what consititutes "enough understanding" is subjective! And for for that matter, so is "what should be banned." Any hard cutoff you set is a value judgment, and it's imprecise. Saying "people under this line don't understand the metagame, people over this line do" is going to get some people wrong in either direction. It's also much much simpler and easier to stomach than trying to decide subjectively, which is why we do it. But the OP's proposal softens the cutoff somewhat, which has benefits for people on both sides of the issue.

Many people think that reqs are too easy, over-representing low-level players. Likewise, many people think reqs are too hard, over-representing the players who are most willing to grind for them. By removing the hard cutoff between "no vote" and "vote," we can alleviate both of those concerns. We can give one vote to people who play the game basically competently, by setting the lowest level of reqs lower than they're normally set. For players who play the game very well (say, to a "normal" level of reqs), we give two votes. For players who play extremely well (much better than typical reqs), we give three votes. For players who don't want to grind, it's much easier to stomach "I only get 1 vote" than "I don't get any vote." You can recognize that you're good enough at the game to have a say, but not dedicated enough to grind for 3 votes. On the other hand, for players who think reqs are too easy, the best players' voices now count more. How can you complain about bad players ruining the vote when good players count 2-3 times as much?

If there's not a big discrepancy between how the different tiers vote, it doesn't make a difference to the outcome, but it gives everybody a way to feel better about it. If there is a big discrepancy, then it might affect the outcome, if there are many more voters in the lower brackets -- but this already happens in our current system sometimes (see FayaWizard's post). This is the most contentious point, but I think this is a fair outcome. If the people who play a game overwhelmingly feel that something should go, then it should probably go, even if the top players think it's manageable.

We're playing this game for fun, not for any kind of real stakes. What really matters with these votes is investment. Someone who's more invested in the metagame should have their voice matter more than someone who doesn't care that much. But if a lot of players are invested, and they overwhelmingly feel a certain way -- why should a few players who care an inordinate amount get to overrule them? The goal of suspects is to make the game the best it can be, not to make it as satisfying as possible to the top 0.1% of players. A game that's better to the top 0.1% of players, and worse for 99.9% of people, is not the best it can be.

I have some thoughts about other posts but I'm just gonna omit them, I feel like this says what I want to say anyway. I genuinely think the main issue with this proposal is that we would probably have to rework TC, but I'm nowhere near qualified to figure out how that should work.

 

[Tree69420]

We're playing this game for fun, not for any kind of real stakes. What really matters with these votes is investment. Someone who's more invested in the metagame should have their voice matter more than someone who doesn't care that much. But if a lot of players are invested, and they overwhelmingly feel a certain way -- why should a few players who care an inordinate amount get to overrule them? The goal of suspects is to make the game the best it can be, not to make it as satisfying as possible to the top 0.1% of players. A game that's better to the top 0.1% of players, and worse for 99.9% of people, is not the best it can be.

I have some thoughts about other posts but I'm just gonna omit them, I feel like this says what I want to say anyway. I genuinely think the main issue with this proposal is that we would probably have to rework TC, but I'm nowhere near qualified to figure out how that should work.

I'm not necessarily against the idea that players who are less informed about/skilled in the metagame should not have an ability to vote, however I view it like this. Imagine you are a governmental body trying to work towards solving the issue of climate change. The vast majority of people, including regular citizens as well as climate scientists, have the same end goal, that being solving or reversing climate change. However, not everyone should have the same ability to vote regarding this. An ordinary citizen, though well meaning, may be very vocal about ineffective solutions peddled by the oil and gas industries, such as the idea that every person must be personally responsible regarding their carbon footprint. Asking climate scientists, even less dedicated ones, will likely result in a more effective solution to the problem than asking ordinary people. In this way, voting requirements are akin to a test you must take, with say 100 questions, to prove that you understand the problem. However, it is possible to pass this test despite not understanding the root of the problem, just by memorizing answers. By increasing the test length, it will be more effective at weeding out people who are less informed, but at the cost of many people who are well-informed but cannot sit through such a tedious test.

Similarly, the top players may have differing opinions on exact solutions to improving the meta, but they are much more likely to be able to diagnose the problem and present a solution than the average player. I heavily disagree with the idea that a metagame can be great for the top 0.01% but bad for the other 99.99%, as a good metagame, with diversity in sets, pokemon, and team styles, will be a good and healthy metagame regardless of how good you are at the game. A 1500s player may be well-meaning, but ultimately incapable of analyzing the metagame and improving it, which is why we only open votes to the top percentage of players who can achieve voting requirements. Brushing off metagame concerns as "just for fun" misunderstands the end goal of providing a healthy, balanced, and fun metagame for all players, regardless of skill level.

If we had a real way to figure out how much skill/understanding someone had of the game, then different tiers of voting could possibly work, but as increasing the difficulty of tests much more so increases the tediousness, this solution is not very effective in my opinion. Solutions such as giving each user only 1 account could potentially solve this issue.

 

[Ren]

idk how to say this and im largely retired now but i heavily dislike any proposal that says "people can vote but some people's votes matter more than others". all votes should be equal if people care enough to earn the right to cast one and discussion + a bigtime player's opinion is enough additional sway, use your words to change minds and the outcomes will follow. or they wont and thats still fine because thats how group decision making works. no opinion on any proposal that changes or adds ways for people to earn a vote, suspect laddering is bad for life havers

 

[FlamPoke]

I think the OP by Nat has good intent on making the Smogon tiering process more effective. There are also some good dissenting opinions about not wanting to make suspects more grindy in nature, which I can respect, but I think the general premise of improving the tiering process is a good one and the way to move forward with that seems pretty clear to me.

Providing the casual ladder player an opportunity to impact Smogon tiering via ladder reqs was a good decision overall, but not having a mechanism to emphasize the less-casual tournament player's opinion is doing a disservice to the site, I believe. I won't get into the actual requirements of the suspect in depth as that isn't relevant to this discussion, though I will just voice my opinion that ladder reqs are probably too easy across the site as a whole. What is more important to discuss here is that tournament players should have more influence on tiering by virtue of being involved in the tier at a higher level than the casual player.

I also won't touch on why surveys are probably more bad than good... but the surveys do split opinions into two groups: a general player base and a qualified player base. The fact that councils separate the two is proof of the belief that people who play tournaments are likely more inclined to understand the tier at a nuanced level, which should be a concept most people can at least understand if not align with. Placing an emphasis on their opinions to impact tiering decisions, such as what mons are worth suspecting in the first place, but not giving any sort of respect to their knowledge without requiring them to also get ladder reqs manually is redundant.

I think providing reqs to players who had good showings in recent tournaments is a direct improvement to the tiering process and does very little to hinder the casual player from still having input into the suspect. The list by Greybaum showing how few tournament players actually participate in suspect laddering should prove beyond a reasonable doubt that some of the most involved, influential players aren't really making a difference on the tier because it's just a redundant task to ladder for a tier you already have to be super involved with for the tournament scene.

And lastly, the point Nat made in the OP; having multiple tiers of qualification for ladder reqs. It still requires the grind but incentivizes people willing to slave away on ladder to go further and get more out of the suspect process by demonstrating their capabilities with a higher elo/gxe peak. I think this is an interesting approach to providing more opportunities for people who are good to influence tiering, but have no clue how to define how many tiers / what elo/gxe gaps need to be / etc.

tldr
Should reqs be harder? In my opinion, yes.
Should tournament players be given reqs based on being "qualified"? In my opinion, yes, but I don't really know how to quantify "qualified" besides w/l or placement most likely (like surveys do).
Is Nat's initial suggestion of having tier-based ladder reqs worth pursuing? Maybe, as it does offer a little more proof of your capabilities at the cost of having to grind ladder harder which is going to still make the tiering process in favor of those with time > skill, though it also requires a hard skill check that current reqs kinda don't. I'd want two tiers personally, maybe like 1750/80 and 1900/85.

 

[Giagantic]

This isn't some official response but...

Reqs should be left alone, you can fiddle with the Coil all you want but the process of getting a vote is perfectly fine. If you can't be bothered to go through with this process you simply don't have the right to vote. No one should be able to avoid this, you ladder or win a suspect tour to earn the requirements to vote. If you can't be bothered to go get the reqs, too bad, you clearly didn't care enough to put in a bit of effort.

Skipping going through this process is absolutely a terrible idea. It creates a clique centred around whatever metric you use to get said status.

Furthermore there is no good argument from my perspective in which you can justify one vote having more weight then another, this is simply antithetical to a democratic voting system. Every vote is equal is a fundamental aspect to this system. Doesn't matter if you are the greatest player to ever walk this earth, or the worst, if you went and got the reqs, then your vote is the same.