OU Suspect Testing Proposals

[X5Dragon]

A lot of competitive players and mods/admins have gathered recently on the IRC and in many places to discuss the changes they would like to see to the OU Suspect testing process, things such as speeding up the process, voting reqs, percentage of votes needed for a ban decision, etc.

These are some of the most agreed upon proposals:

1) The OU council should set clear guidelines on how to judge a pokemon for brokenness, a clear stance on what defines a pokemon, complex bans, and so on.

2) The percentage of votes needed to ban a pokemon should be 66% (two thirds) or above, a decisive majority instead of the current simple majority (50%) in place now.

3) Speeding up the testing process which we feel is too slow. We also believe discussions and subsequently suspect testing should have a fixed interval, every month or so, to decide on whether or not there is something that merits testing.

4) If the metagame has been found stable, retesting OU pokemon that have been voted uber based on metagame changes, new releases and new pokemon.

Any thoughts or suggestions are welcomed, hopefully this thread isn't stepping on any red lines or such.

 

[UltiMario]

I honestly think the biggest issue that needs to be discussed for future gens is if we should take the game as given to us when considering banning or ban based on shaping the metagame to the desire of the players.

Should we take the game as raw as possible while still attaining balance, taking whatever power creep GF throws at us, or should we decide where we want the metagame to be balanced at (I don't literally mean "Balance to DPPt OU" I mean like "Balanced so how strong stall is compared to offense is somewhere around the same as DPPt's" AS AN EXAMPLE DON'T TREAT THESE STATEMENTS LITERALLY thanks) and go from there.

I feel that question is the single most important thing for what the suspect process needs to focus on. I feel that there is a tear in interests between players for what we're banning for, and we really need to figure out which it is. AFAIK some people follow something closer to the former idea and some to the latter, and people choose to ban or not to ban for reasons that conflict with each other on principle. Figuring this out will be crucial to the future of comp IMO.

 

[Ada]

This is probably off-topic, but I'd like to see a test of Sand Rush-less Excadrill, even though it may require a complex ban. I think it's just what we need in order to break the stale meta that is OU (pretty much every team I see on ladder runs a Rapid Spin user nowadays). I think it's justified since there's always been a shortage of viable spinners - OU only has 4; 5 if you include Cloyster (lol). I think it'd make an awesome utility poke with a set like:

exca

Excadrill @ Leftovers
Trait: Mold Breaker
EVs: 252 Atk / 4 Def / 252 Spd
Adamant Nature
- Stealth Rock
- Rapid Spin
- Swords Dance
- Earthquake

No more getting rocks reflected by troll bounce users or having rapid spin blocked by ghosts (aka jellicent) even though you have a super effective move with life orb + analytic and doesn't leave you pursuit bait afterwards or walled by ferrothorn the one who set the hazards in the first place...

calc

+2 252+ Atk Excadrill Earthquake vs. 252 HP / 252+ Def Jellicent: 331-391 (81.93 - 96.78%) -- guaranteed 2HKO

LO guarantees the kill but most jellicent runs some speed, and SR + sand kills anyway. Mono ground isn't as bad as it sounds on exca because of mold breaker (so you can still hit gengar, the only other ghost in ou).

Also, being immune to both thunder wave and toxic and taking almost nothing from SR is really nice.

 

[Soul Fly]

Complex ban is a big no-no unless it's some new precedent (aldaron's proposal for example) and if you ask me i'd like to see an exca test as a whole without all the sand rush-less bs. The metagame is unrecognizable 2 years after the BW1 ban, back when we were still under the prejudiced shackles of DPPt. A lot of new checks and counters have emerged

 

[peng]

warning this is tl;dr and probably a lot of stuff that has been brought up in the past, I just haven't seen that much direct discussion on it

I'm not a fan of going back to such high supermajorities for voting. I don't understand why a pokemon that only gets 65% would stay OU when well over half the voters find it to be uber. On paper, 66% doesn't look that high but by setting that threshold you are effectively saying pro-ban votes are only worth half that of a pro-OU vote, which I don't feel is right. I agree that there needs to be something to stop 51-49 votes being passed when its basically just a 50:50 call, but I think 66:34 is too high.

I really liked the idea of paragraphs in DPP suspect and I'd like to see something similar come back, because at the moment I feel its too easy to get reqs without actually knowing the meta that well. The reason for this is that, for a long time, the ladder hasn't necessarily been an appropriate test of "skill". Its all too easy to just spam SmashPass / full BP on the ladder and getting the right to vote whilst not actually knowing much at all about the meta you are playing. Similarly, we play a partially luck-influenced game and its definitely possible for sub-par players to get reqs exclusively based on lucky streaks, or by avoiding better players by playing at less populated times of day. I think paragraphs for non-TCs (or like 5 good posts in a suspect discussion thread) just to show you know what you are talking about would go a long way to weeding out "bad" players or people who are planning on voting something uber for the "wrong" reasons (i.e. like the guy who wanted landorus gone because he doesnt like people using mamoswine)

I'm not bothered about "characteristics of broken mons" because every suspect is a different case and it just ends up as people reciting definitions without actually looking at the pokemon in practice. As long as theres some way to prevent people with very flawed arguments from voting (e.g. mamoswine guy, and the "we need to keep x otherwise we'd have to ban y and z as well!" and "not enough time" arguments)

Finally, I think everything would be easier if there was a standardised, integrated system across Smogon. Something like 3 suspect cycles a year (i.e. one every 4 months) would just make everything easier to follow and would make the system more transparent, the lack of which is a common criticism of the OU council. I think a go-to calendar for everything that happens on Smogon can only be a good thing, with everything integrated around suspect so you don't get the annoying scenarios of Pokemon being banned mid-tournament and the tournament substantially changing from one round to the next. This is the kind of thing that doesn't really work early in a gen because there are probably like 8/9 outright broken mons that we shouldnt be waiting 4 months to get rid of, but I think it would work well after the obviously broken stuff is gone. Something like:

Hide

Jan 1st - Suspect nominations
Jan 8th - Suspect Test starts
Feb 1st - Test Ends, Voting Starts
Feb 8th - Voting Ends
OST runs between Feb 8th and Jun 8th

Apr 1st - Suspect nominations
Apr 8th - Suspect Test starts
Jun 1st - Test Ends, Voting Starts
Jun 8th - Voting Ends
WCOP runs between Jun 8th and September 8th

Sep 1st - Suspect nominations
Sep 8th - Suspect Test starts
Oct 1st - Test Ends, Voting Starts
Oct 8th - Voting Ends
SPL runs between Oct 8th and Feb 8th

Again, all stuff thats likely been brought up before but some discussion points nonetheless.

 

[ShootingStarmie]

I really don't like the idea of making it 66%. 51% or over is enough. If 51% vote that is the majority, and should be enough to decide if something is broken. I don't see why the pro ban side needs more votes. Doesn't seem really fair to me.

I do however like the idea of speeding up the process.

 

[Schpoonman]

About those "characteristics", I do agree that every suspected Pokemon is different, but there are certain groups of Pokemon that we can identify as problems and set some guidelines. Excadrill, Genesect, Blaziken. These are all Pokemon that were damn-near uncontrollable sweepers. Deoxys-D and -S. Support Pokemon who you could not stop from doing their jobs, and in the case of Deoxys-S, a potential sweeper if you weren't ready for it.

 

[Soul Fly]

I really don't like the idea of making it 66%. 51% or over is enough. If 51% vote that is the majority, and should be enough to decide if something is broken. I don't see why the pro ban side needs more votes. Doesn't seem really fair to me.

No it is not. It's not about harassing the pro-ban side. It's about doing something drastic as metagame shifting with more rationale than something which is a split-even 50-50 with a lucky vote thrown in. It has to reflect purpose. The onus is already on the pro-ban side since they are the ones who want to change the meta. In fact until before BW2 we always had the supermajority system in place. There is no urgency or mandate to change the meta, so if the voting is found to be inconclusive it doesn't change. And there are a lot of plain stupid/asshole/idiotic/retarded/clueless voters, (example: http://www.smogon.com/forums/thread...orus-is-now-uber.3484919/page-13#post-4759744) who can be neutered with a supermajority system in place. Some idiot making reqs and tipping the metagame on a 50-50 vote doesn't seem fair to me.

 

[ShootingStarmie]

No it is not. It's not about harassing the pro-ban side. It's about doing something drastic as metagame shifting with more rationale than something which is a split-even 50-50 with a lucky vote thrown in. It has to reflect purpose. The onus is already on the pro-ban side since they are the ones who want to change the meta. In fact until before BW2 we always had the supermajority system in place. There is no urgency or mandate to change the meta, so if the voting is found to be inconclusive it doesn't change. And there are a lot of plain stupid/asshole/idiotic/retarded/clueless voters, (example: http://www.smogon.com/forums/thread...orus-is-now-uber.3484919/page-13#post-4759744) who can be neutered with a supermajority system in place. Some idiot making reqs and tipping the metagame on a 50-50 vote doesn't seem fair to me.

I agree some some people's reasoning for banning Landorus-I were a joke, but making the percentage of pro banners doesn't fix this. We either need to make reqs harder, or make voters give a valid reason for their choice. Also, why is it 66%? I know this is 2/3, but isn't that a bit too much if we are going to change this? If 65% vote something to be broken that isn't enough?

 

[Halcyon.]

The same could be said of the anti-ban side as well. Some idiot who loved Genesect in OU could have tipped the scale to 49-51 and kept him here. I think that changing it to a super majority would fix a symptom, but not the disease. The problem is that ladder requirements for suspect tests are too luck based. getting a high ladder ranking depends entirely on who you battle in your first 10 matches. If you face some idiot who was able to get to 17-1900 and win, it'll skyrocket your own rating to well past 2000. I think paragraph voting makes the most sense, since it means that we can weed out the people who don't know anything about the metagame. Also, placing less weight on people's ladder results and instead focus more on people's posting ability, knowledge of the metagame, etc.

 

[Soul Fly]

The same could be said of the anti-ban side as well. Some idiot who loved Genesect in OU could have tipped the scale to 49-51 and kept him here.

No it couldn't. Genesect got banned by a 67% supermajority (8 OU, 37 Uber, 1 Abstain = 67.27% Uber).
Check facts please rofl. This only strengthens my case.

Edit: I agree with your other points though, but I still feel that it would make suspect testing too 'exclusive', which isn't conducive for smogon who are actually accused of being very elitist in the first place.

 

[Halcyon.]

No, I was using Genesect as an example of a broken 'mon. I didn't mean one person would have LITERALLY tipped the scale for Genesect. red my post agin and substitute Genesect with "broken Pokémon X." Honestly, who cares if Genesect was banned with a super majority? That wasn't the point of the post and you know it.

 

[ryan]

I am mainly posting to echo the points that PenguinX made in his post.

It seems entirely illogical to push the percentage required to ban something up to 66% when it has the potential to go against what the majority of the voters wants. I understand that voter error, things like uninformed voters or biased voters, should be accounted for, but I don't think that this is the best way to go about it. If there is an issue with the voting process, it's not one related to numbers. The main causes of voter bias can all be patched up reasonably well by forcing voters to explain their vote, be it through a paragraph in the vote itself or in suspect threads. Right now, NU is suspecting Jynx, and the requirements to vote for the suspect were to play a reasonable number of battles (not a set Glicko2 or dev requirement, as we all know how flawed that is in practice when we have voters forfeiting to get their dev down at the end which doesn't really seem like something we would want to promote) in addition to making intelligent posts in the Now Playing thread and the suspect thread. This helps to show an understanding of the metagame far better than what battling on the ladder alone does.

However, I understand that this is a lot less reasonable with a tier that has a playerbase as big a OU does, so it only seems logical to force voters to explain their votes when they cast them. From that point, the OU council can look over the votes and decide if the reasoning behind someone's vote is valid enough for the vote to count. I know that this takes more work, but I do believe that it would help with the quality of suspect tests. In an ideal world, this wouldn't be a concern, but there has been a lot of controversy surrounding suspect testing as of late, likely due to how close the Landorus-I vote came to. People have been whining about it on IRC a lot lately, and there has been a lot of bandwagoning behind moving to a super-majority instead of a strict-majority. I just don't see what this accomplishes better than what requiring some ounce of logic behind a voter's reasoning does.

 

[Soul Fly]

No, I was using Genesect as an example of a broken 'mon. I didn't mean one person would have LITERALLY tipped the scale for Genesect. red my post agin and substitute Genesect with "broken Pokémon X." Honestly, who cares if Genesect was banned with a super majority? That wasn't the point of the post and you know it.

No I didn't know. And if that's your real argument then you don't have a case in the first place, since you didn't read carefully enough I guess.

The onus is already on the pro-ban side since they are the ones who want to change the meta.

It's called the Burden of Proof. It always lies on the party which is opposing established norm.
---------------------------

Edit: @Treecko
Just saying, but this system worked just fine since ADV right till the beginning of BW, no one had any issues with it, and it was accepted norm. So if there was any discrepancy of the sort you mention, it would have been cataloged.

Latios is a perfect example of a pokemon who is nowhere near getting banned at this moment while it would have been sent in the Uber tier with the current simple majority requirement. So please excuse me if I have a hard time believing that simple majority has any serious merit as a voting requirement, which doesn't demand a solid outcome in order to function..

 

[Pocket]

Well said, Soul Fly. Simple majority is too divisive to make a drastic change to the status quo. That's hardly a resounding "yes, (un)ban this shit, plz" from the community. Note that this change also includes bringing down a formally uber mon to OU as well, so it's not only affecting the pro-ban side, either. Not attaining supermajority to me appears as the community as an entity is still split on the suspect's banworthiness. I wouldn't mind a compromise between simple + super with 59% approval rate to change the status quo, either. However, anything lower is an indicator that the community need more time with the suspect to come to a more decisive decision.

Here are my thoughts on testing in XY

As for the initial testing of XY, I believe we need a quick succession of 4-6 suspect tests to clean up the initial meta (this is assuming OU council will start with a minimal banlist of 670+ BST Pokemon banned and everything else unbanned). It would pretty much be similar to the initial 5 suspect tests of BW, except for one change: Replace free-for-all nomination thread with Individual Suspect Discussion Threads. This way we can have an intelligent discussion on potential suspects rather than people simply running off lists of Pokemon they want to ban. Just like NU, this thread would go up on the 2nd or 3rd week of the suspect test, after the OU Council read through the NP thread and pick out the suspects for further discussion. Once the OU council is convinced that said threat is a suspect, qualified voters would require 1 supermajority to instantly ban it or 1 simple majority to make it an auto-suspect for the upcoming test (in this case, it will only require another simple to be banned).

The OU Council did state that SR would be one of the first test target, though. This should probably be tested before the initial clean up phase that I mentioned above, because the ability for the metagame to handle certain threats would change drastically depending on SR's fate. This would be a test that would probably require supermajority or at least 59% approval rate in order to be banned imo.

After the initial phase of intense clean up of meta, then we can return the more relaxed state of testing that we are in now, with a break period of 3-4 weeks between tests to help the meta stabilize.

 

[X5Dragon]

For what is certainly a great debate to make this metagame the best possible, I just want to point out to simple things. I don't want any of the proposals mentioned in the OP or subsequent posts to be taken literally, number for number, but rather focus on the point being made. Decisive majority instead of simple, with the number suggested (66%) subject to change. Also, the point about the council being specific could be in numerous ways, a list of criteria, general guidelines or even a simple "do not even attempt to argue for/against based on this way of thinking or else you visiting rights to this forum will be restricted to Circu Maximus forever".

 

[acheron01]

I like the idea of retesting pokemon. especially now. even if it doesnt mean anything in the short run because 5th gen is about to end.

 

[Erico9001]

What I have issue with...

2) The percentage of votes needed to ban a pokemon should be 66% (two thirds) or above, a decisive majority instead of the current simple majority (50%) in place now.

I understand that voter error, things like uninformed voters or biased voters, should be accounted for, but I don't think that this is the best way to go about it.

I'm assuming that the reason for this is to account for voter error as Treecko said. In which case I hope you can hear me out, this is a bad decision. It does not make sense to raise the difficulty to bann pokemon in trying to account for biased views, because there are biased views towards both banning pokemon and not banning pokemon. As we know, the bias towards banning pokemon comes from people who try to use teams that could benefit from one of their main checks being removed, and bias towards not banning pokemon comes from people who use the suspected pokemon or have teams that easily counter the suspected pokemon.

warning this is tl;dr and probably a lot of stuff that has been brought up in the past, I just haven't seen that much direct discussion on it

I'm not a fan of going back to such high supermajorities for voting. I don't understand why a pokemon that only gets 65% would stay OU when well over half the voters find it to be uber. On paper, 66% doesn't look that high but by setting that threshold you are effectively saying pro-ban votes are only worth half that of a pro-OU vote, which I don't feel is right. I agree that there needs to be something to stop 51-49 votes being passed when its basically just a 50:50 call, but I think 66:34 is too high.

I really liked the idea of paragraphs in DPP suspect and I'd like to see something similar come back, because at the moment I feel its too easy to get reqs without actually knowing the meta that well. The reason for this is that, for a long time, the ladder hasn't necessarily been an appropriate test of "skill". Its all too easy to just spam SmashPass / full BP on the ladder and getting the right to vote whilst not actually knowing much at all about the meta you are playing. Similarly, we play a partially luck-influenced game and its definitely possible for sub-par players to get reqs exclusively based on lucky streaks, or by avoiding better players by playing at less populated times of day. I think paragraphs for non-TCs (or like 5 good posts in a suspect discussion thread) just to show you know what you are talking about would go a long way to weeding out "bad" players or people who are planning on voting something uber for the "wrong" reasons (i.e. like the guy who wanted landorus gone because he doesnt like people using mamoswine)

I'm not bothered about "characteristics of broken mons" because every suspect is a different case and it just ends up as people reciting definitions without actually looking at the pokemon in practice. As long as theres some way to prevent people with very flawed arguments from voting (e.g. mamoswine guy, and the "we need to keep x otherwise we'd have to ban y and z as well!" and "not enough time" arguments)

Finally, I think everything would be easier if there was a standardised, integrated system across Smogon. Something like 3 suspect cycles a year (i.e. one every 4 months) would just make everything easier to follow and would make the system more transparent, the lack of which is a common criticism of the OU council. I think a go-to calendar for everything that happens on Smogon can only be a good thing, with everything integrated around suspect so you don't get the annoying scenarios of Pokemon being banned mid-tournament and the tournament substantially changing from one round to the next. This is the kind of thing that doesn't really work early in a gen because there are probably like 8/9 outright broken mons that we shouldnt be waiting 4 months to get rid of, but I think it would work well after the obviously broken stuff is gone. Something like:

Hide

Jan 1st - Suspect nominations
Jan 8th - Suspect Test starts
Feb 1st - Test Ends, Voting Starts
Feb 8th - Voting Ends
OST runs between Feb 8th and Jun 8th

Apr 1st - Suspect nominations
Apr 8th - Suspect Test starts
Jun 1st - Test Ends, Voting Starts
Jun 8th - Voting Ends
WCOP runs between Jun 8th and September 8th

Sep 1st - Suspect nominations
Sep 8th - Suspect Test starts
Oct 1st - Test Ends, Voting Starts
Oct 8th - Voting Ends
SPL runs between Oct 8th and Feb 8th

Again, all stuff thats likely been brought up before but some discussion points nonetheless.

 

[BLARG weather BLARG]

We have a complex ability ban around Sand Veil/Snow Cloak, why not just ban Sand Rush altogether and bring us Excadrill back? Why should Garchomp be arbitrarily be brought back from Ubers but not Excadrill? Sure, we lose out on Sand Rush Stoutland, just as we lose out on Sand Veil Gliscor, neither of which was broken with their abilities, but in turn, Garchomp flourished in OU and brought a healthy balance to the metagame. Likewise, Excadrill can do the same in a metagame plagued with only Starmie as an offensive spinner.

 

[Agent Gibbs]

There was no complex ban involving Sand Veil and Snow Cloak. They were just banned, just like Moody was. In addition, there were many people that complained about Sand Veil actually being broken on Gliscor since it could take advantage of a Sand Veil miss just like Garchomp (albeit to a lesser extent) and get off a free Substitute, SD boost, or attack. I've even heard stories of Cacturne being able to take advantage of those misses. Besides, there were also the arguments that Sand Veil and Snow Cloak brought another unnecessary luck element to the battles involving them and that they should be banned as part of the Evasion Clause. Sand Rush doesn't have that same problem. Only one of the three users was anywhere near broken, and unlike Sand Veil/Snow Cloak, there is not a luck element to Sand Rush that can allow Stoutland and Sandslash to hax their way to a kill or a win.

 

[vyomov]

I like the idea of individual suspect threads which the OU council evaluate to see the quality of arguments before deciding whether to suspect or not.

 

[Sturdynips]

We have a complex ability ban around Sand Veil/Snow Cloak, why not just ban Sand Rush altogether and bring us Excadrill back? Why should Garchomp be arbitrarily be brought back from Ubers but not Excadrill? Sure, we lose out on Sand Rush Stoutland, just as we lose out on Sand Veil Gliscor, neither of which was broken with their abilities, but in turn, Garchomp flourished in OU and brought a healthy balance to the metagame. Likewise, Excadrill can do the same in a metagame plagued with only Starmie as an offensive spinner.

The sand veil/snow cloak ban is not a complex ban. The abilities themselves were deemed broken, and as a result garchomp was not "arbitrarily" brought back. Sand rush is far from broken as an ability, only excadrill was.
edit: ninja

That said, i agree that a 2/3 majority to ban is unnecessary and potentially unjust. The issue of idiots influencing the vote can be argued no matter what the majority requirement is, someone can bump the vote from 65% to 66% as easily as from 49% to 50%, and in reality in an online vote there's no way to determine who is trolling and who isn't. The requirements to vote are the only way to control it right now, because realistically there is no other way to stop someone from voting the opposite to what they think is right. However, I would like to see a list of criteria set out by the OU council with each suspect test, basically listing the factors that voters need to consider in a clear and concise way. This might be a way to cut down on uninformed or biased voting.

 

[TheFourthChaser]

I see you removed the fifth point, good.

1) Yes it would be nice to have clear guidelines but they're kind of difficult to form. We should just use Doug's Characteristics.

2)I may be a bit biased with this one but I still disagree, a simple majority is fine in the current system. Supermajorities existed when we had the community vote on suspects and they were a good check to prevent things from being banned by a bandwagon but now that a council decides what the suspects are this isn't as large a problem. Council existed for some time in BW and I never felt results were swayed by a bandwagon. Our metagame makes this harder for me to judge since we have needed bans for BW2's entirety.

3) This is easy to agree on but just how long does a metagame need to settle? There is no concrete amount of time and should really be judged on a case by case basis.

4) I feel this already exists and if you think it doesn't, we haven't had a balanced meta for quite a long time now.

I like Pocket's idea of having an NU-style system where an individual suspect receives its own thread but I don't expect them to be much different from the NP threads. I would prefer having a system of poster approval (for known competitive players and those who posted well in the NP threads) for these kinds of threads.

 

[MCBarrett]

I think that if we were to adopt a "Paragraph Reqs" system it would clear up a ton of problems with the current voting populous, since as of right now, we cannot tell if the voters truly know (or care) about the state of the OU metagame. I think this system would be extremely beneficial for many reasons, since a voter will be forced to think about their decision and showcase logical reasoning for said decision, they will be that much more likely to make the correct decision. Also, it will promote more quality posting activity in the Suspect Thread and it may get some lurkers to come out of their shell and post in meaningful discussion to showcase their knowledge of the metagame. While it may make Smogon seem even more elitist it will prevent people who just simply do not care about the metagame, and only vote for their own personal agenda, from voting. Which, of course, is much more important than seeming welcoming to the guy who got lucky and made ladder reqs, and subsequently made a careless vote that shifted the outcome of the poll.

 

[PDC]

I definitely like the idea of paragraphs like we did for the OU Council on Salamence way back, and to be honest that would mean it would make voters really have to think about what they are voting and choosing to shape the metagame. It could be done in a way where an insufficient paragraph would made the vote not count or so, so that we can really get an incite in what the voter means, and how skillful and knowledgeable they are in the OU metagame.