Ubers suspect testing aftermath

[ApplepieFTW]

why is the decision being reversed when the anti-ban votes were rejected for flawed reasoning?

did 1. chaos disagree with the reasoning
or 2. is something else going on, because this shit makes no sense.

 

[steve_man]

why is the decision being reversed when the anti-ban votes were rejected for flawed reasoning?

did 1. chaos disagree with the reasoning
or 2. is something else going on, because this shit makes no sense.

Voters should have never had to provide reasoning in the first place. Every other suspect test I've seen, you just say "Ban" or "No Ban."

 

[Celticpride]

Yeah but they rejected 43 anti-ban votes and only 18 ban votes. The most BULLSHIT thing about this suspect test was the paragraph / reasoning part. Every other suspect test just has you put your vote in, no paragraph necessary. I asked if they could provide some guidelines as to what would be accepted / rejected, but was ignored. The tier leaders being able to reject your vote for reasons you don't know until AFTER you voted made this whole thing rigged. I'm glad the community as a whole voted for anti-ban (before rejections) but the fact that there were 61 rejections all together means that the tier leaders have failed in providing information to the general public in order to manipulate their own agenda.

I'm not sure it's that black and white. Paragraphs make things very subjective, but it can also be a filter against blind stupidity. Visit some non-Smogon Pokemon forums after a suspect is over (I'm thinking of one in particular...) and you'll see some people have terrible logic. A very simple example that I just made up is "I voted no ban for Mega Mawile because I can use Fire and Ground attacks." If the reqs are too low, you can get some real shitheads voting. Even if they regurgitate a decent anti-ban argument nearly word for word at least it makes them stop and think for half a second. So submitting paragraphs isn't entirely bad.

 

[ApplepieFTW]

Voters should have never had to provide reasoning in the first place. Every other suspect test I've seen, you just say "Ban" or "No Ban."

Yes they should have. If they have no idea you're not supposed to ban something from ubers because it is broken, then their vote will be fucked up. Maybe they think : "stag is broken, it should get banned. it isnt uncompetitive, but that doesnt matter anyway!". Yeah, thats not what we want. These votes should not get accepted because of the ubers policy on banning stuff.

 

[shrang]

- Either pick objective criteria and count all votes that meet the criteria, or make it a council vote.

Here's something Chaos said earlier. Obviously, the criteria was not exactly objective. That's why there was such a problem.

 

[Jibaku]

As the third member of the paragraph review, I would like to share some of my thoughts on this test, past tests, and the bright future of Ubers that awaits us.

A little preface

Now Gengarite was not the first suspect test ran in Ubers (well...it's the first in this gen, but that's not relevant). Me and ibo ran some suspect tests which most notably included Evasion and OHKO moves last gen, in hopes of accomplishing a few goals.

1) The community really liked participating in suspects at the time and we chose to satisfy that hunger because Ubers was going through a somewhat barren period. Having more people play our tier is always a great goal to have.

2) We wanted to establish a message: Ubers is the tier that bans the absolute least. This was meant to highlight what's unique about the Uber tier and separate it from how every other tier functioned. The suspect tests were geared towards UNbanning things and ensuring that everything that remained banned has good reason to stay there. From this, Evasion was deemed manageable and was thus unbanned.

Thoughts on the Shadow Tag Suspect Test

Somewhere down the line, I felt that our intent was twisted. Instead of sending a message about minimizing bans, I think our Gen 5 Suspect Tests were used to justify the classic Suspect Tests for Ubers. This made me really uncomfortable, but I did not attempt to stop this as I was more focused on LoL at the time and felt that my intervention would be overstepping my bounds.

It wasn't until this particular Suspect Test that things got really messy though. Dice informed me that the Shadow Tag thread has degenerated to Melee Mewtwo and shrang battering each other over and over again, and I gave serious consideration about destroying it, but instead I chose to tell MM2 about this so maybe he and shrang can cool things down since I did not want to override another mod's decision. Fast forward and it still continued to happen, which made me realize that perhaps a suspect test wasn't the right approach. This is when I began to express my opinion to MM2 that I'm beginning to feel really uncomfortable about this, and we went through some philosophical mumbo jumbo and ended with hypotheticals. I then tried to join the tag thread (if you can't beat them, join them, right? WRONG!), and I found out how easily I can get caught up in it, but reeling back to my previous stance, I backed off.

Now I'd like to remind everyone that I only signed up for paragraph review. No offense to Fireburn and MM2 (they have been excellent partners to work with and I'd like to work with them again in the future), but anyone who has paid any attention to the thread knows that they are very biased against Tag (especially MM2), and I felt uncomfortable with that so I volunteered to help out because I have a neutral stance. In the end, the results still looked incredibly biased but I did whatever I could to absolutely minimize bias. Collectively, we felt that our review was overall just, but it was obvious from a numbers perspective and some background knowledge that it did not seem fair at all. I can understand that, because I too felt uncomfortable with the numbers before the results were released, although I decided then that the numbers were only an illusion. I foresaw some backlash which prompted me to tell MM2 and Fireburn to be more transparent with this test moreso than any other test, but we poorly executed it and it only gave even more suspicions of bias, and the backlash was far bigger than what I imagined.

For that, I sincerely apologize for the outrages it has caused. I felt that I had the power to stop some of these from happening, but I was too late and the suspect test broke the gates of hell. Perhaps I was just too weak.

The bright future ahead

During this suspect test, theorymonning over ORAS stuff was how I entertained myself against all the shadow tag back and forths. Now with the game out only in a few days, I hope people will put aside this incident and focus on making the best of what ORAS offers us. I'll be trying to do the same, and the Uber metagame is also due for a shakeup.

The destructive PR incident also opened up a permanent reminder to consider and may be of significant help when we revise our Ubers policy. I look forward to finding out how this episode ends, but for now, I just want to enjoy ORAS



Happy Groudoning

I'm not sure how I feel about not suspecting anything in Ubers anymore... *cough* evasion

If this is a big enough issue this will likely just be banned. But first we'll need to sort out our philosophy which will have to wait until sometime after ORAS' release imo.

 

[Cirdec]

I'm not sure it's that black and white. Paragraphs make things very subjective, but it can also be a filter against blind stupidity. Visit some non-Smogon Pokemon forums after a suspect is over (I'm thinking of one in particular...) and you'll see some people have terrible logic. A very simple example that I just made up is "I voted no ban for Mega Mawile because I can use Fire and Ground attacks." If the reqs are too low, you can get some real shitheads voting. Even if they regurgitate a decent anti-ban argument nearly word for word at least it makes them stop and think for half a second. So submitting paragraphs isn't entirely bad.

Then make the reqs higher instead of asking a paragraph.

 

[Halcyon.]

I'm sorry but I just don't undertand this this whole situation. It seems to me that Fireburn and the other tier leaders decided to suspect test Shadow Tag, required reasoning via paragraphs (btw steve_man this is an old practice that goes waaay back and is certainly not unheard of). Then, when the vote ended, they rejected arguments they found to be lacking, which resulted in about half the votes being stricken, which lead to a ban of Shadow Tag despite the community as a whole voting otherwise. Now I see many people saying that a suspect test should never have happened at all and that it should have just been left up to the tier leaders to decide, and people chalking this up as a "mistake" by the tier leaders as a whole.

If that's what happened then I honestly don't understand why the decision is being reversed at all. In my opinion, it's pretty lame to hold a suspect test and then reject more than double the anti ban votes as ban votes. To call that a "mistake" seems pretty odd and honestly offensive. It wasn't a mistake, it was a decision made by the people who lead the tier to operate a suspect test they ran in a specific way. You can call it "rigging" or you can call it following the formula they set forth, but it was certainly not a mistake. But that's not even what I don't understand. Why are people now saying that tiering decisions should be decided by tier leaders only if the decision has been reversed? The way I see it, the tier leaders DID make a decision, and have every right to operate the suspect test however they want to. Even if they did "rig" the test (which I don't think they did), how is that a bad thing compared to the tier leaders simply making the decision on their own? Furthermore, if a suspect test is seen as an inferior method of tiering for Ubers, why is the decision being changed by Chaos instead of just being put to a vote by tier leaders? In the end, the result will be the same.

So Chaos' intervention means that he and others must see this as a willful tampering of the votes, in which case it's not a "mistake" on the TLs' part, rather it's an attempt to change the outcome of a vote to meet their desires. If that is not the case, I see no reason to change the decision. You can't beat around the bush in this case and say the TLs are new and prone to mistakes and then reverse the decision they made unless they did something wrong, in which case it's no longer a mistake. And regardless, if you would rather that they never put it to a suspect test in the first place, then the outcome would have been one where Shadow Tag was banned regardless. And it's not like there aren't tiers where bans are made solely on the whims of the council (see: UU). Is there anything stopping this same group of leaders from coming to a decision amongst themselves and declaring a ban of Shadow Tag by council decision tomorrow? And if there isn't, how is that any different from what occurred in this test? I just don't see what was done wrong here given what has been said and what I understand about the situation.

These are all genuine questions, by the way. I hope I didn't come across as rude/accusatory, because that was definitely not my intention. I really don't understand what happened and would love some clarification.

 

[Minority]

To those that say the rejecting of paragraphs was extremely stupid, the original purpose behind the vote filtering should have been to prevent a Ubers community decision from being decided by the masses of players who don't main in this tier. By reviewing paragraphs the idea was that only people informed enough to vote (those that regularly play Ubers) would have their votes counted. It was a good idea to try and moderate the vote somehow, but the acceptance of votes based upon placing value on an opinion can become messy fast. I feel like I would have been more comfortable with an objective and standardized way to determine who is qualified to vote.

As for the future of Ubers tier policy, there is no reason not to suspect things in Ubers so long as the community feels strongly about it. However, considering the nature of the Ubers tier being the tier with the least amount of bans, it should therefore be difficult to ban anything. That is not to say that banning something should be as difficult as it is for an Amish community to unban a technology (for those not familiar with Amish culture, it takes virtually everyone in an Amish community to agree to unban a technology) but it certainly should be more difficult to ban something than a three vote majority.

ORAS is practically here and the tier is going under major changes, in both the metagame itself and the community. ADV Ubers will have PS! support when Showdown is up and running, and two official Ubers tours will be starting within a week.

 

[Disaster Area]

It's not sure it's nessecarily implied that it's a wilful tampering of votes, however chaos has done nothing more than declaring it has highly controversial. It'll be interesting to see how the thread deteriorates proceeds, and more quality posting like that of Halcyon. above me (just an example, but this post is I guess a response to it so whatever) emerges.

Also I feel personally that the voting removal was fine (I know there's some great players who didn't get their vote through, but the arguements which for either side weren't permitted seem reasonable), I've seen some players who I feel from having witnessed them play don't even understand the fundamentals of competitive battling have gotten their posts through. As a note I feel this thread could turn as sour as other suspect related threads in ubers has before, I'm hoping I'm not affected by it if it happens.

(less relevant but was sparked by Minority Suspect mentioning ADV Uber support on PS; my server already supports that as well as RBY and GSC (and possibly soon RBY 251 - RBY Mechanics and Moves, but GSC Pokemon usable too)) (if this is deemed irrelevant edit it out rather than delete the post please c:)

 

[Karxrida]

Removal of votes based off of subjective reasoning is completely unethical because you are esstentially giving control of the outcome to the people counting them. I'm not exactly an expert on voting laws but I assume it's highly illegal to ignore/throw out votes from qualified voters just because you believe their reasoning is "flawed". If someone got reqs to vote then their vote should count, and if you want to cull bad players either make reqs harder to get or figure out a way to weed out people who aren't part of the Ubers community.

I may not be a part of the Ubers community, but the outcome of this test makes me hesitant to join because the people in charge have abused their powers to a degree and I honestly can't trust them to be responsible anymore.

 

[trillyntruly]

why is the decision being reversed when the anti-ban votes were rejected for flawed reasoning?

to be fair, after reading the vote thread, i wondered how flawed the reasoning was. not because i have faith in strangers' words that i've never read. but because there were a few (not many, but i'm pretty sure i counted more than 1) proban votes that were awful. one i can recall didn't even have an argument. obviously i'm not in a position to make that call (and i am not accusing fireburn and mm2 of rigging the test or anything stupid like that), but as a voter, i felt as though the effort i put into thinking about the topic, and to coming to the best, most objective answer i could could have been (and should have been) mirrored by all accepted pro-ban and no-ban votes. but again, i'm not in a position wherein i'm allowed to comment on which votes should have been accepted and which votes should not have. i'll even accept the strong possibility that my bias made me be more forgiving of non-ban votes with flawed reasoning. but i don't think so. i guess i was just a bit disappointed to see that some of the counted votes for pro-ban, in my personal opinion, had very flawed reasoning, yet were accepted.

 

[ZoroDark]

Rather than focusing on "who are we gonna blame for this" (some people are guilty of this), the main priority should be "how can we prevent this from happening again?". I'm sure staff members are putting a lot of thought into this. An option that hasn't been yet talked about (at least in this thread) is introducing the requirement of a supermajority when tallying the votes. This has been done before in OU and it's the most fitting way of deciding on something controversial. Ubers is still defined as the tier with the least amount of bans, so adding a ban to that short list should always be viewed as something radical and controversial.

I guess deciding which supermajority (60:40 , 66:33 , …) would also cause some fierce discussions but in the end it should still work better than the way things were handled this time around.

 

[ApplepieFTW]

I'm sorry but I just don't undertand this this whole situation. It seems to me that Fireburn and the other tier leaders decided to suspect test Shadow Tag, required reasoning via paragraphs (btw steve_man this is an old practice that goes waaay back and is certainly not unheard of). Then, when the vote ended, they rejected arguments they found to be lacking, which resulted in about half the votes being stricken, which lead to a ban of Shadow Tag despite the community as a whole voting otherwise. Now I see many people saying that a suspect test should never have happened at all and that it should have just been left up to the tier leaders to decide, and people chalking this up as a "mistake" by the tier leaders as a whole.

If that's what happened then I honestly don't understand why the decision is being reversed at all. In my opinion, it's pretty lame to hold a suspect test and then reject more than double the anti ban votes as ban votes. To call that a "mistake" seems pretty odd and honestly offensive. It wasn't a mistake, it was a decision made by the people who lead the tier to operate a suspect test they ran in a specific way. You can call it "rigging" or you can call it following the formula they set forth, but it was certainly not a mistake. But that's not even what I don't understand. Why are people now saying that tiering decisions should be decided by tier leaders only if the decision has been reversed? The way I see it, the tier leaders DID make a decision, and have every right to operate the suspect test however they want to. Even if they did "rig" the test (which I don't think they did), how is that a bad thing compared to the tier leaders simply making the decision on their own? Furthermore, if a suspect test is seen as an inferior method of tiering for Ubers, why is the decision being changed by Chaos instead of just being put to a vote by tier leaders? In the end, the result will be the same.

So Chaos' intervention means that he and others must see this as a willful tampering of the votes, in which case it's not a "mistake" on the TLs' part, rather it's an attempt to change the outcome of a vote to meet their desires. If that is not the case, I see no reason to change the decision. You can't beat around the bush in this case and say the TLs are new and prone to mistakes and then reverse the decision they made unless they did something wrong, in which case it's no longer a mistake. And regardless, if you would rather that they never put it to a suspect test in the first place, then the outcome would have been one where Shadow Tag was banned regardless. And it's not like there aren't tiers where bans are made solely on the whims of the council (see: UU). Is there anything stopping this same group of leaders from coming to a decision amongst themselves and declaring a ban of Shadow Tag by council decision tomorrow? And if there isn't, how is that any different from what occurred in this test? I just don't see what was done wrong here given what has been said and what I understand about the situation.

These are all genuine questions, by the way. I hope I didn't come across as rude/accusatory, because that was definitely not my intention. I really don't understand what happened and would love some clarification.

This exactly. Criteria were made, these were objective. Discuss: is stag uncompetitive, yes or no" and you vote will be accepted. If you didnt, rejcecto. 40 votes or so were rejected, this means these votes didnt follow the criteria and thus got rejected. Its really that simple. We are not in the position to doubt the rejection/acception of these votes.

I dont get why chaos did this. Him disagreeing with the rejected votes is the only logic reason. I mean no one is gonna stop him, but it wouldn't be unfair to tag chaos to ask why this happened. It says in the op the "results were controversial". Who made them controversial? Why are they being accepted all of the sudden? I mean chaos can overrule anything (duh) but how does can he judge what votes were good and which werent, arent the tier-leader and forum-mods supposed to do that? I'd like clarification too.

(btw sorry if I werent supposed to tag you chaos)

 

[Karxrida]

This exactly. Criteria were made, these were objective. Discuss: is stag uncompetitive, yes or no" and you vote will be accepted. If you didnt, rejcecto. 40 votes or so were rejected, this means these votes didnt follow the criteria and thus got rejected. Its really that simple. We are not in the position to doubt the rejection/acception of these votes.

I dont get why chaos did this. Him disagreeing with the rejected votes is the only logic reason. I mean no one is gonna stop him, but it wouldn't be unfair to tag chaos to ask why this happened. It says in the op the "results were controversial". Who made them controversial? Why are they being accepted all of the sudden? I mean chaos can overrule anything (duh) but how does can he judge what votes were good and which werent, arent the tier-leader and forum-mods supposed to do that? I'd like clarification too.

(btw sorry if I werent supposed to tag you chaos)

The results were controversial because, at the end the day, what the mods did was tamper with the votes. Regardless of their methods or reasoning, removing any votes from being counted is tampering with the outcome, no matter how you present it.

 

[Lemonade]

Making required COIL higher does not help much, it just means you need more time to get reqs, also there is no "voting law" appicable here, it's not a democratic process nor do I believe it should be. Balancing for competitive games shouldn't be handled by a majority, it should be handled by those who play at the most competitve level. Like 70% of the ladder at least will probably lose to a well played GeoXern but that doesn't mean it's banworthy. Just look at every actually competitve game atm Dota, LoL, Hearthstone, CS, in pub games you can stomp with anything but when players are skilled, things that lose all the time are suddenly stronger. For that reason I abstained since I'm not all that good.

Bascially I think filtering votes was perfectly fine but having no anti ban people looking over them or changing the % needed was more suspect, so stop whining about the wrong things.

 

[Disaster Area]

The results were controversial because, at the end the day, what the mods did was tamper with the votes. Regardless of their methods or reasoning, removing any votes from being counted is tampering with the outcome, no matter how you present it.

I disagree with your accusation of this. And it being considered 'tampering' in your eyes I don't feel makes it illegitimate, it's been used before waaay back, and, especially since we've thusfar found no tiering policy for ubers which is golden and perfect, I feel it's unfair to accuse like this.

At least there is behind the scenes focus on making sure this doesn't happen again. But for now, we have questions that are meritorious of being answered, in my opinion.

 

[Jibaku]

Can this not be a thread for controversy?

This thread was intended to help heal the scars caused by this suspect test, and for us to reflect on what we can learn from this and recall what made Ubers a happy place. If you guys need to debate over paragraphs, do so in another thread. Not that it's relevant anymore since we're abolishing Ubers suspect tests anyways.

 

[PISTOLERO]

R.I.P. Theorymon

unless you're into therapeutic pain I don't think you're gonna get any healing here

 

[Soul Fly]

Okay I'm reading a lot of ambiguity here. Basically tl;dr'ing Jibaku, but the issue here was that paragraphs became a really subjective criterion, especially when 2 of the members of the review board were very vocally pro-ban and had their logical approach strongly confirm to it against most arguments any anti-ban user could provide, therefore basically introducing an element of privileged bias. (really, really harsh analogy, but imagine the Pope chairing a debate over the existence of god)

It's natural then that people would feel that it was really unfair and not really democratic because their opinion could be rejected simply because a counter didn't agree with their line of logic (flawed or not). That's all it boils down to. Let's not fling wild accusations please.

[PS: users claiming paragraphs aren't a thing in other tiers are wrong. OU has required it on multiple occasions to weed out troll voters (before the rating system rejig happened). The clutch here is drawing the line between checking for basic metagame competency and holding up the argument to your internal confirmation bias.]

That and the entire philosophy of ubers going a radical redefinition over ibo's interpretation set against the classical definition of Ubers, which despite all precautions and countless debates both internally and in public ended up being a sticky bone of contention between key members of the community.

But guys please. Melee Mewtwo is one of the finest players and community contributors I've had a privilege of knowing, and Fireburn has been a tier leader with good intent. No matter how it went down, both of them had a vision to try make ubers better and that's what really counts, and at the end of the day is behind any sort of inspired leadership. This may not have turned out nicely but let's not please hold it against them. They are really really good people. And I hope they continue being a productive presence in the tier.

Back to Groudoning guys! Excellent changeup ahead of us with the ORAS release..... As Theorymon said, expect the unexpected.

 

[Toshhy]

I don't think it is right for anybody, unless you are a supreme court justice or something, to take a person's reason and say that it is right or wrong. First of all, what is the person in charge of the suspect testing based their decision on? Right, their own reasons.

Secondly, even if the some voters have "flawed" reasoning (which we are in no place to define in the first place) their votes should still count for the reason being that they are a part of the playerbase. Whether experience or inexperience, knowledable or not, they play on Pokemon Showdown like everybody else, thus they are entitled to a vote like everybody else.

I think a big part of what went wrong was too much ego. Established players and moderators assuming that they know better than the majority and should make the desicision for them as some of the common players are incompetent in their eyes. Moreover, they want to prove that they know better and go out of their way, even writing entire essay to convey their point, so they have a lot at stake as established players for the vote to go their way.

I think people should tune in to the news and media more and what's happening in politic in the real world. Do all people vote for Governors and Presidents due to their qualifications or do some people look at celebrity status, race, or attachment?

I'm glad that the desicion to overturn was made and I hope we can all learn from this. ORAS will be right around the corner with Mega Rayquaza, Primal Groudon and a bunch of other new things. It will be a chance at an exicting new start, let make this a healthy and enjoyable tier to play in!

Edit: I agree that they should make the reqs higher but no paragraphs.

 

[The Manifold Curiosity]

Removal of votes based off of subjective reasoning is completely unethical because you are esstentially giving control of the outcome to the people counting them. I'm not exactly an expert on voting laws but I assume it's highly illegal to ignore/throw out votes from qualified voters just because you believe their reasoning is "flawed". If someone got reqs to vote then their vote should count, and if you want to cull bad players either make reqs harder to get or figure out a way to weed out people who aren't part of the Ubers community.

I may not be a part of the Ubers community, but the outcome of this test makes me hesitant to join because the people in charge have abused their powers to a degree and I honestly can't trust them to be responsible anymore.

I do want this thread to be used as a healing process for the community, but I have to admit Kaxrida is totally right. There were so many aspects about the S-tag suspect process that seemed to be biased for banning S-tag that it really got under my skin. Had Chaos not overturned the result I would have rage-quit from Ubers, possibly from Smogon overall cause of how unfair the entire process had been.

I could write a page out of all the serious concerns I have and Jibaku's post also concerned me. But I don't want to. I joined Smogon partly because I really liked the Uber community. It was tight knitted and harmonious and looked like a fun place to be part of. But shortly after I joined the shadow tag suspect began and I saw the community fall into very aggressive infighting. By then end of the voting I had detached myself from Ubers.

But now that the matter is resolved and there are going to be no more suspect tests I really hope we can all forget the past and get back to the way Ubers used to be before I joined. ORAS came at the perfect time. And reading past posts I know MM2 and Fireburn are intelligent and genuine people that would not deliberately rig the vote. I have full confidence in them and once everyone has had their say and the appropriate changes made Ubers will be a great community once again.

 

[Celticpride]

Then make the reqs higher instead of asking a paragraph.

Again, that doesn't work in every case. Personally, I could see throwing out votes if someone wrote a paragraph about how Shadow Tag was broken, when the whole deal with Tag was about uncompetativeness. Those are two entirely different things, and if you don't understand that broken =/= uncompetitive, you don't really understand why the test is happening. Anything beyond that gets into a gray area immediately, but I could see paragraphs being useful for weeding out who don't even understand the purpose/idealism behind the test, especially where Ubers doesn't even consider bans for brokenness, only for being uncompetitive. Other tiers ban for brokenness, and people who don't main or lurk Ubers may not understand that the tiering philosophy here is very different from OU, UU, and the like.

 

[Minority]

For those who don't know, the purpose of the laddering requirement was not to filter out "bad" players, rather it was to filter out people not dedicated enough to the tier to spend the time to get reqs. Making the reqs higher does not really solve the problem, it just makes it more painful for everyone to vote and possibly deters some of the players that should be voting.

 

[MoxieInfinite]

but when shit players make it, its a problem