Bold Voting and Rating/Deviation requirements—A happy medium?

I would be fine with something like that, except for the checkpoint system. It feels rather arbitrary for someone to have to meet certain benchmarks each week. I can see it being bad for those interested in meeting the requirements but are unable to participate due to outside matters. To use myself as an example, I'm a full time student and work 5 nights a week. I often don't have time to get on Shoddy during the week. Weekly benchmarks works against people in similar situations who can possibly hit the final result required but are unable to keep up on a weekly basis. Simply starting a testing account would be fine, as forgoing one's normal laddering to help test is already a commitment.
 
I see... that's fair for a system with the Suspect Ladder, which we lacked this month. I would like to see how many more voters would qualify under a deviation of 65 on a Suspect Ladder seeing as some barely made it by the skin of their teeth with Garchomp voting.
Don't you think that a week benchmark would be a bit much especially when people go on family vacations and such? I think if we tried what we were trying during the Skymin test, but with a Suspect Ladder I think the results would reflect more participation and less theorymon.
 

Jumpman16

np: Michael Jackson - "Mon in the Mirror" (DW mix)
is a Site Content Manager Alumnusis a Top Team Rater Alumnusis a Battle Simulator Admin Alumnusis a Smogon Discord Contributor Alumnusis a Researcher Alumnusis a Top Tiering Contributor Alumnusis a Top Contributor Alumnusis an Administrator Alumnus
i think if we made it 10 days "it could work", since that would neatly divide a 30-day test into three parts.

i also would all the more encourage people to appeal to aeo and myself and plainly state their cases if they were to entirely miss a 10-day checkpoint, and we would be able to see when they actually played and just how well they did over the time period, etc. this would further filter out the people who don't really care about actually testing the suspects while at the same time allow aeolus and i to subjective allow those who fall short based on their own reasoning

i kind of like the idea of two 10-day checkpoints....i dont think 15-days would really solve that much since someone would only have to "whore the ladder for a few hours" twice and still not necessarily learn anything about the given suspect, whereas 10 days really seems long enough to me. i think that if you really can't find it in yourself to make a 1650/70 (or whatever, arbitrarily not as hard at 1665/65)) at any point in an entire 10-day span then "sorry". what do you guys think
 

Chou Toshio

Over9000
is an Artist Alumnusis a Forum Moderator Alumnusis a Community Contributor Alumnusis a Contributor Alumnusis a Top Smogon Media Contributor Alumnusis a Battle Simulator Moderator Alumnus
@Tleilax/MI-- Earlier Jumpman posted somewhere "if you're you're not willing to put in the time and participate, you shouldn't have a voice in the matter." I agree entirely. If you can't find the time to play the game, why should we care about your opinion? After all, the policies affect the people who play the game.

The reason why we're discussing problems with the current format is because it encourages to just "cram" a bunch of play-time in at the end of the month, where as what we want are votes based on regular play over the period. Check Points would be the obvious way to solve this. If you need to go on vacation, why should your voice matter? It's that simple.

Jumpman16-- the 1 week scheme I wrote up was completely arbitrary for the sake of illustration. 2 10 day checks leading to the 30 day final seems fair and well spread. I'm not sure where the requirements should be at those points, maybe we can get some folks who know the ranking system's functions better (X-Act and Doug?) to chime in on that. Anyway I think that sounds like it would be good if you and Aeolus don't mind the extra work.
 
Yeah I've given it some thought and 10 day checks would work. Let's be glad this whole thing happened during the third suspect test and not the tenth one.
 

X-Act

np: Biffy Clyro - Shock Shock
is a Site Content Manager Alumnusis a Programmer Alumnusis a Smogon Discord Contributor Alumnusis a Top Researcher Alumnusis a Top CAP Contributor Alumnusis a Top Tiering Contributor Alumnusis a Top Contributor Alumnusis a Smogon Media Contributor Alumnusis an Administrator Alumnus
I like this idea too.

I ran a simulation of the glicko2 system to see what would be good criteria for the first 10 days, the second 10 days and the third 10 days. The final criteria are, of course, a >=1655 rating and a <=65 deviation.

I assumed that a person would need to play 20 battles in 10 days in order to show interest. When that happens, his deviation was never greater than 95. Hence, for the first 10 days, my criterion would be <=95 deviation.

After playing a further 20 battles, his deviation was never greater than 73. Hence, for the second 20 days, my criterion would be <=73 deviation.

(Also, after playing 60 battles, the deviation was never greater than 65. This means that if a person plays 60 battles in 30 days, his deviation should drop to at most 65.)

I'm of the opinion that the criterion for the first 10 days and the second 10 days shouldn't include rating. Maybe the person is testing something, especially in the first 10 days, and loses rating as a result. In my view, the rating should only be checked at the end of the test. The deviation, however, would show that the person is playtesting relatively satisfactorily.

My criteria would hence be:

After 10 days: Deviation is 95 or lower.
After 20 days: Deviation is 73 or lower.
After 30 days: Deviation is 65 or lower and rating is 1655 or higher.

The above assumes a brand new ladder. I'd suggest that a brand new ladder is always used for suspect testing even if it would have the same bans as the Standard ladder one (as what happened with the Skymin test). The fact that for the Skymin test there was no new ladder was something that shouldn't have happened in my opinion. This would make everyone start from scratch and make their way up, and (more importantly) the rating/deviation would conform to the criteria better. It would also show who is really interested in the test by actually playing in a ladder designated for suspect testing, and not just play in the normal ladder.

Feel free to comment or criticize.
 
Great idea, X-Act.

20 battles every 10 days should be enough and much of the Skymin debate is centered around theorymon, which would not be as big of a a problem had we had a Suspect Ladder in November. So, I think we really need one for every test we run.
 

Chou Toshio

Over9000
is an Artist Alumnusis a Forum Moderator Alumnusis a Community Contributor Alumnusis a Contributor Alumnusis a Top Smogon Media Contributor Alumnusis a Battle Simulator Moderator Alumnus
I'm fine with that if we are talking about a 2 ladder test, in other words the player had to play 40 battles total, not 20. In my opinion, twenty battles still seems rather sparse, though if others feel differently I would not put up a big fuss over it.

Just to be clear, we're also going with the "voter registration" part of the proposal? IE everyone starts new accounts and registers them in a threat at Stark Mountain at the start of the period.

I was thinking when Aeolus starts the thread he could announce a certain word that has to be part of the test name. IE it could look like this:

Thread Title: Lati@s Voter Registration!

Starting in four days we will officially begin the testing period for Lati@s, Dec. 20. There will be check points at the end of each 10 day period for your deviation, and a rank/deviation requirement for the last 30 day point. All those who are interested in participating must start a new account for the suspect AND standard ladder each, and post here to register as a potential voter.

The User name you create must include: -LTS


The reason to include the -LTS (or whatever arbitrary word for each different test) in all the registered usernames would be to make sure people can't create an account and start laddering with it until the process has been offically started by Aeolus. It would also give people an awareness as they ladder that they are doing it with the intention of testing/paying attention to the suspect.
 
I just had a couple more thoughts, but I'm not sure how well received they might be.

For one we could have people agree to an agreement upon voting. Something for example that would state that you agree to vote with a relevant nature to the subject and not something completely unrelated. Ex. you wouldn't vote based on what Nintendo would ban at their tournaments.

Also, for additional proper checks should we could require users to give reasons why they voted.

If there is a vote that does not make sense it could come under the discretion of a group of people that would agree or disagree to keep it or throw that vote out or we could just leave it up to the discretion of the people here in PR.

That way it would force people to think why they would vote a certain way and not throw up some random vote that cannot be checked.

I also think we should keep debates out of the voting threads, just vote and state your reasons why.

It would be a user's decision to take part of a vote, but if we had more rules upon signing up to do so I think it might go along more smoothly.

What do you guys think?
 

Aeolus

Bag
is a Top Tutor Alumnusis a Tournament Director Alumnusis a Site Content Manager Alumnusis a Battle Simulator Admin Alumnusis a Smogon Discord Contributor Alumnusis a Top Tiering Contributor Alumnusis a Top Contributor Alumnusis an Administrator Alumnusis a Top Dedicated Tournament Host Alumnus
I don't disagree with the checkpoint system in principle. I think it would go a long way toward forcing people to play the ladder. I seriously doubt the practicality of implementing it successfully and getting 75-100 people to maintain the one-month attention span necessary to hit all the marks.
 

Chou Toshio

Over9000
is an Artist Alumnusis a Forum Moderator Alumnusis a Community Contributor Alumnusis a Contributor Alumnusis a Top Smogon Media Contributor Alumnusis a Battle Simulator Moderator Alumnus
I do not see a need for 75-100 people voting. I rather we feel good about the process, and whether or not people choose to participate is their responsibility, not smogon's.
 

Jumpman16

np: Michael Jackson - "Mon in the Mirror" (DW mix)
is a Site Content Manager Alumnusis a Top Team Rater Alumnusis a Battle Simulator Admin Alumnusis a Smogon Discord Contributor Alumnusis a Researcher Alumnusis a Top Tiering Contributor Alumnusis a Top Contributor Alumnusis an Administrator Alumnus
Yeah, I think the fact that we "only" got 118 actual voters when they didn't even need to play on two ladders is indication enough that 75-100 is wishful thinking.

Besides, we can always just allow the people who were "close" to vote to get to 75 if we have to.
 

X-Act

np: Biffy Clyro - Shock Shock
is a Site Content Manager Alumnusis a Programmer Alumnusis a Smogon Discord Contributor Alumnusis a Top Researcher Alumnusis a Top CAP Contributor Alumnusis a Top Tiering Contributor Alumnusis a Top Contributor Alumnusis a Smogon Media Contributor Alumnusis an Administrator Alumnus
The 10 and 20 day check would actually help people to achieve 1655/65 easier, because they would be playing against more people of similar deviation that way. When you play against people with a deviation much larger than yours, your deviation won't lower much. Having to be forced to have at most a 95 deviation after 10 days would help everyone achieve their targets at the end of the test.
 
So, uh, have we come to a formal agreement upon using the checkpoint system?

What do you guys think about requiring users to give reasons behind a vote? And... what do you guys think about having voters agree to a formal agreement in order to vote? Making sure they make a vote with logical reasoning would ensure that this process went along smoother and it would make things a bit more professional too.

Since this sparked up so much discussion too I think we could keep on seeing a rise in potential voters in the future. I mean look how long the Skymin discussion threads are. People might complain, but that isn't going to stop all of them from participating.
 
I don't know if a formal agreement would be in order. However, I think it would be prudent to state at both the beginning of the test and at the start of the voting that explanation is required for the vote to count, and fallacious reasoning will not be allowed. Examples can be posted to give an idea of what is being looked for. This will likely lower the voter pool, and we will still get junk votes, but it could also filter out people who are voting just because they can.
 
I don't know if a formal agreement would be in order. However, I think it would be prudent to state at both the beginning of the test and at the start of the voting that explanation is required for the vote to count, and fallacious reasoning will not be allowed. Examples can be posted to give an idea of what is being looked for. This will likely lower the voter pool, and we will still get junk votes, but it could also filter out people who are voting just because they can.
I agree and providing examples is a great idea.

I'm just playing devil's advocate here, but what if someone continually signed up and continually made some junky reasons for voting? Would that user be banned from voting on future suspects even if they qualified?
 
I agree and providing examples is a great idea.

I'm just playing devil's advocate here, but what if someone continually signed up and continually made some junky reasons for voting? Would that user be banned from voting on future suspects even if they qualified?
I don't see the harm in letting them vote. Maybe through repeated failures they will eventually figure it out.

To simplify the matter, have the poll up as usual supplemented by the bold voting. Rather than having to count up good and bad votes, votes can simply be looked over and any votes that don't meet the criteria will be removed from the total.

I think it would be fair to PM anyone who's reasoning isn't up to par and let them know. That way they have a chance to correct themselves. Some people are simply not good with words, and that way they get a chance to redeem their vote if they legitimately tried but didn't get it right the first time.
 

Tangerine

Where the Lights Are
is a Top Team Rater Alumnusis a Community Leader Alumnusis a Smogon Discord Contributor Alumnusis a Tiering Contributor Alumnusis a Top Contributor Alumnusis a Smogon Media Contributor Alumnus
Here are the issues with "requiring" reason for voting.

The thing is, there will be certain reasoning that will be "accepted", but you can literally just give template reasoning for that. For example, let's take into consideration my recent encounter with Phillip.

When designing teams in this metagame, I made sure to pack multiple Pokemon who can outspeed with a scarf, use a powerful priority, make sure that SR is somewhere, and resist Skymin's STAB attacks, in case of a rare Scarfmin. Was I able to do this successfully, and make a team that succeeded overall? Yes. However, the pool of Pokemon that fit this criteria was so slim, that I could not vary my teams up much. Having to pack so many fail-safes on one Pokemon is what we call overcentralization. In the Garchomp days, I was rarely seen actually using Garchomp, but I was still successful because I packed enough things to always be able to revenge.
This is the generic uber argument. "It is centralizing", and therefore, it is uber. The issue with this is exactly the problem we are having right now deciding what is "uber" or not. The issue is "how centralizing is too centralizing?" This is precisely the reason we are letting them vote to get a general feel of "what is too much".

Of course, the above reasoning isn't the best one. The issue with the above argument is that the user has failed to apply such reasoning uniquely to Shaymin S - but "is that really a problem"? Do we really want to require pages and pages of reasoning to back up their justification? Face it, most kids who are going to vote aren't capable of writing that much on Pokemon. They play the game because it is fun, and most of them don't study the game enough like some of us apparently do.

This is why "I accepted shitty votes" in the bold voting thread in the past. I was literally "too nice" to users and I gave them the benefit of the doubt (which I will never do again!!!). The obviously shitty votes would be out, but the rest of the votes are hazy nonsense that are just regurgitated bullshit that they have overheard on the forums. Most players' understanding of the game is limited to what they literally just overhear from these threads. This is why many words literally lost their meanings - they have been "buzz words" that make them sound like they know what they are talking about.

Of course, in my opinion, Smogon should promote such a studying of the metagame, and educate the userbase in such a way. This is why I believe that more knowledgeable users should be participating in discussions and recognize such knowledgeable users. But of course, this was literally one of my disagreements with the Administration - this doesn't appear to be one of the administration's priorities. Is it any surprise then that most people find Stark laughable? How can it be anything but laughable, how can the reasoning be anything but shallow if the site doesn't do much to promote understanding of how the game works, just "here are some tools, go out and battle" and "here listen to me talk"? I don't think it is feasible for Smogon to demand reasoning behind the vote with such a system we have now. I think the current system, with deviation/rank + voting without a reason reflects the current site the best and it is unreasoanble to expect any more than that.
 
One way the "kids" can prove that they have studied the metagame is by requring writing a well-thought, analyitical vote. What's wrong with giving this a chance?

Smogon is recognizing some of those users who have a deep understanding of the game (ex. the number of people in PR has increased recently). With the suspect tests some of these players have emerged.

If the voter's reason isn't sufficient we could give them a nice PM and kindly tell them what's wrong as Tlielax suggested. We can educate people along the process, what's wrong with that? In the long run I think the metagame will be better off because we would be encouraging more people to think and analyze it.
 

Seven Deadly Sins

~hallelujah~
is a Site Content Manager Alumnusis a Forum Moderator Alumnusis a Tiering Contributor Alumnusis a Top Contributor Alumnusis a Top Smogon Media Contributor Alumnusis a Battle Simulator Moderator Alumnus
Except it isn't. Suspect Test is doing nothing but stroking the egos of everyone that made the cutoff, no matter how ridiculous or incompetent their decision actually is. That's where the problem comes from.
 

Tangerine

Where the Lights Are
is a Top Team Rater Alumnusis a Community Leader Alumnusis a Smogon Discord Contributor Alumnusis a Tiering Contributor Alumnusis a Top Contributor Alumnusis a Smogon Media Contributor Alumnus
One way the "kids" can prove that they have studied the metagame is by requring writing a well-thought, analyitical vote. What's wrong with giving this a chance?

Smogon is recognizing some of those users who have a deep understanding of the game (ex. the number of people in PR has increased recently). With the suspect tests some of these players have emerged.
If you haven't noticed, the Definition/Portrait of Uber threads are just sitting there "waiting" for your opinion.

Secondly, stop flattering yourself, it's rather disgusting. I would rather have you read my post thoroughly and try to understand what I was trying to say.

If the voter's reason isn't sufficient we could give them a nice PM and kindly tell them what's wrong as Tlielax suggested. We can educate people along the process, what's wrong with that? In the long run I think the metagame will be better off because we would be encouraging more people to think and analyze it.
http://www.smogon.com/forums/showpost.php?p=1654291&postcount=423

Also, "way to read my post before responding to it"
 

Jumpman16

np: Michael Jackson - "Mon in the Mirror" (DW mix)
is a Site Content Manager Alumnusis a Top Team Rater Alumnusis a Battle Simulator Admin Alumnusis a Smogon Discord Contributor Alumnusis a Researcher Alumnusis a Top Tiering Contributor Alumnusis a Top Contributor Alumnusis an Administrator Alumnus
Except it isn't. Suspect Test is doing nothing but stroking the egos of everyone that made the cutoff, no matter how ridiculous or incompetent their decision actually is. That's where the problem comes from.
Tell that to FiveKRunner.
 

Seven Deadly Sins

~hallelujah~
is a Site Content Manager Alumnusis a Forum Moderator Alumnusis a Tiering Contributor Alumnusis a Top Contributor Alumnusis a Top Smogon Media Contributor Alumnusis a Battle Simulator Moderator Alumnus
I most certainly would like to. The system as it is right now is telling people that it's a-ok to act the fool when it comes to metagame decisions. I think that Bold Voting with a well-thought out reason, while alienating some people, will force others to actually think through their votes instead of just validating terrible votes like FiveKRunner's and scorchedsky's. The end result is that while some of the more childish members may be miffed by the whole thing, it will improve Smogon on the whole by forcing people to actually study the metagame and prove it with logic.
 

reachzero

the pastor of disaster
is a Senior Staff Member Alumnusis a Top CAP Contributor Alumnusis a Tiering Contributor Alumnusis a Battle Simulator Moderator Alumnus
Having spent a few days just thinking over the issue of voting, especially as it has come to the forefront in the Shaymin-S vote, I think that several of our issues are tied only indirectly to the method of voting. The real difficulty is the ambiguity of Smogon's philosophy and polity. This ambiguity is evidenced by the lack of community consensus on the definition of Uber, as well as by the blatant disregard for Smogon philosophy displayed by many voters in the Shaymin-S vote, and the subsequent uproar over voter ignorance.
First, the issue of the confusing over defining Uber and the need to ban. Smogon's philosophy, as found at www.smogon.com/philosophy, describes bans and the need to ban this way:
"Smogon attempts to avoid bans as much as possible—only when it becomes very apparent that a Pokémon is far too powerful to be in line with a balanced metagame is it banished permanently from the standard arena."
This definition contains two key points: "far too powerful", and "balanced metagame". If I am interpreting this correctly, then an Uber is a pokemon that is too powerful to be dealt with by reasonable means; unusual centralizing means are needed to deal with it. If Skymin-S compels the opponent to use Registeel or sacrifice two pokemon to kill it, for example, it should clearly be Uber. Likewise, compelling Blissey to use Shed Shell would be a similar argument for Wobbuffet to be Uber. Simple enough so far. What complicates this is that in practice, long-standing and respected members of the Smogon Community, such as imperfectluck, seem to believe that for a pokemon to be Uber, it need not be "far too powerful", only undesirable for producing the best possible metagame, namely one that rewards skill over luck. Since Uber is a ban list, it seems reasonable to question whether the deficiency here lies with variant interpretations such as this one, or with Smogon's official philosophy. Should Ubers be banned by community consensus, without regard for specific reasons? This would not necessarily be truly populist, since rating/deviation restrictions could be required. For that matter, a possible (though not necessarily desirable) method would be to limit such a vote to consensus of Smogonites with badges. This would intentionally allow for philosophical pluralism. It was the method used to vote for Shaymin-S, as demonstrated by the fact that Aeolus' initial post reminded us that we were not required to give any reason whatsoever for our vote. If we follow this method, we must tolerate philosophical pluralism, and Smogon's official philosophy should reflect it in some way. Remember, however, that our banlist is a distinctly Smogon banlist, not a wider "Shoddy Battle" banlist. It is perfectly acceptable for us to insist on following a relatively narrow definition of Uber. If this includes the tag of "far too powerful to be in line with a balanced metagame", then so be it. However, the problem of perceived bias is inevitable when Bold Voting is used. This springs from the lack of clear Smogon polity. Everyone knows that we need to listen to and respect badged members, but the lack of clear hierarchy means that we have no clear arbiter of Smogon's philosophy. In other words, we have pluralism and varying interpretations of Smogon's philosophy even within the top stratum of Smogon (badged members). So along with the method of voting, the question needs to be asked, "Do we want the community to accept the Smogon official philosophy, or the philosophy to tolerate the community, even when it's (in our opinion) unreasonable?"
 

Jumpman16

np: Michael Jackson - "Mon in the Mirror" (DW mix)
is a Site Content Manager Alumnusis a Top Team Rater Alumnusis a Battle Simulator Admin Alumnusis a Smogon Discord Contributor Alumnusis a Researcher Alumnusis a Top Tiering Contributor Alumnusis a Top Contributor Alumnusis an Administrator Alumnus
the definition of uber is going to have a lack of consensus and be ambiguous for much the same reason that people are going to have their own interpretation of what uber really means. instead of appealing to the authority of imperfectluck, whose bold vote on dx-s i actually did not accept (which may or may not be a coincidence), i would ask how much his own interpretation of uber aligns with smogon's philosophy, not the other way around just because he is a respected member of smogon. (obviously none of this is personal, i love ipl and he knows it, and politely understood why i did not accept his votes on dx-s.)

and Aeolus and i have talked about making our policy very, very evident when the time comes to bold vote, but we have both agreed that it won't really make any different because people are still going ot vote the way they want. i will point out 5KR's case again—he has no intention of aligning with our philosophy at all, even when made a public example of "how not to vote". he doesn't care, and we don't have reason to believe we can convince anyone to align with our philosophy (and that people like 5KR will stop voting, as he did the same thing again).

so yes, we can consider tweaking and modifying our philosophy, but it shouldn't be to cater to the whims and interpretations of established battlers even a posteriori, because i would argue that doing so is much, much more dangerous than allowing a handful of "nonaligners" to vote.
 

Users Who Are Viewing This Thread (Users: 1, Guests: 0)

Top