Latias Suspect Qualifier Statistics

[DougJustDoug]

I really didn't want to do this at first, but after a few discussions with Jumpman, I concluded that it would probably be worthwhile to collect some specific statistics in support of our Suspect testing. I wrote some programs that collect specific statistics on players' use of a specific pokemon on a specific ladder. In this case, I collected stats on the use of Latias by players on the Suspect ladder.

Basically, I parsed every log (over a quarter-million of them!) and found all the battles on the suspect ladder (several thousand). From there, I collected special statistics about battles where Latias was used. I recorded which players in the battle had Latias on their team, and I collected how often Latias was switched in. I also recorded who won the battle, and how many turns it lasted.

From those statistics, I made a report of all the battles involving the players that qualified for voting, based on rating criteria. I have all the battles, but I'm only generating a report for the voting qualifiers. I then collected summaries of the qualifiers' experiences using and facing Latias, separated for December and January. Here's the summary report:

+--------+---------+-----------------+---------------+------------+----------------+----------------+----------------+----------------+-----------+----------------+
| Period | Suspect | Player Name     | Total Battles | Total Wins | Plyr Only Used | Plyr Only Wins | Opp. Only Used | Opp. Only Wins | Both Used | Both Used Wins |
+--------+---------+-----------------+---------------+------------+----------------+----------------+----------------+----------------+-----------+----------------+
| 200901 | Latias  | BOLTS           |            45 |         34 |              0 |              0 |             45 |             34 |         0 |              0 |
| 200812 | Latias  | DSUPREME        |           221 |        149 |             42 |             25 |             93 |             71 |        86 |             53 |
| 200901 | Latias  | DSUPREME        |            18 |         12 |             10 |              7 |              4 |              3 |         4 |              2 |
| 200812 | Latias  | DUSKIE          |            85 |         60 |             22 |             14 |              0 |              0 |        63 |             46 |
| 200901 | Latias  | DUSKIE          |           104 |         74 |             45 |             32 |              0 |              0 |        59 |             42 |
| 200812 | Latias  | EREBYSSIAL      |            27 |         20 |              4 |              4 |              0 |              0 |        23 |             16 |
| 200901 | Latias  | EREBYSSIAL      |            22 |         16 |              4 |              3 |             13 |             10 |         5 |              3 |
| 200812 | Latias  | EXECUTIVE KOALA |            62 |         50 |             30 |             24 |              0 |              0 |        32 |             26 |
| 200901 | Latias  | EXECUTIVE KOALA |            41 |         28 |             30 |             22 |              0 |              0 |        11 |              6 |
| 200901 | Latias  | F.B.I.          |            70 |         46 |             32 |             25 |              0 |              0 |        38 |             21 |
| 200812 | Latias  | GF-FAILURE      |            64 |         41 |             26 |             17 |              5 |              3 |        33 |             21 |
| 200901 | Latias  | GF-FAILURE      |            55 |         32 |             22 |             16 |              9 |              5 |        24 |             11 |
| 200812 | Latias  | HAUNTER         |            46 |         27 |             18 |             10 |              0 |              0 |        28 |             17 |
| 200901 | Latias  | HAUNTER         |            56 |         41 |             13 |             10 |              0 |              0 |        43 |             31 |
| 200812 | Latias  | HEC.            |             4 |          2 |              0 |              0 |              4 |              2 |         0 |              0 |
| 200901 | Latias  | HEC.            |            32 |         19 |              2 |              0 |             29 |             19 |         1 |              0 |
| 200812 | Latias  | HIDDEN          |            16 |          8 |              1 |              0 |              0 |              0 |        15 |              8 |
| 200901 | Latias  | HIDDEN          |            38 |         25 |             17 |             12 |              1 |              1 |        20 |             12 |
| 200812 | Latias  | IMPERFECTLUCK.  |            20 |         19 |              0 |              0 |             20 |             19 |         0 |              0 |
| 200901 | Latias  | IMPERFECTLUCK.  |            34 |         31 |              0 |              0 |             34 |             31 |         0 |              0 |
| 200812 | Latias  | INFINITY        |           162 |        109 |             52 |             33 |              3 |              2 |       107 |             74 |
| 200901 | Latias  | INFINITY        |            54 |         31 |              0 |              0 |             54 |             31 |         0 |              0 |
| 200812 | Latias  | JANE AUSTIN     |           108 |         86 |             33 |             30 |              0 |              0 |        75 |             56 |
| 200901 | Latias  | JANE AUSTIN     |            69 |         57 |             30 |             25 |              0 |              0 |        39 |             32 |
| 200812 | Latias  | KEVIN GARRETT   |            40 |         24 |             16 |             13 |              4 |              2 |        20 |              9 |
| 200901 | Latias  | KEVIN GARRETT   |            26 |         22 |              0 |              0 |             26 |             22 |         0 |              0 |
| 200812 | Latias  | KNUCKLES_6      |            12 |          7 |              0 |              0 |             12 |              7 |         0 |              0 |
| 200901 | Latias  | KNUCKLES_6      |            25 |         15 |              0 |              0 |             25 |             15 |         0 |              0 |
| 200901 | Latias  | NI CHI LE MA    |            52 |         31 |             31 |             19 |              0 |              0 |        21 |             12 |
| 200812 | Latias  | PANAMAXISTEST   |           158 |        118 |             44 |             35 |              0 |              0 |       114 |             83 |
| 200901 | Latias  | PANAMAXISTEST   |            18 |         18 |              9 |              9 |              0 |              0 |         9 |              9 |
| 200812 | Latias  | ROOT            |            42 |         26 |              0 |              0 |             42 |             26 |         0 |              0 |
| 200901 | Latias  | ROOT            |            22 |         17 |              0 |              0 |             22 |             17 |         0 |              0 |
| 200812 | Latias  | SHIBA           |            15 |         14 |              6 |              5 |              1 |              1 |         8 |              8 |
| 200901 | Latias  | SHIBA           |            57 |         37 |             18 |             13 |              4 |              2 |        35 |             22 |
| 200812 | Latias  | SITZMARK        |           315 |        200 |             23 |             10 |            200 |            133 |        92 |             57 |
| 200901 | Latias  | SITZMARK        |            61 |         43 |              0 |              0 |             61 |             43 |         0 |              0 |
| 200812 | Latias  | SON OF BLUNDER  |            71 |         56 |             22 |             18 |              0 |              0 |        49 |             38 |
| 200901 | Latias  | SON OF BLUNDER  |            32 |         24 |             21 |             14 |              0 |              0 |        11 |             10 |
| 200901 | Latias  | TESTING.        |            83 |         62 |             38 |             27 |              0 |              0 |        45 |             35 |
| 200812 | Latias  | TWIST OF FATE.  |            73 |         54 |              4 |              2 |             63 |             48 |         6 |              4 |
| 200901 | Latias  | TWIST OF FATE.  |           138 |         86 |             68 |             37 |             10 |              8 |        60 |             41 |
| 200812 | Latias  | XENOGEARS       |           108 |         71 |             27 |             20 |             33 |             20 |        48 |             31 |
| 200901 | Latias  | XENOGEARS       |            30 |         22 |             14 |             10 |              2 |              1 |        14 |             11 |
+--------+---------+-----------------+---------------+------------+----------------+----------------+----------------+----------------+-----------+----------------+

Plyr Only Used shows the number of battles where Latias was only used by the named player.
Plyr Only Wins shows the number of those "Plyr Only Used" battles won by the named player.
Opp. Only Used shows the number of battles where Latias was only used by the opponent of the named player.
Opp. Only Wins shows the number of those "Opp. Only Used" battles won by the named player.
Both Used shows the number of battles where Latias was by both the named player and their opponent.
Both Used Wins shows the number of those "Both Used" battles won by the named player.
Total Battles simply sums up all the battles involving the named player, where Latias was involved.
Total Wins shows the number of those "Total Battles" won by the named player.

So basically the report above tells you how often a given player dealt with Latias in suspect battles. For example, you can see that several players (Imperfectluck, Bolts, Knuckles, Root) never actually used Latias on their team in any of their Suspect battles. They faced Latias several times, but they never used it on their team. I'm not commenting whether that is good or bad. I'm just showing the stats. This also gives a rough idea of how often the player tested Latias over the length of the suspect testing period.

The primary purpose of these stats, is to give the "essay evaluators" some tangible data to refer to, when reading player's reasoning for voting privileges.

(NOTE: The report above does not report summary information for any battles where NEITHER side used Latias. I wasn't too interested in those battles for summary purposes. But, those non-Latias battles are included below.)


The base battle records used to compile the summaries for the qualifiers are listed here. I could not include all that data in this post; it's just too much data. Here is a small sample of the information in that file.

+---------+---------------------+-------------+--------------------+-------------+---------+---------+---------+---------+--------------+
| Suspect | Battle Time         | Player 1    | Player 2           | Winner      | P1 Used | P1 Sent | P2 Used | P2 Sent | Battle Turns |
+---------+---------------------+-------------+--------------------+-------------+---------+---------+---------+---------+--------------+
| Latias  | 2009-01-04 12:16:33 | JANE AUSTIN | GEMINI             | JANE AUSTIN | Yes     |       1 | No      |       0 |           18 |
| Latias  | 2009-01-04 12:52:04 | JANE AUSTIN | OZBORNE            | JANE AUSTIN | Yes     |       2 | No      |       0 |           23 |
| Latias  | 2009-01-04 10:41:11 | SITZMARK    | ASSFACE            | ASSFACE     | No      |       0 | No      |       0 |           45 |
| Latias  | 2009-01-04 16:01:57 | JANE AUSTIN | TEHFURY            | JANE AUSTIN | Yes     |       0 | No      |       0 |            6 |
| Latias  | 2009-01-04 03:50:49 | HAUNTER     | NERFD              | HAUNTER     | Yes     |       0 | No      |       0 |            9 |
| Latias  | 2009-01-04 20:51:40 | SHIBA       | TEHFURY            | SHIBA       | No      |       0 | No      |       0 |           31 |
| Latias  | 2009-01-04 14:38:37 | SITZMARK    | BLUE OF SENSUALITY | SITZMARK    | No      |       0 | No      |       0 |           64 |
| Latias  | 2009-01-04 18:44:04 | ROOT        | LRT                | ROOT        | No      |       0 | Yes     |       5 |           23 |
| Latias  | 2009-01-04 14:30:48 | SITZMARK    | GURU_TEST          | GURU_TEST   | No      |       0 | Yes     |       2 |           56 |
| Latias  | 2009-01-04 15:47:22 | JANE AUSTIN | TEHFURY            | JANE AUSTIN | Yes     |       2 | No      |       0 |           22 |
+---------+---------------------+-------------+--------------------+-------------+---------+---------+---------+---------+--------------+

Used means that Latias was on the player's team.
Sent is the number of times the player switched Latias into the battle.
Battle Turns are the total number of turns in the battle.


So there it is. I spent quite a bit of time putting this stuff together -- which is one of the reasons I made a not-so-nice post in the Latias Voting thread in Stark Mountain. Writing a bunch of custom program code, and sifting through all that battle data, just to end up collecting information about 24 people -- feels like a big waste of time. But, once I got started with it, I felt like I should see it through to the end -- even if few people are actively involved.

On a positive note -- now that I'm done, I can reuse these programs for future suspect tests. Even though there aren't many players, I still think the statistics are interesting. But, I'm just weird when it comes to stuff like this...

I hope this data helps the Suspect testing project.

 

[Mekkah]

Thanks a lot for this, Doug.

They faced Latias several times, but they never used it on their team. I'm not commenting whether that is good or bad. I'm just showing the stats.

I don't think I'm too happy about this. I think Jump emphasized before that you are supposed to try out the Suspect yourself. Besides the fact that it makes your vote more educated, it also gives your opponent more opportunities to fight against Latias. And with the Suspect Ladder being as quiet as it is, every opportunity to have a Latias involved could count.

 

[X-Act]

Well, testing whether a team without Latias can handle Latias itself is also important. If a team without Latias can handle Latias fine, as well as other threats, then that would be a relatively important argument in favour of allowing Latias in OU.

The problem is the "other threats" part. A player could build a team specifically to counter Latias that wouldn't take into account most other threats, taking the fact that Latias is obviously going to be played to his advantage, so that he rises up the ladder in order to be able to vote. That's something that wouldn't conclude anything remarkable about Latias' status.

So there is both a good side and a bad side of it.

 

[Mekkah]

taking the fact that Latias is obviously going to be played to his advantage, so that he rises up the ladder in order to be able to vote.

I would say that is more of a problem with Suspect testing in general, though.

 

[Aeroblacktyl]

sounds more like they were just using the advantage of 'knowing the opponents' team which pretty much means counter teaming and anti-latias teaming to where it doesn't really help prove anything at all..

 

[Jumpman16]

Can't thank you enough for actually going through with this, Doug. This is the kind of selfless effort to a brand new and important cause I referenced in Tangerine's "Recognizing the Contributors of the Testing Processes" thread.

Anyway, here's the not-so-private message between Aeolus and I on December 18 when I first thought of this (im deleting the little that is remotely actually "private" so I'm sure Aeo wont mind):


[16:54] <Jumpman16> that's fine
[16:54] <Jumpman16> i think either point can be argued lol
[16:54] <Aeolus> but a possible solution i thought of was two separate suspect ladders run simultaneously
[16:55] <Aeolus> one with latios and one with latias
[16:55] <Jumpman16> workable
[16:55] <Jumpman16> hm

[16:55] <Aeolus> because honestly, i see latias getting very very little usage on a ladder where they are both allowed but only one per team
[16:56] <Jumpman16> both are definitely viable
[16:56] <Jumpman16> yeah
[16:56] <Aeolus> and we could have two separate votes
[16:56] <Jumpman16> however, the first concern with that that pops into my head
[16:56] <Jumpman16> is that people will be less inclined to "test" both
[16:56] <Jumpman16> like, the "latias ladder...ok..."
[16:56] <Jumpman16> and 14 people qualify
[16:56] <Aeolus> we could require it
[16:56] <Aeolus> lol
[16:56] <Jumpman16> lol
[16:57] <Aeolus> to vote on either, must qualify on both
[16:57] <Jumpman16> the best way would be to spend the month
[16:57] <Jumpman16> but that's gay
[16:57] <Jumpman16> hm
[16:57] <Jumpman16> yeah
[16:57] <Aeolus> i just don't know

[16:58] <Jumpman16> i think your way may be a little more viable
[16:58] <Jumpman16> but "lol two ladders"
[16:58] <Aeolus> the other way to do it is test latias first
[16:58] <Aeolus> and then test latios
[16:59] <Aeolus> and not do it simultaneously
[16:59] <Jumpman16> though we would be "making" them do latias over a month if we did it the two-month way
[16:59] <Jumpman16> that reminds me though
[16:59] <Jumpman16> at what point do we start caring about time out loud
[16:59] <Jumpman16> like, we cant take "forever"
[17:00] <Aeolus> oh i know
[17:00] <Aeolus> but i think we'd be deluding ourselves if we thought firmly that there is an end anywhere in sight
[17:00] <Jumpman16> yeah
[17:00] <Jumpman16> uh, lol
[17:01] <Jumpman16> we have what 8-9 suspects left
[17:01] <Aeolus> yeah
[17:01] <Aeolus> lol
[17:01] <Aeolus> we'll have gen 5 before we're done
[17:01] <Aeolus> and more problems to deal with
[17:01] <Jumpman16> yeah

[17:01] <Aeolus> as i sit here...
[17:01] <Jumpman16> it is a valid question
[17:01] <Aeolus> I'm thinking it would be better to test Lati@s first
[17:01] <Aeolus> and then test Latios
[17:01] <Jumpman16> lol
[17:01] <Aeolus> i mean latias
[17:01] <Aeolus> damn it
[17:02] <Aeolus> just because i think it is more likely to be OU

k here is where the important part starts

[17:02] <Aeolus> I really want a way to do them at the same time
[17:03] <Aeolus> but doing them on the same ladder seems to be a bad way of doing it
[17:03] <Aeolus> and 2 ladders is... well shitty
[17:03] <Aeolus> especially when one can be ignored
[17:03] <Jumpman16> "unless we require it"

[17:04] <Jumpman16> maybe if there were an easy way to assess how many times a batter "experienced" latias
[17:04] <Jumpman16> used + saw
[17:04] <Jumpman16> i think i would actually like that
[17:04] <Aeolus> on a per battler basis?
[17:05] <Jumpman16> yeah
[17:05] <Jumpman16> if your "Latias Exp" is low with five days left
[17:05] <Aeolus> well that would be something
[17:05] <Jumpman16> that's probably your fault for not using it
[17:05] <Jumpman16> "so use it"
[17:05] <Jumpman16> something like that though

[17:05] <Aeolus> you know.... we should implement that anyway
[17:05] <Jumpman16> really depends on how advanced "our" stats-grabbing is
[17:05] <Aeolus> some sort of "exp" rating
[17:05] <Jumpman16> yeah, lol
[17:05] <Aeolus> we need doug here
[17:05] <Aeolus> we should send him a pm to just ban skymin
[17:06] <Aeolus> and find us for a talk before setting up the ladders for lati@s
[17:06] <Jumpman16> yeah

[17:10] <Aeolus> ok i've just shot off a pm to doug to hold off on the lati@s ladder
[17:10] <Aeolus> until he talks to one of us
[17:10] <Aeolus> i wish he was just here now
[17:10] <Jumpman16> yeah

[17:11] <Aeolus> I really love the idea of an "EXP" rating
[17:11] <Aeolus> it encourages usage of the suspect beyond the normal allure
[17:12] <Aeolus> but honestly, i have no idea what to judge as a "high rating"
[17:12] <Aeolus> measured by usages/appearances or whatever
[17:13] <Jumpman16> well, we can just use our first exposure to it as a barometer
[17:13] <Jumpman16> then go from there
[17:13] <Jumpman16> that kind of thing has to be somewhat easy to determine though
[17:14] <Jumpman16> "Latias" is coded into every battle, etc
[17:14] <Jumpman16> that's the kind of thing other besides doug can really work on.
[17:14] <Aeolus> i agree
[17:14] <Jumpman16> like, the people who can honestly respond to chaos's thread in IS about programming
[17:14] <Jumpman16> can do this.

[17:14] <Aeolus> oh oh
[17:15] <Aeolus> well my main concern was that we'd have to announce this EXP thing to the public and set a threshold
[17:15] <Aeolus> but we wouldn't have to do that if we adopted the new system
[17:15] <Jumpman16> yeah
[17:15] <Aeolus> we talked about the other day
[17:15] <Jumpman16> just "be on the lookout"
[17:15] <Jumpman16> oh well even still.
[17:15] <Aeolus> also, were you ok with the final numbers i proposed?
[17:16] <Aeolus> 1760/55
[17:16] <Jumpman16> we could just say that "this new system will be implemented. there will be no EXP requirements for the first iteration, while we determine a sufficient barometer"
[17:16] <Jumpman16> etc
[17:16] <Jumpman16> yeah they're fine

[17:16] <Aeolus> well really we could even use it this time
[17:16] <Aeolus> because the vast majority of voters will fall into the limbo category
[17:16] <Aeolus> and will need our approval
[17:17] <Aeolus> we can just look at their EXP rating relative to the others to determine if we approve them or not
[17:17] <Jumpman16> yeah
[17:17] <Jumpman16> yep.
[17:17] <Jumpman16> 100% objective


So obviously the idea now is "how much is enough?"

Bolts (5KR) didn't use Latias once but was one of the few to actually make the upper tier. I took the liberty of analyzing the battles he played against someone who also did not use Latias because I love scrutinizing him, and this is what I found. He was 57-23 in those battles. That is a 71.25% winrate. In the battles where he did face Latias, he was of course 34-11, which is a 75.56% winrate. Now I obviously, obviously know that rating isn't just determined by winning percentage, but I think it does say something, in a metagame where Latias was evidently the runaway #1 pokemon used throughout, that someone can actually have many more battles where he literally didn't experience it than ones where he did. That statistic is: 45 (battles with Latias) / 45 + 80 (total battles on the Suspect Ladder) = 36%.

This point is crucial: those 80 almost literally non-Suspect battles he played were indeed in an environment where the threat of Latias was obviously acknowledge by all battlers, and therefore to beat those players does indicate your proficiency in a metagame where one is prepared for any given Suspect. However, we must remember that the "any given Suspect" here is just one Suspect, Latias, and the fact that for those 80 battles, both 5KR and his opponent used teams that only had to address the threat of one Suspect, which is the entire reason that we cannot learn much from tournament settings where, if present, everyone is overprepared for the likely single solitary "Suspect" (unusual) thing they will have to face in order to win.

Therefore, it very much is not expected to be difficult to "overprepare" for literally one Suspect and win for a month, but winning battles with no Suspect EXP, while it may help your rating, literally cannot possibly give you any experience to vote on a Suspect's Stage 2 tag. How well you are able to play in a metagame that doesn't have the Suspect 64% (100% - 36%) of the time should literally have virtually no bearing on your impression on how the Suspect impacted OU. To give you a better reason of that, check my sticky thread on PR with the Characteristics of Uber. Offensive, Defensive and Support. Latias literally cannot figure into the definition of the first two chars when neither player uses her, and only in a roundabout way can someone state something like "the threat of Latias set up a situation in which it makes it substantially easier for other pokemon to sweep".

That's really, really hard to argue: one can start to say that "since I knew my opponent would be overprepared for Latias and carry the pokemon that counter her, like Tyrantiar and Scizor and Magnezone and Blissey, I recognized Latias set up this situation and therefore made it substantially easier for my Swords Dance Lucario to sweep, and therefore Latias is uber." Like, really? This kind of hearkens back to the "triple dragon triple steel" phenomenon people are pointing out, but if none of those dragons are Latias on either team (no Latias EXP), how can you specifically point to Latias as the cause of the phenomenon, and not Salamence, when you remember the reason that we're doing this Suspect Test Process and the reason that Latias herself is Suspect is expressly because she may not have been tiered correctly in the first place, and therefore may have always been every bit as OU as Salamence?

Please reread that if you don't understand me or why I would point it out. I can't say "I'm not trying to single out 5KR" cause that's a lie but I will say that I'm not really looking for a reason to DQ him—the fact that he didn't use Latias at all, and therefore is objectively not as qualified as most of the rest of the voters, is 100% a coincidence. If he had used Latias on his team(s) I would actually almost applaud him, but aside from the fact that I don't like him and his philosophy on competitive pokemon is seriously the most ridiculous one I've ever heard voiced, the numbers I pointed out literally indicate that his experience with the Suspect was very, very lacking.

Also, what just occurred to me is that if we wanted to "we" could decide to look at something like "1600/70 people" who obviously didn't qualify by the numbers we gave but, if they had a significant degree of literal experience with the Suspect, they should really be allowed to sound off. People like Veedrock for example, who may have just missed the cut but if he literally experienced Latias more than like 90% of the other people in the 1600/70 range and up, he should be allowed to vote, but mainly because he was "close" to the given requirement which was the only one we knew about. I know most of that falls on Doug pulling up even more stats but yeah...I was seriously when I mentioned in my PM to Aeo that he isn't the only one who can sift through the pertinent logs, and he has confirmed this to me in PM.

I know that if we publish our desire for people to experience the Suspect to a TBD degree, it could and probably will result in pretty much everyone of "Veedrock's caliber and worse" always, always cramming this kind of EXP by always using the Suspect in question. This isn't something we necessarily want, even if it's obvious the the mere use of a Suspect doesn't guarantee you a win (which means these people will be farther fI propose that we could actually straight up state that:

"The Suspect Test Facilitators have determined an unpublished but 100% objective requirement in addition to the 1760/55 and 1655/65 requirements posted, and are not publishing this requirement in order to maintain the integrity of the efforts put forth to vote and the Suspect Test Process in general. We will be granting voting privileges to those who meet this unpublished requirement in addition to those who meet those published."

I actually think that's really, really fair, and that anyone who suspects that this "requirement" is actually us filtering out the people we don't like can seriously eat a dick.

Finally, I will state that this, owing to the "unpublished" idea, should not leak to the public unless and until we decide it should (which I don't think we should but that is what discussion is for). Remember, this is Inside Scoop. This is probably the most "secret" thing I have ever posted here, and I don't want it getting out lest Doug, Aeolus and my efforts and time thinking about this and running statistics be wasted. We just now need to quantify how much weight we should attribute to "Used" and "Sent" values in Doug's link (and probably in general even if as of Doug's first post we don't have "Sent" values for those who always faced the Suspect).

 

[reachzero]

I can see the value in both using and not using the Suspect in any given suspect test, since some questions related to the Suspect need to be answered without using it. For example, even though IPL didn't use Latias even once, I know firsthand that he changed the makeup of his team specifically to handle Latias. As X-Act pointed out, the knowledge that teams without Latias can beat teams with Latias is significant information in itself. However, without experimentation and innovation, you can't know everything about the quality of the Suspect, and you can't count on your opponent to experiment for you. For example, I realized fairly late in the test that I had only tested Latias for one of the three characteristics of Uber (as a sweeper), while it was possible that because of her fast dual screen ability, she might be Uber under the Support Clause. Only late in the test did I see DS Latias much, and I never really got around to using her myself. In other words, I'm glad to see that the stats can show us what effect using/not using the Suspect has on winning and losing for each player (thanks Doug!), yet I'd rather have a voter that could tell me that he/she had used LO Latias, stall-based Latias, and DS Latias, as well as trying the ladder without Latias. I must admit that I'm not such a voter, I used only two or three Latias sets, and I don't think I laddered seriously at all without Latias. My point is that experience with using the Suspect is not everything in testing, but we should definitely try to get as much experience with the Suspect into the voter pool as we can. We have a much better chance of knowing how well the various "types" of Latias work that way. What we decide on this issue, by the way, is important for later in the test process: Latias is a pretty good fit in the standard metagame in terms of its uses, it's not an "oddball" requiring a specialized team to either utilize or deal with. Can you imagine when we start testing Manaphy? Rewarding experience with the Suspect can only be a good thing (within reason, we shouldn't set the standards too low), in my opinion.

 

[ToF]

Reachzero's post is precisely the reason I think people without once utilizing the suspect should not have the right to vote. I mean, if you can't even utilize the suspect on a team (regardless of how overprepared you are for it), you won't have much of an idea of where it falls into in terms of our newly established Uber classifications. When I started off suspecting, I used an anti-Latias team, as 5KR and IPL used, and won with a decent percentage; however, in this manner I was not testing the suspect itself, but merely metagaming against the suspect, a rule which I think needs clarifying and possibly banning.

As time progressed, I decided to try Latias in a few different manners. Ridding myself of the anti-Latias team, I tried out DS Latias, to some degree of success. Afterwards, on the F.B.I. account, I used an all-out offensive Latias, which worked quite nicely with a decent amount of bulk. In this regard, I feel that I tested Latias in multiple manners, and have a much better sense of where its tiering placement should be.

I'm not crediting myself with this comment, but rather proposing that some grounds should be set in later suspect tests to use the suspect, even in one manner at minimum. This way, the quality of the voting pool is heightened, and we won't have people like 5KR who simply anti-metagame the suspect and vote based on the ease of taking it out.

 

[DougJustDoug]

I was thinking along the same lines as Jumpman, when I posted this data. I think we can make an "unpublished objective qualification criteria" and add voters to the mix. This possibility is the reason I posted the data in IS, and not in Stark Mountain. This is also the reason that I ONLY published information about the battlers that have ALREADY qualified for voting. As I mentioned in the OP, I collected stats and summaries for EVERY suspect ladder match held in December and January. I have full "Latias Experience Data" for every player that battled -- not just the qualifiers. If we decide to look at other players' Latias experience, combined with a different rating minimum -- perhaps we will find some other battlers that should have the right to vote.

I have all that information, but I don't plan to post it here. Nothing against most badgeholders -- but IS leaks are way too common. Until we figure out what we are going to do, I don't want data related to a possible "unpublished qualification requirement" to get out to everyone and cause a shitstorm of second-guessing on what that requirement should be. As it stands, the stats in the OP are simply the underlying battle data for people that have already qualified. I don't want the OP data published throughout Smogon -- but I wanted to use it as a springboard for discussion in IS about how to proceed with Suspect testing. The rest of the "unpublished" data is going to remain truly unpublished for now.

BTW, after reading Jumpman's comments about the relevance of battles in which NEITHER side used Latias -- I'm going to add that to the player summary report. I originally thought that a non-Latias battle was almost irrelevant. However, considering that every player was battling under the premise that Latias was a possibility -- then every Suspect battle was relevant and should be summarized to get a good picture of a player's experience in the Suspect test. I'll rerun the reports and post an update here.

 

[darkie]

I definitely think that keeping the exact voting requirements private - from anyone - is a good idea. Public requirements tend to make people ladder for a little while - until they reach the requirements - and then stop. On the other hand, private requirements would force people to ladder the entire time. Of course, private requirements could also repel voters completely, which is obviously not something we want. Keeping two requirements, one public and the other private, could still attract battlers, but of course the fact that keeping stuff completely private is easier said than done means this method may not be as effective as we would like.

Regardless, I firmly believe that secret voting requirements (even if they were secret from badgeholders themselves) is the best way to get the most competent voters.

 

[Aeolus]

I really just want to reiterate how important it is for you all to keep this info to yourselves. Doug, thank you so much for this.

 

[Kevin Garrett]

I went in the opposite order that Twist of Fate did. I started off testing out Latias and with varying success. Most of the teams when I first started playing had Latias reinforced by a lineup of Pokemon that counter opposing Latias. In Janurary, I switched to an anti-metagame team and had a more consistent success. Nevertheless, I had experience with battling opposing Latias and using one myself. I don't see how one can have a full understanding of a suspect if they don't use it at least once. And in the case of FiveK, I wonder how much anti-metagame he really used on the ladder. I battled him on his "Bolts" account on both standard and suspect and he was using the same team as he always has. That makes me wonder if he put in any effort at all, aside from battling.

The unpublished requirement in addition to the rating requirement is an excellent idea since it will hopefully keep players involved throughout the entire suspect process. The only thing I would add to that is an inquiry could be sent to potential voters in a 1600/70 range that actively used the suspect to ensure that none of them would determine their vote by anything other then their experience on the ladder. Other players might have a system of beliefs along the lines of FiveK's beliefs that we do not know about. It would just prevent the voting process from receiving cloudy results.

Thanks for all your time and effort, Doug. These stats are remarkable.

 

[Jumpman16]

The only thing I would add to that is an inquiry could be sent to potential voters in a 1600/70 range that actively used the suspect to ensure that none of them would determine their vote by anything other then their experience on the ladder. Other players might have a system of beliefs along the lines of FiveK's beliefs that we do not know about. It would just prevent the voting process from receiving cloudy results.

By virtue of their not having reached the posted requirements, wouldn't these people still have to address the same topics everyone who doesn't make the upper tier does? Meaning Aeolus and I would throw out their votes in an instant?

I will also make clear that I think this unpublished requirement should go for everyone. I'm not really averse to forbidding people like 5KR and even ipl vote if they literally didn't use the Suspect at all. Especially when the same unpublished requirement is bound to let others who are not quite as good at battling as ipl vote in his stead.

 

[DougJustDoug]

Here are the updated qualifier summaries, including the battles where neither player used Latias.

+--------+---------+-----------------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+
|        |         |                 |          |          |  Player  |  Player  | Opponent | Opponent |          |   Both   |          |  Neither |
|        |         |                 |  Total   |  Total   |   Only   |   Only   |   Only   |   Only   |   Both   |   Used   |  Neither |   Used   |
| Period | Suspect | Player Name     | Battles  |  Wins    |   Used   |   Wins   |   Used   |   Wins   |   Used   |   Wins   |   Used   |   Wins   |
+--------+---------+-----------------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+
| 200901 | Latias  | BOLTS           |       80 |       58 |        0 |        0 |       45 |       34 |        0 |        0 |       35 |       24 |
| 200812 | Latias  | DSUPREME        |      268 |      182 |       42 |       25 |       93 |       71 |       86 |       53 |       47 |       33 |
| 200901 | Latias  | DSUPREME        |       26 |       17 |       10 |        7 |        4 |        3 |        4 |        2 |        8 |        5 |
| 200812 | Latias  | DUSKIE          |       85 |       60 |       22 |       14 |        0 |        0 |       63 |       46 |        0 |        0 |
| 200901 | Latias  | DUSKIE          |      105 |       74 |       45 |       32 |        0 |        0 |       59 |       42 |        1 |        0 |
| 200812 | Latias  | EREBYSSIAL      |       27 |       20 |        4 |        4 |        0 |        0 |       23 |       16 |        0 |        0 |
| 200901 | Latias  | EREBYSSIAL      |       34 |       25 |        4 |        3 |       13 |       10 |        5 |        3 |       12 |        9 |
| 200812 | Latias  | EXECUTIVE KOALA |       62 |       50 |       30 |       24 |        0 |        0 |       32 |       26 |        0 |        0 |
| 200901 | Latias  | EXECUTIVE KOALA |       41 |       28 |       30 |       22 |        0 |        0 |       11 |        6 |        0 |        0 |
| 200901 | Latias  | F.B.I.          |       70 |       46 |       32 |       25 |        0 |        0 |       38 |       21 |        0 |        0 |
| 200812 | Latias  | GF-FAILURE      |       64 |       41 |       26 |       17 |        5 |        3 |       33 |       21 |        0 |        0 |
| 200901 | Latias  | GF-FAILURE      |       67 |       42 |       22 |       16 |        9 |        5 |       24 |       11 |       12 |       10 |
| 200812 | Latias  | HAUNTER         |       46 |       27 |       18 |       10 |        0 |        0 |       28 |       17 |        0 |        0 |
| 200901 | Latias  | HAUNTER         |       56 |       41 |       13 |       10 |        0 |        0 |       43 |       31 |        0 |        0 |
| 200812 | Latias  | HEC.            |        6 |        3 |        0 |        0 |        4 |        2 |        0 |        0 |        2 |        1 |
| 200901 | Latias  | HEC.            |       65 |       49 |        2 |        0 |       29 |       19 |        1 |        0 |       33 |       30 |
| 200812 | Latias  | HIDDEN          |       16 |        8 |        1 |        0 |        0 |        0 |       15 |        8 |        0 |        0 |
| 200901 | Latias  | HIDDEN          |       38 |       25 |       17 |       12 |        1 |        1 |       20 |       12 |        0 |        0 |
| 200812 | Latias  | IMPERFECTLUCK.  |       35 |       32 |        0 |        0 |       20 |       19 |        0 |        0 |       15 |       13 |
| 200901 | Latias  | IMPERFECTLUCK.  |       73 |       66 |        0 |        0 |       34 |       31 |        0 |        0 |       39 |       35 |
| 200812 | Latias  | INFINITY        |      180 |      123 |       52 |       33 |        3 |        2 |      107 |       74 |       18 |       14 |
| 200901 | Latias  | INFINITY        |       76 |       47 |        0 |        0 |       54 |       31 |        0 |        0 |       22 |       16 |
| 200812 | Latias  | JANE AUSTIN     |      108 |       86 |       33 |       30 |        0 |        0 |       75 |       56 |        0 |        0 |
| 200901 | Latias  | JANE AUSTIN     |       69 |       57 |       30 |       25 |        0 |        0 |       39 |       32 |        0 |        0 |
| 200812 | Latias  | KEVIN GARRETT   |       41 |       25 |       16 |       13 |        4 |        2 |       20 |        9 |        1 |        1 |
| 200901 | Latias  | KEVIN GARRETT   |       54 |       39 |        0 |        0 |       26 |       22 |        0 |        0 |       28 |       17 |
| 200812 | Latias  | KNUCKLES_6      |       17 |       11 |        0 |        0 |       12 |        7 |        0 |        0 |        5 |        4 |
| 200901 | Latias  | KNUCKLES_6      |       49 |       31 |        0 |        0 |       25 |       15 |        0 |        0 |       24 |       16 |
| 200901 | Latias  | NI CHI LE MA    |       52 |       31 |       31 |       19 |        0 |        0 |       21 |       12 |        0 |        0 |
| 200812 | Latias  | PANAMAXISTEST   |      158 |      118 |       44 |       35 |        0 |        0 |      114 |       83 |        0 |        0 |
| 200901 | Latias  | PANAMAXISTEST   |       18 |       18 |        9 |        9 |        0 |        0 |        9 |        9 |        0 |        0 |
| 200812 | Latias  | ROOT            |       63 |       40 |        0 |        0 |       42 |       26 |        0 |        0 |       21 |       14 |
| 200901 | Latias  | ROOT            |       34 |       25 |        0 |        0 |       22 |       17 |        0 |        0 |       12 |        8 |
| 200812 | Latias  | SHIBA           |       15 |       14 |        6 |        5 |        1 |        1 |        8 |        8 |        0 |        0 |
| 200901 | Latias  | SHIBA           |       67 |       46 |       18 |       13 |        4 |        2 |       35 |       22 |       10 |        9 |
| 200812 | Latias  | SITZMARK        |      431 |      276 |       23 |       10 |      200 |      133 |       92 |       57 |      116 |       76 |
| 200901 | Latias  | SITZMARK        |      125 |       93 |        0 |        0 |       61 |       43 |        0 |        0 |       64 |       50 |
| 200812 | Latias  | SON OF BLUNDER  |       71 |       56 |       22 |       18 |        0 |        0 |       49 |       38 |        0 |        0 |
| 200901 | Latias  | SON OF BLUNDER  |       32 |       24 |       21 |       14 |        0 |        0 |       11 |       10 |        0 |        0 |
| 200901 | Latias  | TESTING.        |       83 |       62 |       38 |       27 |        0 |        0 |       45 |       35 |        0 |        0 |
| 200812 | Latias  | TWIST OF FATE.  |       97 |       71 |        4 |        2 |       63 |       48 |        6 |        4 |       24 |       17 |
| 200901 | Latias  | TWIST OF FATE.  |      143 |       89 |       68 |       37 |       10 |        8 |       60 |       41 |        5 |        3 |
| 200812 | Latias  | XENOGEARS       |      136 |       90 |       27 |       20 |       33 |       20 |       48 |       31 |       28 |       19 |
| 200901 | Latias  | XENOGEARS       |       32 |       22 |       14 |       10 |        2 |        1 |       14 |       11 |        2 |        0 |
+--------+---------+-----------------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+

 

[Hipmonlee]

I dont like this idea.

It seems unnecessary.. It's almost like you are making too large a demands of these people to qualify.

I mean, I know you guys put a hell of a lot of time and effort into this, I know I have put a hell of a lot of time and effort into the suspect tests and I am only a peripheral figure in this whole thing. But we cant expect everyone to match that time and effort in order to vote.

Bolts has battled 80 times. At 4 minutes per battle that is 5 hours worth of battling. 45 of those times involved Latias. 45 battles is really quite a lot.. I am guessing that that is about as many battles as a lot of people would have had to qualify for previous tests..

I mean, if a person genuinely thinks that latias sucks balls then what good is it to force them to use it. It will be a self fulfilling prophecy at best. You could argue that maybe it just so happens that their team just happens to be very good at dealing with Latias, but even if that is true, then in a vote, it's kinda appropriate that that person would vote Latias OU - if over half of all teams dont have any problem with it, then it probably shouldnt be uber.

Not to mention the possibility of really exaggerating the issue of the test being unnatural because people only test when they want to use Latias. If you force people to use Latias then you are forcing people to counter Latias as well..

Latias was used in over half of every players battles on the suspect test ladder.. I really think that is enough.

Have a nice day.

 

[Caelum]

I really agree with Hip on this matter (and for the record I'm opposed to any "secret" requirements just solely based on principle - not that I question the integrity of either Jump or Aeolus, it's a principle thing). For example, in my Shaymin-S vote I really didn't use it on the ladder that often because I genuinely believe it isn't a very good Pokemon. I shouldn't be forced to use something that I don't think is that great just to prove that I used it (and if I don't consider something "good", I'm obviously not going to vote it uber!). I also disagree with the idea that I can't understand how a Pokemon functions in the metagame unless I actually used it. By playing against it and common sense I can figure that out and I think we might be overestimating the necessity of actually using a suspect consistently to understand it. Granted, in this situation, I'd be surprised if most players didn't use Latias at least initially (possibly on alts) just out of curousity.

I do like the idea of giving people who were "close" a shot at least :)

Edit: If this is implemented I wouldn't want this weighted too heavily but I'm not opposed to at least requiring some usage of the suspect, just not excessive usage.

 

[X-Act]

The ideal thing would be the following: the rating and deviation of the players wouldn't be updated at all if a battle did not contain at least one instance of the suspect Pokemon. Note that, in this way, one of the players would be allowed not to use the suspect Pokemon, as long as the opponent is.

In addition to that, one would need to have used the suspect Pokemon a few times in order to be able to vote.

I repeat that the above would be the ideal thing to do, but, of course, it might not be practical to implement.

 

[Misty]

I agree fully with Hip and Caelum. If people are winning without Latias, and they're winning against both Latias users AND non-Latias users, it means that they can play against Latias without using it AND they aren't just using a retarded counterteam (if they were, they would lose against non-Latias users). Remember, the objective of a suspect test is to find out what a metagame looks like with the suspect injected. If everyone just uses the suspect, you don't get a real idea of what people who DON'T use it are going to be capable of, which IMO is really critical to finding out whether something breaks the game or not - consider that everyone was using Garchomp prior to its banning.

 

[Jumpman16]

Remember, the objective of a suspect test is to find out what a metagame looks like with the suspect injected.

What? The point of the Suspect Test is to put the Suspect in isolation and determine, through experience, whether or not the Suspect is uber. It may sound like an ego trip to say it but "I think I would know". I can't help but remember what chaos said like eight months ago about the Suspect Ladder idea and suggesting we should "ask them which metagame they like better" after a two month comparison" and wonder if that's why you said that. We're not really doing that anymore.

If everyone just uses the suspect, you don't get a real idea of what people who DON'T use it are going to be capable of, which IMO is really critical to finding out whether something breaks the game or not - consider that everyone was using Garchomp prior to its banning.

The reason this doesn't matter mainly ties into why "what a metagame looks like with the suspect" isn't the main point, but I will state that the idea behind keeping this requirement private is to not influence people to whore the Suspect either way. We're not going to encourage a Suspect's usage either way. And even when I actually did encourage usage of Wobbuffet it never broke #43 in usage, but was overwhelmingly considered uber. Since Colin had allowed them on the Official Server and they were available for months, we already knew what people were capable of when not everyone was using Deoxys-S and Garchomp (and Wobby) too, which to your argument contributed a lot to their banishment.

 

[Jumpman16]

I dont like this idea.

It seems unnecessary.. It's almost like you are making too large a demands of these people to qualify.

I think you are missing the point of having these statistics at our disposal. We want to increase the emphasis on experience with the Suspect, and decrease emphasis on "this is how good i am at pokemon". We are literally going to increase the number of voters that garner the kind of experience we are really looking for. What we stand to "lose" in people like Bolts and ipl we will gain in people like Veedrock and M_Dragon who just missed out on the requirement.

And by "lose" I underline what the point of this thread should be—"how much is enough?" We haven't even decided on a threshold yet. And even then, I would actually think it's practical that if someone has marks of 1950/40 but doesn't use the Suspect, they shouldn't be forbidden to vote at all, but they should then have to address the two Paragraph topics like everyone else who doesn't make the Upper Limit.

I mean, if a person genuinely thinks that latias sucks balls then what good is it to force them to use it. It will be a self fulfilling prophecy at best. You could argue that maybe it just so happens that their team just happens to be very good at dealing with Latias, but even if that is true, then in a vote, it's kinda appropriate that that person would vote Latias OU - if over half of all teams dont have any problem with it, then it probably shouldnt be uber.

Not to mention the possibility of really exaggerating the issue of the test being unnatural because people only test when they want to use Latias. If you force people to use Latias then you are forcing people to counter Latias as well..

Latias was used in over half of every players battles on the suspect test ladder.. I really think that is enough.

Have a nice day.

I honestly think you just missed the part where the players wouldn't know they "have to" use the Suspect in question.

 

[Hipmonlee]

It just occured to me that this may cause a slight bias to the voting.

You are adding voters from people that are using the suspect on their teams a lot. It seems to me that people that use the suspect are going to be more likely to vote uber. Surely an overwhelming majority of people who dont use the suspect during the test are going to vote OU at any rate.

Have a nice day.

 

[RBG]

I would seriously like to disagree hip. Many of the people who voted Uber in the Skymin test admitted to not having used it because they knew it would be Uber and I used it plenty and I voted OU

 

[reachzero]

I agree strongly with RB Golbat, having used Skymin and Latias a ton during the testing of each. The more you use a Pokemon, the more you learn what it can and can't do. Nothing tells you more about Latias then having it die to Tyranitar or Weavile for the fourth or fifth consecutive battle. You don't get that perspective without using it. If anything, I think not using the Suspect would tend to bias a voter toward Uber, since the player has only seen the Suspect one battle at a time. He or she has not seen how any particular set plays out over a longer period of time.

 

[Aeolus]

I also disagree with Hipmonlee. The best way to learn the capabilities of the suspect is to use it. I'd like to emphasize, however, that we have never said that people who do not use the suspect will be denied the right to vote. Such users will still be able to gain experience with the suspect by playing it in battle and will be able to accrue "exp" in that way.

 

[skarm]

I agree strongly with RB Golbat, having used Skymin and Latias a ton during the testing of each. The more you use a Pokemon, the more you learn what it can and can't do. Nothing tells you more about Latias then having it die to Tyranitar or Weavile for the fourth or fifth consecutive battle.

I fully agree with this -- I resisted using Skymin originally and didn't have too many problems dealing with it, however then I switched to a Skymin team so I experienced both ends.

You don't get that perspective without using it. If anything, I think not using the Suspect would tend to bias a voter toward Uber, since the player has only seen the Suspect one battle at a time. He or she has not seen how any particular set plays out over a longer period of time.

I cannot agree with this however. I didn't make the Latias cut, but I did use it in several "phases". Originally I didn't use Latias on the ladder, and did fine. I thought Latias was terrible since, clearly, if it was an Uber it should be sweeping my team (or walling it utterly, depending on which category etc.) I then made a team built around Latias and had zero success using Latias. Perhaps it wasn't a Pokemon a was familiar with and not using to its potential, but it did fairly nothing while the rest of my team constantly had to work hard.

Finally, I switched back to a team without Latias which was one I had used on the standard ladder. This was near the end of December and I found several other uses not bothering to use Latias because it wasn't that good.

Personally if you can defeat a suspect without using it time and time again (FiveK for instance did have a good win percentage) that should tell you something about the suspect, in my opinion. If the suspect isn't good enough that you don't even want to use it, well, I wouldn't want to be penalized my vote because I didn't need it to win.

However this really only applies in the case of Latias as that was the easiest vote ever and I could have called that result a week in. Latios I think will be closer, but realistically I don't think we should penalize people voting rights simply because they could win without using a Pokemon tiered as Uber (however wrong that may be).

In short, if people beat an "Uber" Pokemon time and time again, enough to make the voting cut, without using it that should really end the debate right there.