Afternote: This post has johnnyg2 and Jirachee's contributions omitted. See the post below for more accurate VRs. Results of the analyses are largely the same.
Hi everyone,
I have worked with
Emeral on this year's VR update as per last year. Again, this update will be based on an extension of my original methodology described
here, taking inspiration from a previous post by Jorgen. Thanks to
Kristyl M Dragon Sakito McMeghan BIHI Excal Pideous ABR oiponabys Tomahawk Christo Le Don SFG Emeral for your rankings. The raw data is in the zip file, but I suppose Emeral might post or edit into this post an online spreadsheet for your convenience.
Okay, TLDR stuff first:
The average outlier-compensated ranks from everyone are
Aggregate Rankings
01 Jirachi
02 Tyranitar
03 Latias
04 Clefable
05 Skarmory
06 Flygon
07 Breloom
08 Heatran
09 Metagross
10 Rotom
11 Gyarados
12 Gengar
13 Swampert
14 Lucario
15 Gliscor
16 Dragonite
17 Magnezone
18 Empoleon
19 Zapdos
20 Hippowdon
21 Azelf
22 Bronzong
23 Starmie
24 Suicune
25 Quagsire
26 Machamp
27 Roserade
28 Scizor
29 Forretress
30 Blissey
31 Kingdra
32 Infernape
33 Milotic
34 Magneton
35 Nidoqueen
36 Mamoswine
37 Celebi
38 Aerodactyl
39 Uxie
40 Gastrodon
41 Froslass
42 Donphan
43 Ludicolo
44 Qwilfish
45 Camerupt
46 Abomasnow
47 Raikou
48 Gallade
49 Togekiss
50 Shaymin
51 Cresselia
52 Heracross
53 Rhyperior
54 Tentacruel
55 Slowbro
56 Moltres
57 Vaporeon
58 Hitmontop
59 Slowking
60 Weavile
61 Crobat
62 Claydol
63 Smeargle
64 Mesprit
65 Kabutops
66 Jolteon
67 Venusaur
68 Exeggutor
69 Dusknoir
70 Snorlax
71 Victreebell
72 Porygon-2
73 Registeel
74 Porygon-Z
75 Feraligatr
76 Alakazam
77 Omastar
78 Yanmega
79 Aggron
80 Umbreon
81 Walrein
82 Torterra
83 Toxicroak
84 Ambipom
85 Magcargo
86 Muk
87 Staraptor
88 Sceptile
89 Cradily
90 Hariyama
91 Steelix
92 Probopass
93 Wailord
94 Nidoking
95 Spiritomb
96 Azumarill
97 Ariados
Removing Pokemon that are ranked by less than 4 voters yields
Aggregate Rankings
01 Jirachi
02 Tyranitar
03 Latias
04 Clefable
05 Skarmory
06 Flygon
07 Breloom
08 Heatran
09 Metagross
10 Rotom
11 Gyarados
12 Gengar
13 Swampert
14 Lucario
15 Gliscor
16 Dragonite
17 Magnezone
18 Empoleon
19 Zapdos
20 Hippowdon
21 Azelf
22 Bronzong
23 Starmie
24 Suicune
25 Quagsire
26 Machamp
27 Roserade
28 Scizor
29 Forretress
30 Blissey
31 Kingdra
32 Infernape
33 Milotic
34 Magneton
35 Nidoqueen
36 Mamoswine
37 Celebi
38 Aerodactyl
39 Uxie
40 Gastrodon
41 Froslass
42 Donphan
43 Ludicolo
44 Qwilfish
45 Camerupt
46 Abomasnow
47 Raikou
48 Gallade
49 Togekiss
50 Shaymin
51 Cresselia
52 Heracross
53 Rhyperior
54 Tentacruel
55 Slowbro
56 Moltres
57 Vaporeon
58 Hitmontop
59 Slowking
60 Weavile
61 Crobat
62 Claydol
63 Smeargle
64 Mesprit
65 Kabutops
66 Jolteon
67 Venusaur
68 Exeggutor
69 Dusknoir
70 Snorlax
71 Victreebell
72 Porygon-2
73 Porygon-Z
74 Feraligatr
75 Alakazam
76 Yanmega
77 Aggron
78 Umbreon
79 Walrein
80 Muk
81 Staraptor
82 Steelix
83 Probopass
84 Spiritomb
85 Azumarill
The aggregate VR tiers obtained are
Final VR Tiers
S1: :Jirachi:
S2: :Tyranitar::Latias:
S3: :Clefable::Skarmory:
A1: :Flygon::Breloom:
A2: :Heatran::Metagross::Rotom::Gyarados::Gengar::Swampert:
A3: :Lucario::Gliscor::Dragonite:
B1: :Magnezone::Empoleon::Zapdos:
B2: :Hippowdon::Azelf::Bronzong::Starmie:
C1: :Suicune::Quagsire::Machamp::Roserade:
C2: :Scizor::Forretress::Blissey::Kingdra::Infernape:
D1: :Milotic::Magneton::Nidoqueen::Mamoswine::Celebi::Aerodactyl::Uxie::Gastrodon::Froslass::Donphan::Ludicolo::Qwilfish::Camerupt::Abomasnow::Raikou::Gallade::Togekiss:
D2: :Shaymin::Cresselia::Heracross::Rhyperior::Tentacruel::Slowbro:
E: E Rank
:Moltres::Vaporeon::Hitmontop::Slowking::Weavile::Crobat::Claydol::Smeargle::Mesprit::Kabutops::Jolteon::Venusaur::Exeggutor::Dusknoir::Snorlax::Victreebel::Porygon2::Porygon-Z::Feraligatr::Alakazam::Yanmega::Aggron::Umbreon::Walrein::Muk::Staraptor::Steelix::Probopass::Spiritomb::Azumarill:
Let's go through the whole process.
First the data is cleaned by compensating outliers 1 standard deviation away from the edge of the percentiles expected to contain +/- 1 standard deviation of a normal distribution. This is a modification of the conventional interquartile range (IQR), which I have not chosen to use because 50% of the sample doesn't capture the full variation from what I've seen. The compensation is done by bringing these points to the edge of this extended range. This results in mostly zero, but sometimes one or two outlier corrections. We then plot the outlier-removed data as a function of the integer rank to obtain this graph.
VR Tiering Decisions
Full VR Ranking Plot
and we can zoom in to the top 35 mons where tiering decisions are expected to make more sense,
We use hierarchical clustering to help obtain the tiers. We form a dissimilarity matrix where the distances between Pokemon X and Y are given by the following: Take the rate at which voters ranked Pokemon X over Pokemon Y, take the logit transform as is done in logistic regression of a Bernoulli-distributed variable, and take the absolute value. Performing what we call a Ward linkage, this yields a dendrogram of the following sort, where the clusters (what we are going to call tiers) formed by setting a reasonable threshold are represented by different colors, and the
dissimilarity between each cluster can be thought of as the vertical height of the nearest branch that connects the two clusters.
For example, the Flygon-Breloom tier is a lot closer to the Swampert-Gyarados tier (connecting height 10, note the log scale) than the Clefable-Skarmory tier (connecting height 20). This justifies placing Flygon-Breloom within the same tier as Swampert-Gyarados as opposed to Clefable-Skarmory. Note that the order is not preserved by the algorithm, and hierarchical clustering throws away information, so this is just a rough guide to defining the tiers.
Nominally, we can obtain the VR tiers by setting a threshold (Ward Distance ~14) below which we consider everything to be a tier, and another threshold (Ward Distance ~6) below which we consider different branches to be subtiers. The tier divisions should be checked for the distinguishability conditions laid out in my methodology post; the main discrepancy is that the ranking plot shows that
Jirachi is almost unanimously ranked #1 even though the thresholds place it in the same subtier as Tyranitar and Latias.
Thus, we will assign Jirachi to its own subtier S1. The other discrepancy is such that the distinguishability conditions in the ranking plot mandate that Jirachi, Tyranitar-Latias, Clefable-Skarmory, and Flygon-Breloom form their own tiers S, A, B, C, though it would violate conventional impressions of viability to do so. Thus, we stick to the dendrogram-generated tiers for this.
To visualize the clusters obtained from the dendrogram, we next plot the dissimilarity matrix and draw out the tiers specified.
To read the dissimilarity matrix, note that zero (the darkest value) corresponds to equal number of people voting in favor and against the Pokemon on the Y axis > X axis, and the higher the value, the more one-sided the voting becomes. In other words, the darker, the more indistinguishable the Pokemon on the X and Y axis become, and a well-defined tier would be a fully dark square (read my methodology thread for explanations).
This yields the following subdivision,
S1: :Jirachi:
S2: :Tyranitar::Latias:
S3: :Clefable::Skarmory:
A1: :Flygon::Breloom:
A2: :Heatran::Metagross::Rotom::Gyarados::Gengar::Swampert:
A3: :Lucario::Gliscor::Dragonite:
B1: :Magnezone::Empoleon::Zapdos:
B2: :Hippowdon::Azelf::Bronzong::Starmie:
C1: :Suicune::Quagsire::Machamp::Roserade:
C2: :Scizor::Forretress::Blissey::Kingdra::Infernape:
D1: :Milotic::Magneton::Nidoqueen::Mamoswine::Celebi::Aerodactyl::Uxie::Gastrodon::Froslass::Donphan::Ludicolo::Qwilfish::Camerupt::Abomasnow::Raikou::Gallade::Togekiss:
D2: :Shaymin::Cresselia::Heracross::Rhyperior::Tentacruel::Slowbro:
E: E Rank
:Moltres::Vaporeon::Hitmontop::Slowking::Weavile::Crobat::Claydol::Smeargle::Mesprit::Kabutops::Jolteon::Venusaur::Exeggutor::Dusknoir::Snorlax::Victreebel::Porygon2::Porygon-Z::Feraligatr::Alakazam::Yanmega::Aggron::Umbreon::Walrein::Muk::Staraptor::Steelix::Probopass::Spiritomb::Azumarill:
Numerical ranks represent partial tiers, whereas letter ranks represent a more complete separation. As explained in my methodology thread, I choose to adopt numerical subranks because there is no reason a priori to believe that Pokemon are grouped in viability by a tripartite scheme of +/-.
Metagame Shifts
I would usually be able to compare rankings across different VR editions, but this is the first time we're doing this for DPP.
Analysis of Camps
I tried to look out for divided opinions in different groups of subtiers. I clustered voters based on their rankings of Pokemon within the selected tiers, then extrapolated to find patterns in the rest of the OU Pokemon.
This lets you observe how a camp's preferences within some group of higher ranked mons bleed into the lower ranked mons, and vice versa.
S to A1: Clefable/Skarmory, Flygon/Breloom Split
Dendrogram of S-A1, Jirachi to Breloom
From the Dendrogram, we identify a camp from M Dragon-ABR, and from Tomahawk-BIHI. In this case, I do not find the average ranks to be as telling as the dissimilarity matrix, so I will explain my observations when we get there. Nevertheless, we can plot the mean rankings of each Pokemon for the two camps,
Relative Ranks of camps formed from S-A1 split
Note again the error bars are uncertainties in means, not the actual deviations (ie divided by sqrt(N-1)). Looking at M Dragon's camp for reference, and using the z-score to see the most significant changes, we obtain the following bar chart. We see that the M Dragon camp has general preferences for Tyranitar, Skarmory, Breloom, Heatran, Swampert, Starmie, and Roserade, while Tomahawk's camp has general preferences for Latias, CLefable, Flygon, Gengar, Lucario, Gliscor, Bronzong, and Forretress.
z-score ordered by rank
z-score ordered by aggregate rankings as an alternate POV
Average ranks are simple to read but don't answer the question, "the M Dragon camp likes Breloom, but in comparison to what?".
The dissimilarity matrix helps us to compare any two Pokemon, giving us a more complete picture. To read it, note that when you see a strongly red square, then the camp being analyzed frequently ranks the corresponding Pokemon on the Y axis more favorably than that on the X axis. Another way to see this is by looking across the diagonal line. A strongly red square should be accompanied by a strongly blue square reflected across the line, and we can say the camp prefers the Pokemon on the Y axis of the red square more than that of the blue square. For example, in the data below, the M Dragon camp prefers Tyranitar to Latias.
The defining features of the M Dragon camp are
- Within the subtiers analyzed (S-A1),
- Strong preference for Skarmory > Clefable
- Strong preference for Breloom > Flygon
- Moderate preference of Tyranitar > Latias
- Outside the subtiers analyzed,
- Strong preference for Heatran, especially > Gengar
- Moderate preference for Swampert, especially > Lucario and Gengar
- Moderate preference for Starmie, especially > Zapdos
- Strong preference for Roserade, especially > Bronzong,
A2 to A3: Gengar/Swampert, Heatran Split
Dendrogram of A2/A3, Heatran to Dragonite
From the Dendrogram, we identify a camp from Oipon-Pideous, and from Sakito-SFG. In this case, I find the mean rankings and resulting z-scores to provide sufficient information, i.e. most differences are broad ranking changes rather than specific Pokemon pair comparisons.
Relative Ranks of camps formed from A2/A3 split
The defining observation from the z-scores is that the Oipon camp strongly prefers Gengar while the Sakito camp strongly prefers Swampert. To a lesser extent, the Sakito camp also prefers Heatran. The dissimilarity matrix also demonstrates that the Oipon camp favors Flygon to Breloom.
z-score ordered by aggregate rankings as an alternate POV
Dissimilarity Matrix of A2/A3
B1 to B2: Azelf/Zapdos Split
Dendrogram of B1-B2, Magnezone to Starmie
From the Dendrogram, we identify a camp from Le Don-Oipon, and from Pideous-Emeral. As with S-A1, I do not find the average ranks to be as telling as the dissimilarity matrix, so I will explain my observations when we get there. Making the same series of plots again,
Relative Ranks of camps formed from S-A1 split
z-score ordered by rank
z-score ordered by aggregate rankings as an alternate POV
Based on the dissimilarity matrix, the defining features of the Le Don camp are
- Within the subtiers analyzed (B1-B2),
- Strong preference for Azelf, especially > Zapdos
- Strong preference for Hippowdon > Zapdos
- Strong preference of Azelf and Hippowdon > Bronzong and Starmie
- Moderate preference of Magnezone > Zapdos
- Outside the subtiers analyzed,
- Broad dislike for Gliscor, especially < Lucario, Gyarados, and Rotom
- Moderate preference for Blissey > Suicune and Starmie
- Moderate preference for Infernape > Roserade and Scizor
As I don't play this tier, I invite you to comment based on your metagame knowledge what you think these different camps of opinions mean.
Individual Analyses
For those who are interested to see whose S to C rankings are closest to theirs, you can refer to the chart below. The numbers inside the box go from -100% (full anticorrelation) to 100% (full correlation). They are sorted by the S to C dendrogram order (and the light squares represent the camps).
Ranking Correlations between Voters, S-C
And finally, these are the relative ranks of everyone. Blue = disfavor, Red = favor. Cyan lines demarcate tier cutoffs.
Individual Ranking Graphic
Closing Remarks
The large number of graphs may seem daunting, and to people who aren't quantitatively trained, this may be really confusing. I recommend just glancing over the spoilers on the first read, only thoroughly analyzing them after you've gone through the more important graphs that have been left unhidden. I'm interested to know what you can infer from these trends even though I don't play the metagame, and I hope this can generate some discussion.
Finally, I have attached the Jupyter notebook for this in a zip file. You'll need python and the associated packages to open and run it, and the easiest way to do that is to install the anaconda library
here.
DPP OU VR 2022 Public.zip