Discussion Using WinShare instead of Usage

[Akiak]

Hi, so over the course of my involvement with competitive Pokemon, I have found that WinShare is a very effective metric for ranking Pokemon by viability, based on data. In fact I would say it's the best metric I've found, despite (or maybe because of) its simplicity.

WinShare is defined as "of the winning teams, how many featured this Pokemon". It is simply calculated by multiplying Usage by Winrate.

When given a large pool of data, if I rank the Pokemon by WinShare, I get the closest approximation of a viability ranking.

I made a website recently that computes this, alongside some other stuff. You can try it here: https://orreplay.uc.r.appspot.com/

When it comes to tiering policy, I am aware that Smogon uses weighted Usage. I don't know enough details about that to have an opinion on it. But I guess I do wonder if using WinShare was ever considered. Or even just why would Usage be preferred over WinShare?

It seems to me that Usage has some fundamental flaws as a metric that WinShare does not have. Usage is affected by, well, players choosing to use something which may actually not be good. This can affect high-skill players too, so using weighted Usage doesn't totally address it. WinShare does not have this issue, since it clearly only tracks the Pokemon that actually win.

I wanted to open this discussion since I have not really seen this stat be used in general, so this isn't just about tiering policy (I also don't play modern gens anyway). If you have enough data, you can use this stat, in my opinion, to get highly accurate viability rankings. I've seen this in practice in the formats I play, but I would invite anyone to try this themselves using the website I shared above.

 

[Ruft]

It's an interesting stat, but I wonder what it actually sets out to do that weighted usage doesn't already accomplish. I would argue weighted usage already tries to address the issue you mention: if players consistently used Pokemon with a low winrate, then, by definition, they were losing more games than they won and, especially in the case of high-rated players, dropped in rating, which means their usage is weighted less heavily. The reverse is also true: Pokemon with a high winrate were used by players who won more and, as such, have a higher rating, and their usage is weighted more heavily. In the highest-rated games, which are weighted the most heavily, the most used Pokemon are those who have been most consistently winning, generally speaking.

 

[Akiak]

I think the main benefit over weighted usage is that it is not as influenced by "popularity amongst good players".

If a Pokemon is considered good and used a lot by good players, it will get a very high ranking, regardless of how well it actually performs overall.

I would agree that the benefit, specifically for tiering policy, is probably quite small. Like I said in the OP, the main reason I'm interested in the stat is in its ability to create pretty accurate viability rankings for any given format. I probably wouldn't trust weighted usage to be able to do this to the same degree, for the reason given above.

 

[vapicuno]

Plainly said, this is the total win count, which sounds like a good metric if we were playing round robin.

But remember that ladder is not round robin. It is constructed to match players similar in elo. Top players only get matched to top players, and amateurs to amateurs. A win between two top players and a win between two 1000's has the same weight. So jank can have a very high win count at low elo.

If this came as a response to the bot concern, I'm not sure it would work better either. A very skilled bot using jank could ironically prop jank up even more by winning a lot.

 

[Finchinator]

This feels unnecessary to me. If you can reach the necessary thresholds on ladder, then your usage should be counted. If you do poorly, you will drop below. If you win enough games, you will maintain it and count towards usage. I think the status quo is sufficient and largely echo what Ruft and Vapicuno stated.

 

[Akiak]

Plainly said, this is the total win count, which sounds like a good metric if we were playing round robin.

But remember that ladder is not round robin. It is constructed to match players similar in elo. Top players only get matched to top players, and amateurs to amateurs. A win between two top players and a win between two 1000's has the same weight. So jank can have a very high win count at low elo.

If this came as a response to the bot concern, I'm not sure it would work better either. A very skilled bot using jank could ironically prop jank up even more by winning a lot.

If differing skill levels was a concern still, you could do "weighted winshare". I feel like that would still be an improvement over weighted usage.

 

[toshimelonhead]

I think the main benefit over weighted usage is that it is not as influenced by "popularity amongst good players".

That influence is why Smogon produces 1825 and 1760 level usage stats. Players want to see what the good players use because that's where most of the meta influencing trends tend to come from.

 

[vapicuno]

If differing skill levels was a concern still, you could do "weighted winshare". I feel like that would still be an improvement over weighted usage.

To me, this is a philosophical question, not a quantitative improvement. Let's for the sake of argument assume ladder is a true round robin.

If the reason BL Pokemon are less used is that the fraction of viable teams with them is lower, even if all viable teams are just as good, then win count is possibly a better metric.

Still, elo is an aggregate quantifier of success over many games, and win rate is a quantifier of success over one game.

A difference appears when people use less consistent Pokemon to appear less predictable, even if the expected winrate of particular teams including an inconsistent Pokemon is lower. This is especially the case in tours, where using an uncommon Pokemon forces one's opponent to avoid fishing. Then the loss still contributes to the overall success of the player, and win count is not a good metric from this perspective.

In my less-populous tier (ADV), it is possible to repeatedly face the same opponents on ladder especially at the high end, and optimizing a ladder run might require mixing up teams.

It is not necessarily better. It depends on what you are optimizing for.

 

[Akiak]

To me, this is a philosophical question, not a quantitative improvement. Let's for the sake of argument assume ladder is a true round robin.

If the reason BL Pokemon are less used is that the fraction of viable teams with them is lower, even if all viable teams are just as good, then win count is possibly a better metric.

Still, elo is an aggregate quantifier of success over many games, and win rate is a quantifier of success over one game.

A difference appears when people use less consistent Pokemon to appear less predictable, even if the expected winrate of particular teams including an inconsistent Pokemon is lower. This is especially the case in tours, where using an uncommon Pokemon forces one's opponent to avoid fishing. Then the loss still contributes to the overall success of the player, and win count is not a good metric from this perspective.

In my less-populous tier (ADV), it is possible to repeatedly face the same opponents on ladder especially at the high end, and optimizing a ladder run might require mixing up teams.

It is not necessarily better. It depends on what you are optimizing for.

I agree that it depends on what you want out of the data.

Sidestepping this for a moment, I think that tracking Pokemon wins would be a pretty cool thing to look at, alongside the player ladder. Imagine having a leaderboard of the Pokemon with the highest wins (or weighted winshare). It would create a natural incentive for players to use certain mons to increase their placement in the ranking.

The question of wether that ranking should also be used for tiering is separate, although I do think it should be.