Make Sense Of Your Shoddy Battle Rating

X-Act

np: Biffy Clyro - Shock Shock
is a Site Content Manager Alumnusis a Programmer Alumnusis a Smogon Discord Contributor Alumnusis a Top Researcher Alumnusis a Top CAP Contributor Alumnusis a Top Tiering Contributor Alumnusis a Top Contributor Alumnusis a Smogon Media Contributor Alumnusis an Administrator Alumnus
This thread explains clearly what the ShoddyBattle rating is and what you can use it for. It is displayed in your ladder record here:


As you can see, the rating has a lower value L and an upper value U. From these two values, you can find two very important numbers related to your rating, called xbar and sigma:
Code:
 xbar = (U+L)/2
sigma = (U-L)/2
xbar is your mean, or average, rating. sigma is a number signifying the uncertainty of your rating. The larger sigma is, the more uncertain your average rating xbar is. For example, a rating displayed as being 1200-2000 is a much more uncertain rating than one displayed as being 1500-1700, even though both have the same average rating (xbar) of 1600.

The xbar value increases when you win games and decreases when you lose them. However, it does not always increase or decrease by the same amount, but proportional to how better or worse your opponent is. To put it in simple terms, if your opponent had a higher xbar value than yours before the start of the battle, your xbar value would increase by a lot if you win against him, but would only decrease by a little if you lose. Conversely, if your opponent had a lower xbar value than yours before the start of a battle, your xbar value would increase only slightly if you win against him, but would decrease drastically if you lose.

The sigma value always decreases whenever you battle, whether you win or lose. This is because the more you battle, the more the rating system gains accurate information of your Pokemon battling skills (or lack of) from the results of said battles, and hence the uncertainty of your xbar value decreases. On the other hand, if you stop battling, the sigma value would increase, indicating that your battling skills are now more uncertain, and this is where the volatility value comes in, since it is the value that governs the speed of your increase in sigma once a player stops battling.

The volatility is a measure of how consistent you are in your battles. If you tend to win against players you are expected to win against (i.e. having an xbar value lower than yours) and lose against players you are expected to lose against (i.e. having an xbar value higher than yours), your volatility would be low. A consistent player would thus have his sigma increase slower when he stops playing, because due to his previously consistent performances, his xbar value should become more uncertain at a slower rate.

Finally, a player is ranked on the ladder by his Conservative Rating Estimate (CRE). This is found as follows:
Code:
CRE = xbar - 4 * sigma
The Conservative Rating Estimate rewards players that have a high xbar (average rating) and low sigma (uncertainty of their average rating), by assigning them a larger number, and hence a higher rank on the ladder. However, while this ranking system is adequate, it has a few disadvantages, and should be replaced by a better one soon.

Earlier, I mentioned that a player is 'better' or 'worse' according to their corresponding xbar's and sigma's. In this final part of this thread, I shall provide a relatively simple approximation (compared to how it's actually calculated) to quantify this. It is written underneath in step-by-step form:

Step 1: Find the sum of the squares of both players' sigma's, add 100000 to this sum, and find the square root of the answer.
Step 2: Find the difference of the players' xbar's. Always subtract your xbar from your opponent's. Afterwards, multiply the result by 0.79 and divide by the number found in Step 1.
Step 3: Raise 10 to the power of the answer. Call this number G.
Step 4: For every game you win against the opponent, your opponent would win G games against you. Alternatively, the probability that you win against the opponent is 1/(1+G), while the probability that you lose against the opponent is G/(1+G).

Here's the equation for G in mathematical form:
Code:
G = 10^((OppXbar - YourXbar) * 0.79 / sqrt(100000 + YourSigma^2 + OppSigma^2))
Example: You have a rating of 1650 - 1738. Your opponent has a rating of 1580 - 1704.

YourXbar = (1738+1650)/2 = 1694
YourSigma = (1738-1650)/2 = 44

OppXbar = (1704+1580)/2 = 1642
OppSigma = (1704-1580)/2 = 62

Step 1: 44^2 + 62^2 = 1936 + 3844 = 5780. Adding 100000 to this we get 105780. The square root of this number is 325.238.
Step 2: 1642 - 1694 = -52. -52 * 0.79 = -41.08. -41.08 / 325.238 = -0.1263.
Step 3: 10^(-0.1263) = 0.7477. This is our G.
Step 4: For every game you win against the opponent, your opponent would win 0.7477 games. Alternatively, the probability that I win against the opponent is 1/(1+0.7477) = 57.22%, while the probability that my opponent wins against me is 0.7477/(1+0.7477) = 42.78%.
 
Yay, it's AP Stats again!

Thanks for the info. For some reason, I feel like doing confidence intervals or something...
 
Does Shoddy Battle filter your search results based on your rating?
ie. Are higher ranked players more likely to be paired up with higher ranked players
 

Caelum

qibz official stalker
is a Site Content Manager Alumnusis a Community Leader Alumnusis a Smogon Discord Contributor Alumnusis a Tiering Contributor Alumnusis a Top Contributor Alumnusis a Smogon Media Contributor Alumnusis a Battle Simulator Moderator Alumnus
Does Shoddy Battle filter your search results based on your rating?
ie. Are higher ranked players more likely to be paired up with higher ranked players
I believe the system organizes all the players currently looking for a match and pairs you with the person who's CRE is nearest to your own.
 

X-Act

np: Biffy Clyro - Shock Shock
is a Site Content Manager Alumnusis a Programmer Alumnusis a Smogon Discord Contributor Alumnusis a Top Researcher Alumnusis a Top CAP Contributor Alumnusis a Top Tiering Contributor Alumnusis a Top Contributor Alumnusis a Smogon Media Contributor Alumnusis an Administrator Alumnus
I can't edit the OP for some reason, but there are some slight mistakes there. In particular, the formula for G contains "/100000" at the end which shouldn't be there.
 
How do you even view your own rating? I can see everyone else's but my own when I'm on the server :(
 
Wow, great work on these. This should be a sticky (or kept somewhere) for information, since new battlers to shoddy would love to know what the rating stands for.
I believe the system organizes all the players currently looking for a match and pairs you with the person who's CRE is nearest to your own.
Really? didn't know that. ;)
 
I think GLIXARE just works from the current rating system to give a more accurate rating to base the ladder on than CRE.
 

Myzozoa

to find better ways to say what nobody says
is a Top Tiering Contributor Alumnusis a Past WCoP Champion
I apologize if this question has already been answered:

When GLIXARE is implemented will our ratings be reset?
 

Cathy

Banned deucer.
Tab already answered it above, but I will explain for greater certainty

The rating system -- Glicko2 -- is not going to be changing any time soon. However, Glicko2 does not assign each player a single figure which can be used to rank the players; it is necessary to impose some total ordering, and the way we have accomplished that up to this point is by introducing the CRE. The predicative percentage win metric ("GLIXARE") is just a quantity used to impose a different total ordering on the set of players -- your rating is completely unaffected; the only thing that is changed is the ordering of the leaderboard.
 

X-Act

np: Biffy Clyro - Shock Shock
is a Site Content Manager Alumnusis a Programmer Alumnusis a Smogon Discord Contributor Alumnusis a Top Researcher Alumnusis a Top CAP Contributor Alumnusis a Top Tiering Contributor Alumnusis a Top Contributor Alumnusis a Smogon Media Contributor Alumnusis an Administrator Alumnus
Just to confirm what Colin and Tab already said...

There's no need to reset the player's ratings when GLIXARE replaces CRE. Contrary to what many people seem to think, GLIXARE is NOT a rating system. It is just a way of interpreting your xbar and sigma as a single estimate for your rating... a way that is much closer to your actual playing strength than CRE. That's all. Your xbar and sigma will not be affected at all when GLIXARE is implemented, and hence the player's ratings do not need to be reset when it is implemented.

Note that this doesn't mean that the ratings won't be reset - it just means that they don't need to be reset. If we decide, for whatever reason, to reset the players' ratings at any time, we'll do it. Moreover, a resetting of the player's ratings can happen even without a change in the rating estimation method.
 
Well, I'm looking forward to seeing if my alt stays on the leaderboard or not once GLIXARE is implemented.

Thanks for the information. I knew the basics (what things mean), but not how to actually calculate expected win rate.
I also didn't know what volatility was based off of.
 

DougJustDoug

Knows the great enthusiasms
is a Site Content Manageris a Top Artistis a Programmeris a Forum Moderatoris a Top CAP Contributoris a Battle Simulator Admin Alumnusis a Smogon Discord Contributor Alumnusis a Top Tiering Contributor Alumnusis an Administrator Alumnus
Now that I have finished the Shoddy client update, and I've gotten May stats published -- I'm close to putting GLIXARE on Smogon University and CAP. You will see some public announcements when that happens.
 

VKCA

(Virtual Circus Kareoky Act)
Nice man you read my mind, I was literally about to go on the server and ask what the differences were. Thanks alot!
 
Thanks X-Act that just perfectly explains a lot! And i also have to imrove my rating now that i have learned how to find it!
 
Thanks for explaining this X-Act! Do these rankings (xbar and sigma) determine how much your rating (and therefore CRE) changes with each battle?
 

X-Act

np: Biffy Clyro - Shock Shock
is a Site Content Manager Alumnusis a Programmer Alumnusis a Smogon Discord Contributor Alumnusis a Top Researcher Alumnusis a Top CAP Contributor Alumnusis a Top Tiering Contributor Alumnusis a Top Contributor Alumnusis a Smogon Media Contributor Alumnusis an Administrator Alumnus
Thanks for explaining this X-Act! Do these rankings (xbar and sigma) determine how much your rating (and therefore CRE) changes with each battle?
Well, they are not rankings, they're just values used to determine how good or bad you are. The xbar and sigma values and those of the opponents you play against are all used to determine how your rating changes.
 

Cathy

Banned deucer.
A lot of people ask me how much their CRE will decrease by if they don't play for a day. The answer is that your deviation changes (approximately) according to this formula:

new deviation ≈ sqrt((old deviation)^2 + 30178 * (volatility)^2)

Strictly speaking, your deviation increases by this much each day whether you play or not, but if you do play, the decrease from the games you play offsets it. However, after you play your first battle in a day, your CRE may sometimes decrease, and that's because you have not offset the daily deviation increase yet, which is applied in the rating window after your first battle in a rating period. After you play more in the same rating period, you will always experience a net increase in CRE.

Here is a fully worked out example. Suppose a user has these stats:

CRE: 1449
Rating: 1521-1569
Volatility: 0.051068226...

What will his CRE be if he doesn't play for a day? First work out his deviation and mean rating:

mean rating = (1521+1569)/2 = 1545
rating deviation = (1569-1521)/2 = 24

Now apply the formula above:

new deviation = sqrt(24^2 + 30178 * 0.051068226^2)
=> new deviation ≈ 25.587167266882

The mean rating is unaffected, so we use the formula for the CRE to find his new CRE.

new CRE = 1545 - 4 * 25.587167266882 ≈ 1443

So by not playing for a day, this user's CRE will decrease from 1449 to 1443. This should give you a tangible idea of what to expect with the daily deviation increases.
 
I just have to say one thing, i'm disturbed that its not in LaTeX!

Other than that, this is amazing! I always wondered what CRE really meant. It took me a while to get the percentage stuff, but it makes sense now.
 
How do you even view your own rating? I can see everyone else's but my own when I'm on the server :(

Go to the memberlist, search for your name, (the names are in alphabetical order) and then right click and click 'View Record''
 

Users Who Are Viewing This Thread (Users: 1, Guests: 0)

Top