1. Welcome to Smogon Forums! Please take a minute to read the rules.

# Make Sense Of Your Shoddy Battle Rating

Discussion in 'Shoddy Battle and Pokémon Lab' started by X-Act, May 28, 2009.

1. ### X-Actnp: Biffy Clyro - Shock Shock

Joined:
Feb 17, 2006
Messages:
4,675
This thread explains clearly what the ShoddyBattle rating is and what you can use it for. It is displayed in your ladder record here:

As you can see, the rating has a lower value L and an upper value U. From these two values, you can find two very important numbers related to your rating, called xbar and sigma:
Code:
``` xbar = (U+L)/2
sigma = (U-L)/2```
xbar is your mean, or average, rating. sigma is a number signifying the uncertainty of your rating. The larger sigma is, the more uncertain your average rating xbar is. For example, a rating displayed as being 1200-2000 is a much more uncertain rating than one displayed as being 1500-1700, even though both have the same average rating (xbar) of 1600.

The xbar value increases when you win games and decreases when you lose them. However, it does not always increase or decrease by the same amount, but proportional to how better or worse your opponent is. To put it in simple terms, if your opponent had a higher xbar value than yours before the start of the battle, your xbar value would increase by a lot if you win against him, but would only decrease by a little if you lose. Conversely, if your opponent had a lower xbar value than yours before the start of a battle, your xbar value would increase only slightly if you win against him, but would decrease drastically if you lose.

The sigma value always decreases whenever you battle, whether you win or lose. This is because the more you battle, the more the rating system gains accurate information of your Pokemon battling skills (or lack of) from the results of said battles, and hence the uncertainty of your xbar value decreases. On the other hand, if you stop battling, the sigma value would increase, indicating that your battling skills are now more uncertain, and this is where the volatility value comes in, since it is the value that governs the speed of your increase in sigma once a player stops battling.

The volatility is a measure of how consistent you are in your battles. If you tend to win against players you are expected to win against (i.e. having an xbar value lower than yours) and lose against players you are expected to lose against (i.e. having an xbar value higher than yours), your volatility would be low. A consistent player would thus have his sigma increase slower when he stops playing, because due to his previously consistent performances, his xbar value should become more uncertain at a slower rate.

Finally, a player is ranked on the ladder by his Conservative Rating Estimate (CRE). This is found as follows:
Code:
`CRE = xbar - 4 * sigma`
The Conservative Rating Estimate rewards players that have a high xbar (average rating) and low sigma (uncertainty of their average rating), by assigning them a larger number, and hence a higher rank on the ladder. However, while this ranking system is adequate, it has a few disadvantages, and should be replaced by a better one soon.

Earlier, I mentioned that a player is 'better' or 'worse' according to their corresponding xbar's and sigma's. In this final part of this thread, I shall provide a relatively simple approximation (compared to how it's actually calculated) to quantify this. It is written underneath in step-by-step form:

Step 1: Find the sum of the squares of both players' sigma's, add 100000 to this sum, and find the square root of the answer.
Step 2: Find the difference of the players' xbar's. Always subtract your xbar from your opponent's. Afterwards, multiply the result by 0.79 and divide by the number found in Step 1.
Step 3: Raise 10 to the power of the answer. Call this number G.
Step 4: For every game you win against the opponent, your opponent would win G games against you. Alternatively, the probability that you win against the opponent is 1/(1+G), while the probability that you lose against the opponent is G/(1+G).

Here's the equation for G in mathematical form:
Code:
`G = 10^((OppXbar - YourXbar) * 0.79 / sqrt(100000 + YourSigma^2 + OppSigma^2))`
Example: You have a rating of 1650 - 1738. Your opponent has a rating of 1580 - 1704.

YourXbar = (1738+1650)/2 = 1694
YourSigma = (1738-1650)/2 = 44

OppXbar = (1704+1580)/2 = 1642
OppSigma = (1704-1580)/2 = 62

Step 1: 44^2 + 62^2 = 1936 + 3844 = 5780. Adding 100000 to this we get 105780. The square root of this number is 325.238.
Step 2: 1642 - 1694 = -52. -52 * 0.79 = -41.08. -41.08 / 325.238 = -0.1263.
Step 3: 10^(-0.1263) = 0.7477. This is our G.
Step 4: For every game you win against the opponent, your opponent would win 0.7477 games. Alternatively, the probability that I win against the opponent is 1/(1+0.7477) = 57.22%, while the probability that my opponent wins against me is 0.7477/(1+0.7477) = 42.78%.
2. ### StarmanXL

Joined:
Jul 1, 2007
Messages:
1,449
Yay, it's AP Stats again!

Thanks for the info. For some reason, I feel like doing confidence intervals or something...
3. ### iKitsune

Joined:
Jul 8, 2008
Messages:
805
ie. Are higher ranked players more likely to be paired up with higher ranked players
4. ### Caelumqibz official stalker

Joined:
Apr 5, 2008
Messages:
1,656
I believe the system organizes all the players currently looking for a match and pairs you with the person who's CRE is nearest to your own.
5. ### X-Actnp: Biffy Clyro - Shock Shock

Joined:
Feb 17, 2006
Messages:
4,675
I can't edit the OP for some reason, but there are some slight mistakes there. In particular, the formula for G contains "/100000" at the end which shouldn't be there.

Joined:
Apr 13, 2009
Messages:
2,946
X-Act, you're a genius. My head would just explode if I tried to do half the stuff you do.
7. ### d2m

Joined:
Apr 18, 2009
Messages:
766
How do you even view your own rating? I can see everyone else's but my own when I'm on the server :(
8. ### A.P.

Joined:
Jul 14, 2008
Messages:
731
Are you planning on implanting GLIXARE anytime soon, or only on SB2?
9. ### tabʕ•͡ᴥ•ʔ

Joined:
Mar 4, 2008
Messages:
960
d2m, type /record <your name> into chat to see it.

Joined:
Apr 14, 2009
Messages:
3,548
Wow, great work on these. This should be a sticky (or kept somewhere) for information, since new battlers to shoddy would love to know what the rating stands for.
Really? didn't know that. ;)
11. ### X-Actnp: Biffy Clyro - Shock Shock

Joined:
Feb 17, 2006
Messages:
4,675
Doug should be implementing GLIXARE soon on ShoddyBattle 1. When that happens, this thread would be updated.
12. ### TheFourthChaser#TimeForChangeSmogon VGC16 Champ

Joined:
Oct 13, 2008
Messages:
1,695
So would all current CREs and Ratings be reset when this is implemented? (Seems like a stupid question to ask but I'm wondering)
13. ### tabʕ•͡ᴥ•ʔ

Joined:
Mar 4, 2008
Messages:
960
I think GLIXARE just works from the current rating system to give a more accurate rating to base the ladder on than CRE.
14. ### Myzozoaa choir master sings sharp so others hit the right note

Joined:
Jan 31, 2009
Messages:
1,851

When GLIXARE is implemented will our ratings be reset?
15. ### CathyBanned deucer.

Joined:
Jul 11, 2007
Messages:
1,060
Tab already answered it above, but I will explain for greater certainty

The rating system -- Glicko2 -- is not going to be changing any time soon. However, Glicko2 does not assign each player a single figure which can be used to rank the players; it is necessary to impose some total ordering, and the way we have accomplished that up to this point is by introducing the CRE. The predicative percentage win metric ("GLIXARE") is just a quantity used to impose a different total ordering on the set of players -- your rating is completely unaffected; the only thing that is changed is the ordering of the leaderboard.
16. ### X-Actnp: Biffy Clyro - Shock Shock

Joined:
Feb 17, 2006
Messages:
4,675
Just to confirm what Colin and Tab already said...

There's no need to reset the player's ratings when GLIXARE replaces CRE. Contrary to what many people seem to think, GLIXARE is NOT a rating system. It is just a way of interpreting your xbar and sigma as a single estimate for your rating... a way that is much closer to your actual playing strength than CRE. That's all. Your xbar and sigma will not be affected at all when GLIXARE is implemented, and hence the player's ratings do not need to be reset when it is implemented.

Note that this doesn't mean that the ratings won't be reset - it just means that they don't need to be reset. If we decide, for whatever reason, to reset the players' ratings at any time, we'll do it. Moreover, a resetting of the player's ratings can happen even without a change in the rating estimation method.
17. ### paramylodon

Joined:
May 24, 2007
Messages:
238
Well, I'm looking forward to seeing if my alt stays on the leaderboard or not once GLIXARE is implemented.

Thanks for the information. I knew the basics (what things mean), but not how to actually calculate expected win rate.
I also didn't know what volatility was based off of.
18. ### DougJustDougKnows the great enthusiasmsCAP Leader

Joined:
Jun 26, 2007
Messages:
3,487
Now that I have finished the Shoddy client update, and I've gotten May stats published -- I'm close to putting GLIXARE on Smogon University and CAP. You will see some public announcements when that happens.
19. ### VKCA(Virtual Circus Kareoky Act)

Joined:
Jan 8, 2009
Messages:
3,694
Nice man you read my mind, I was literally about to go on the server and ask what the differences were. Thanks alot!
20. ### odisseas

Joined:
Jun 5, 2009
Messages:
48
Thanks X-Act that just perfectly explains a lot! And i also have to imrove my rating now that i have learned how to find it!
21. ### Kingdrom

Joined:
Oct 18, 2007
Messages:
380
Thanks for explaining this X-Act! Do these rankings (xbar and sigma) determine how much your rating (and therefore CRE) changes with each battle?
22. ### X-Actnp: Biffy Clyro - Shock Shock

Joined:
Feb 17, 2006
Messages:
4,675
Well, they are not rankings, they're just values used to determine how good or bad you are. The xbar and sigma values and those of the opponents you play against are all used to determine how your rating changes.
23. ### CathyBanned deucer.

Joined:
Jul 11, 2007
Messages:
1,060
A lot of people ask me how much their CRE will decrease by if they don't play for a day. The answer is that your deviation changes (approximately) according to this formula:

new deviation ≈ sqrt((old deviation)^2 + 30178 * (volatility)^2)

Strictly speaking, your deviation increases by this much each day whether you play or not, but if you do play, the decrease from the games you play offsets it. However, after you play your first battle in a day, your CRE may sometimes decrease, and that's because you have not offset the daily deviation increase yet, which is applied in the rating window after your first battle in a rating period. After you play more in the same rating period, you will always experience a net increase in CRE.

Here is a fully worked out example. Suppose a user has these stats:

CRE: 1449
Rating: 1521-1569
Volatility: 0.051068226...

What will his CRE be if he doesn't play for a day? First work out his deviation and mean rating:

mean rating = (1521+1569)/2 = 1545
rating deviation = (1569-1521)/2 = 24

Now apply the formula above:

new deviation = sqrt(24^2 + 30178 * 0.051068226^2)
=> new deviation ≈ 25.587167266882

The mean rating is unaffected, so we use the formula for the CRE to find his new CRE.

new CRE = 1545 - 4 * 25.587167266882 ≈ 1443

So by not playing for a day, this user's CRE will decrease from 1449 to 1443. This should give you a tangible idea of what to expect with the daily deviation increases.
24. ### PI-Dimension

Joined:
Jul 31, 2009
Messages:
254
I just have to say one thing, i'm disturbed that its not in LaTeX!

Other than that, this is amazing! I always wondered what CRE really meant. It took me a while to get the percentage stuff, but it makes sense now.
25. ### SkyDragon

Joined:
Sep 20, 2009
Messages:
7

Go to the memberlist, search for your name, (the names are in alphabetical order) and then right click and click 'View Record''