Make Sense Of Your Shoddy Battle Rating

X-Act** · May 28th, 2009, 11:25:47 AM

This thread explains clearly what the ShoddyBattle rating is and what you can use it for. It is displayed in your ladder record here:

As you can see, the rating has a lower value L and an upper value U. From these two values, you can find two very important numbers related to your rating, called xbar and sigma:

Code:

 xbar = (U+L)/2
sigma = (U-L)/2

xbar is your mean, or average, rating. sigma is a number signifying the uncertainty of your rating. The larger sigma is, the more uncertain your average rating xbar is. For example, a rating displayed as being 1200-2000 is a much more uncertain rating than one displayed as being 1500-1700, even though both have the same average rating (xbar) of 1600.

The xbar value increases when you win games and decreases when you lose them. However, it does not always increase or decrease by the same amount, but proportional to how better or worse your opponent is. To put it in simple terms, if your opponent had a higher xbar value than yours before the start of the battle, your xbar value would increase by a lot if you win against him, but would only decrease by a little if you lose. Conversely, if your opponent had a lower xbar value than yours before the start of a battle, your xbar value would increase only slightly if you win against him, but would decrease drastically if you lose.

The sigma value always decreases whenever you battle, whether you win or lose. This is because the more you battle, the more the rating system gains accurate information of your Pokemon battling skills (or lack of) from the results of said battles, and hence the uncertainty of your xbar value decreases. On the other hand, if you stop battling, the sigma value would increase, indicating that your battling skills are now more uncertain, and this is where the volatility value comes in, since it is the value that governs the speed of your increase in sigma once a player stops battling.

The volatility is a measure of how consistent you are in your battles. If you tend to win against players you are expected to win against (i.e. having an xbar value lower than yours) and lose against players you are expected to lose against (i.e. having an xbar value higher than yours), your volatility would be low. A consistent player would thus have his sigma increase slower when he stops playing, because due to his previously consistent performances, his xbar value should become more uncertain at a slower rate.

Finally, a player is ranked on the ladder by his Conservative Rating Estimate (CRE). This is found as follows:

Code:

CRE = xbar - 4 * sigma

The Conservative Rating Estimate rewards players that have a high xbar (average rating) and low sigma (uncertainty of their average rating), by assigning them a larger number, and hence a higher rank on the ladder. However, while this ranking system is adequate, it has a few disadvantages, and should be replaced by a better one soon.

Earlier, I mentioned that a player is 'better' or 'worse' according to their corresponding xbar's and sigma's. In this final part of this thread, I shall provide a relatively simple approximation (compared to how it's actually calculated) to quantify this. It is written underneath in step-by-step form:

Step 1: Find the sum of the squares of both players' sigma's, add 100000 to this sum, and find the square root of the answer.
Step 2: Find the difference of the players' xbar's. Always subtract your xbar from your opponent's. Afterwards, multiply the result by 0.79 and divide by the number found in Step 1.
Step 3: Raise 10 to the power of the answer. Call this number G.
Step 4: For every game you win against the opponent, your opponent would win G games against you. Alternatively, the probability that you win against the opponent is 1/(1+G), while the probability that you lose against the opponent is G/(1+G).

Here's the equation for G in mathematical form:

Code:

G = 10^((OppXbar - YourXbar) * 0.79 / sqrt(100000 + YourSigma^2 + OppSigma^2))

Example: You have a rating of 1650 - 1738. Your opponent has a rating of 1580 - 1704.

YourXbar = (1738+1650)/2 = 1694
YourSigma = (1738-1650)/2 = 44

OppXbar = (1704+1580)/2 = 1642
OppSigma = (1704-1580)/2 = 62

Step 1: 44^2 + 62^2 = 1936 + 3844 = 5780. Adding 100000 to this we get 105780. The square root of this number is 325.238.
Step 2: 1642 - 1694 = -52. -52 * 0.79 = -41.08. -41.08 / 325.238 = -0.1263.
Step 3: 10^(-0.1263) = 0.7477. This is our G.
Step 4: For every game you win against the opponent, your opponent would win 0.7477 games. Alternatively, the probability that I win against the opponent is 1/(1+0.7477) = 57.22%, while the probability that my opponent wins against me is 0.7477/(1+0.7477) = 42.78%.

StarmanXL · May 28th, 2009, 11:45:07 AM

Yay, it's AP Stats again!

Thanks for the info. For some reason, I feel like doing confidence intervals or something...

iKitsune · May 28th, 2009, 12:50:53 PM

Does Shoddy Battle filter your search results based on your rating?
ie. Are higher ranked players more likely to be paired up with higher ranked players

Caelum · May 28th, 2009, 1:40:13 PM

Quote:

Originally Posted by Fat iKitsune

Does Shoddy Battle filter your search results based on your rating?
ie. Are higher ranked players more likely to be paired up with higher ranked players

I believe the system organizes all the players currently looking for a match and pairs you with the person who's CRE is nearest to your own.

X-Act** · May 28th, 2009, 3:12:28 PM

I can't edit the OP for some reason, but there are some slight mistakes there. In particular, the formula for G contains "/100000" at the end which shouldn't be there.

Son_of_Shadoo · May 28th, 2009, 3:19:32 PM

X-Act, you're a genius. My head would just explode if I tried to do half the stuff you do.

d2m · May 28th, 2009, 3:21:43 PM

How do you even view your own rating? I can see everyone else's but my own when I'm on the server :(

031ap · May 28th, 2009, 3:37:09 PM

Are you planning on implanting GLIXARE anytime soon, or only on SB2?

Tab · May 28th, 2009, 3:49:12 PM

d2m, type /record <your name> into chat to see it.

Blue_Tornado · May 28th, 2009, 4:00:23 PM

Wow, great work on these. This should be a sticky (or kept somewhere) for information, since new battlers to shoddy would love to know what the rating stands for.

Quote:

I believe the system organizes all the players currently looking for a match and pairs you with the person who's CRE is nearest to your own.

Really? didn't know that. ;)

X-Act** · May 29th, 2009, 2:36:42 AM

Quote:

Originally Posted by Fat ap13095

Are you planning on implanting GLIXARE anytime soon, or only on SB2?

Doug should be implementing GLIXARE soon on ShoddyBattle 1. When that happens, this thread would be updated.

TheFourthChaser · May 30th, 2009, 2:55:43 PM

Quote:

Originally Posted by Fat X-Act

Doug should be implementing GLIXARE soon on ShoddyBattle 1. When that happens, this thread would be updated.

So would all current CREs and Ratings be reset when this is implemented? (Seems like a stupid question to ask but I'm wondering)

Tab · May 30th, 2009, 3:31:22 PM

I think GLIXARE just works from the current rating system to give a more accurate rating to base the ladder on than CRE.

brksocsc · May 30th, 2009, 8:34:06 PM

I apologize if this question has already been answered:

When GLIXARE is implemented will our ratings be reset?

Colin** · May 30th, 2009, 9:03:50 PM

Tab already answered it above, but I will explain for greater certainty

The rating system -- Glicko2 -- is not going to be changing any time soon. However, Glicko2 does not assign each player a single figure which can be used to rank the players; it is necessary to impose some total ordering, and the way we have accomplished that up to this point is by introducing the CRE. The predicative percentage win metric ("GLIXARE") is just a quantity used to impose a different total ordering on the set of players -- your rating is completely unaffected; the only thing that is changed is the ordering of the leaderboard.

X-Act** · Jun 1st, 2009, 3:58:33 AM

Just to confirm what Colin and Tab already said...

There's no need to reset the player's ratings when GLIXARE replaces CRE. Contrary to what many people seem to think, GLIXARE is NOT a rating system. It is just a way of interpreting your xbar and sigma as a single estimate for your rating... a way that is much closer to your actual playing strength than CRE. That's all. Your xbar and sigma will not be affected at all when GLIXARE is implemented, and hence the player's ratings do not need to be reset when it is implemented.

Note that this doesn't mean that the ratings won't be reset - it just means that they don't need to be reset. If we decide, for whatever reason, to reset the players' ratings at any time, we'll do it. Moreover, a resetting of the player's ratings can happen even without a change in the rating estimation method.

paramylodon · Jun 1st, 2009, 11:09:22 PM

Well, I'm looking forward to seeing if my alt stays on the leaderboard or not once GLIXARE is implemented.

Thanks for the information. I knew the basics (what things mean), but not how to actually calculate expected win rate.
I also didn't know what volatility was based off of.

DougJustDoug** · Jun 4th, 2009, 2:27:44 PM

Now that I have finished the Shoddy client update, and I've gotten May stats published -- I'm close to putting GLIXARE on Smogon University and CAP. You will see some public announcements when that happens.

Eye · Jun 4th, 2009, 4:38:47 PM

Nice man you read my mind, I was literally about to go on the server and ask what the differences were. Thanks alot!

odisseas · Jun 5th, 2009, 8:01:21 AM

Thanks X-Act that just perfectly explains a lot! And i also have to imrove my rating now that i have learned how to find it!

Kingdrom · Jun 6th, 2009, 1:48:43 PM

Thanks for explaining this X-Act! Do these rankings (xbar and sigma) determine how much your rating (and therefore CRE) changes with each battle?

X-Act** · Jun 7th, 2009, 12:46:34 PM

Quote:

Originally Posted by Fat Kingdrom

Thanks for explaining this X-Act! Do these rankings (xbar and sigma) determine how much your rating (and therefore CRE) changes with each battle?

Well, they are not rankings, they're just values used to determine how good or bad you are. The xbar and sigma values and those of the opponents you play against are all used to determine how your rating changes.

Colin** · Jun 9th, 2009, 11:31:45 AM

A lot of people ask me how much their CRE will decrease by if they don't play for a day. The answer is that your deviation changes (approximately) according to this formula:

new deviation ≈ sqrt((old deviation)^2 + 30178 * (volatility)^2)

Strictly speaking, your deviation increases by this much each day whether you play or not, but if you do play, the decrease from the games you play offsets it. However, after you play your first battle in a day, your CRE may sometimes decrease, and that's because you have not offset the daily deviation increase yet, which is applied in the rating window after your first battle in a rating period. After you play more in the same rating period, you will always experience a net increase in CRE.

Here is a fully worked out example. Suppose a user has these stats:

CRE: 1449
Rating: 1521-1569
Volatility: 0.051068226...

What will his CRE be if he doesn't play for a day? First work out his deviation and mean rating:

mean rating = (1521+1569)/2 = 1545
rating deviation = (1569-1521)/2 = 24

Now apply the formula above:

new deviation = sqrt(24^2 + 30178 * 0.051068226^2)
=> new deviation ≈ 25.587167266882

The mean rating is unaffected, so we use the formula for the CRE to find his new CRE.

new CRE = 1545 - 4 * 25.587167266882 ≈ 1443

So by not playing for a day, this user's CRE will decrease from 1449 to 1443. This should give you a tangible idea of what to expect with the daily deviation increases.

PI-Dimension · Aug 7th, 2009, 11:46:06 PM

I just have to say one thing, i'm disturbed that its not in LaTeX!

Other than that, this is amazing! I always wondered what CRE really meant. It took me a while to get the percentage stuff, but it makes sense now.

SkyDragon · Sep 20th, 2009, 6:48:37 PM

Quote:

Originally Posted by Fat d2m

How do you even view your own rating? I can see everyone else's but my own when I'm on the server :(

Go to the memberlist, search for your name, (the names are in alphabetical order) and then right click and click 'View Record''

May 28th, 2009, 11:25:47 AM	#1
X-Act** Everywhere I look someone dies - Wonder when it's my turn. Join Date: Feb 2006 Location: Malta Posts: 4,488	Make Sense Of Your Shoddy Battle Rating This thread explains clearly what the ShoddyBattle rating is and what you can use it for. It is displayed in your ladder record here: As you can see, the rating has a lower value L and an upper value U. From these two values, you can find two very important numbers related to your rating, called xbar and sigma: Code: xbar = (U+L)/2 sigma = (U-L)/2 xbar is your mean, or average, rating. sigma is a number signifying the uncertainty of your rating. The larger sigma is, the more uncertain your average rating xbar is. For example, a rating displayed as being 1200-2000 is a much more uncertain rating than one displayed as being 1500-1700, even though both have the same average rating (xbar) of 1600. The xbar value increases when you win games and decreases when you lose them. However, it does not always increase or decrease by the same amount, but proportional to how better or worse your opponent is. To put it in simple terms, if your opponent had a higher xbar value than yours before the start of the battle, your xbar value would increase by a lot if you win against him, but would only decrease by a little if you lose. Conversely, if your opponent had a lower xbar value than yours before the start of a battle, your xbar value would increase only slightly if you win against him, but would decrease drastically if you lose. The sigma value always decreases whenever you battle, whether you win or lose. This is because the more you battle, the more the rating system gains accurate information of your Pokemon battling skills (or lack of) from the results of said battles, and hence the uncertainty of your xbar value decreases. On the other hand, if you stop battling, the sigma value would increase, indicating that your battling skills are now more uncertain, and this is where the volatility value comes in, since it is the value that governs the speed of your increase in sigma once a player stops battling. The volatility is a measure of how consistent you are in your battles. If you tend to win against players you are expected to win against (i.e. having an xbar value lower than yours) and lose against players you are expected to lose against (i.e. having an xbar value higher than yours), your volatility would be low. A consistent player would thus have his sigma increase slower when he stops playing, because due to his previously consistent performances, his xbar value should become more uncertain at a slower rate. Finally, a player is ranked on the ladder by his Conservative Rating Estimate (CRE). This is found as follows: Code: CRE = xbar - 4 * sigma The Conservative Rating Estimate rewards players that have a high xbar (average rating) and low sigma (uncertainty of their average rating), by assigning them a larger number, and hence a higher rank on the ladder. However, while this ranking system is adequate, it has a few disadvantages, and should be replaced by a better one soon. Earlier, I mentioned that a player is 'better' or 'worse' according to their corresponding xbar's and sigma's. In this final part of this thread, I shall provide a relatively simple approximation (compared to how it's actually calculated) to quantify this. It is written underneath in step-by-step form: Step 1: Find the sum of the squares of both players' sigma's, add 100000 to this sum, and find the square root of the answer. Step 2: Find the difference of the players' xbar's. Always subtract your xbar from your opponent's. Afterwards, multiply the result by 0.79 and divide by the number found in Step 1. Step 3: Raise 10 to the power of the answer. Call this number G. Step 4: For every game you win against the opponent, your opponent would win G games against you. Alternatively, the probability that you win against the opponent is 1/(1+G), while the probability that you lose against the opponent is G/(1+G). Here's the equation for G in mathematical form: Code: G = 10^((OppXbar - YourXbar) * 0.79 / sqrt(100000 + YourSigma^2 + OppSigma^2)) Example: You have a rating of 1650 - 1738. Your opponent has a rating of 1580 - 1704. YourXbar = (1738+1650)/2 = 1694 YourSigma = (1738-1650)/2 = 44 OppXbar = (1704+1580)/2 = 1642 OppSigma = (1704-1580)/2 = 62 Step 1: 44^2 + 62^2 = 1936 + 3844 = 5780. Adding 100000 to this we get 105780. The square root of this number is 325.238. Step 2: 1642 - 1694 = -52. -52 * 0.79 = -41.08. -41.08 / 325.238 = -0.1263. Step 3: 10^(-0.1263) = 0.7477. This is our G. Step 4: For every game you win against the opponent, your opponent would win 0.7477 games. Alternatively, the probability that I win against the opponent is 1/(1+0.7477) = 57.22%, while the probability that my opponent wins against me is 0.7477/(1+0.7477) = 42.78%. __________________ http://users.smogon.com/X-Act For all your Pokemon needs (and more!) including: the Defensive EVs applet, the Probabilities of Breeding IVs in Pokemon applet, and the Ratings of Pokemon Base Stats applet. And also the IV to PID applet! Last edited by X-Act; May 29th, 2009 at 5:26:41 AM.

May 28th, 2009, 11:45:07 AM	#2
StarmanXL Join Date: Jul 2007 Location: Looking for his inspiration Posts: 998	Yay, it's AP Stats again! Thanks for the info. For some reason, I feel like doing confidence intervals or something... __________________ D/P FC: 4596 5943 1520 Why aren't you playing doubles yet? http://www.smogon.com/dp/articles/double_battles

May 28th, 2009, 12:50:53 PM	#3
iKitsune Join Date: Jul 2008 Location: #Stark Posts: 658	Does Shoddy Battle filter your search results based on your rating? ie. Are higher ranked players more likely to be paired up with higher ranked players __________________ <&GEC> crazy dont like you baby Kiss my bloodclot! yeah you know what i mean

May 28th, 2009, 3:12:28 PM	#5
X-Act** Everywhere I look someone dies - Wonder when it's my turn. Join Date: Feb 2006 Location: Malta Posts: 4,488	I can't edit the OP for some reason, but there are some slight mistakes there. In particular, the formula for G contains "/100000" at the end which shouldn't be there. __________________ http://users.smogon.com/X-Act For all your Pokemon needs (and more!) including: the Defensive EVs applet, the Probabilities of Breeding IVs in Pokemon applet, and the Ratings of Pokemon Base Stats applet. And also the IV to PID applet!

May 28th, 2009, 3:37:09 PM	#8
031ap Join Date: Jul 2008 Location: In my house - when I'm not somewhere else. Posts: 590	Are you planning on implanting GLIXARE anytime soon, or only on SB2? __________________ Avatar by: Teh_1337ness. Highest Ranking on UU: #3 Highest Rating on UU: 1648

May 28th, 2009, 3:19:32 PM	#6
Son_of_Shadoo Join Date: Apr 2009 Posts: 153	X-Act, you're a genius. My head would just explode if I tried to do half the stuff you do.

May 28th, 2009, 3:21:43 PM	#7
d2m Join Date: Apr 2009 Posts: 554	How do you even view your own rating? I can see everyone else's but my own when I'm on the server :(

May 28th, 2009, 3:49:12 PM	#9
Tab mostly gone Join Date: Mar 2008 Location: England Posts: 375	d2m, type /record <your name> into chat to see it. __________________ [SIGPIC][/SIGPIC]

May 30th, 2009, 3:31:22 PM	#13
Tab mostly gone Join Date: Mar 2008 Location: England Posts: 375	I think GLIXARE just works from the current rating system to give a more accurate rating to base the ladder on than CRE. __________________ [SIGPIC][/SIGPIC]

May 30th, 2009, 8:34:06 PM	#14
brksocsc Join Date: Jan 2009 Posts: 106	I apologize if this question has already been answered: When GLIXARE is implemented will our ratings be reset?

May 30th, 2009, 9:03:50 PM	#15
Colin** Administrator Join Date: Jul 2007 Location: Edmonton, Alberta, Canada Posts: 722	Tab already answered it above, but I will explain for greater certainty The rating system -- Glicko2 -- is not going to be changing any time soon. However, Glicko2 does not assign each player a single figure which can be used to rank the players; it is necessary to impose some total ordering, and the way we have accomplished that up to this point is by introducing the CRE. The predicative percentage win metric ("GLIXARE") is just a quantity used to impose a different total ordering on the set of players -- your rating is completely unaffected; the only thing that is changed is the ordering of the leaderboard. Last edited by Colin; May 30th, 2009 at 9:16:09 PM.

Jun 1st, 2009, 3:58:33 AM	#16
X-Act** Everywhere I look someone dies - Wonder when it's my turn. Join Date: Feb 2006 Location: Malta Posts: 4,488	Just to confirm what Colin and Tab already said... There's no need to reset the player's ratings when GLIXARE replaces CRE. Contrary to what many people seem to think, GLIXARE is NOT a rating system. It is just a way of interpreting your xbar and sigma as a single estimate for your rating... a way that is much closer to your actual playing strength than CRE. That's all. Your xbar and sigma will not be affected at all when GLIXARE is implemented, and hence the player's ratings do not need to be reset when it is implemented. Note that this doesn't mean that the ratings won't be reset - it just means that they don't need to be reset. If we decide, for whatever reason, to reset the players' ratings at any time, we'll do it. Moreover, a resetting of the player's ratings can happen even without a change in the rating estimation method. __________________ http://users.smogon.com/X-Act For all your Pokemon needs (and more!) including: the Defensive EVs applet, the Probabilities of Breeding IVs in Pokemon applet, and the Ratings of Pokemon Base Stats applet. And also the IV to PID applet!

Jun 1st, 2009, 11:09:22 PM	#17
paramylodon Join Date: May 2007 Posts: 238	Well, I'm looking forward to seeing if my alt stays on the leaderboard or not once GLIXARE is implemented. Thanks for the information. I knew the basics (what things mean), but not how to actually calculate expected win rate. I also didn't know what volatility was based off of.

Jun 4th, 2009, 2:27:44 PM	#18
DougJustDoug** Knows the great enthusiasms Join Date: Jun 2007 Location: Houston, TX Posts: 2,130	Now that I have finished the Shoddy client update, and I've gotten May stats published -- I'm close to putting GLIXARE on Smogon University and CAP. You will see some public announcements when that happens. __________________ My Art Thread: ArtJustArt - The Art of DougJustDoug

Jun 4th, 2009, 4:38:47 PM	#19
Eye Join Date: Jan 2009 Location: iceland Posts: 403	Nice man you read my mind, I was literally about to go on the server and ask what the differences were. Thanks alot!

Jun 5th, 2009, 8:01:21 AM	#20
odisseas Join Date: Jun 2009 Location: Greece Posts: 34	Thanks X-Act that just perfectly explains a lot! And i also have to imrove my rating now that i have learned how to find it!

Jun 6th, 2009, 1:48:43 PM	#21
Kingdrom Join Date: Oct 2007 Location: Advertising services Posts: 378	Thanks for explaining this X-Act! Do these rankings (xbar and sigma) determine how much your rating (and therefore CRE) changes with each battle?

Jun 9th, 2009, 11:31:45 AM	#23
Colin** Administrator Join Date: Jul 2007 Location: Edmonton, Alberta, Canada Posts: 722	A lot of people ask me how much their CRE will decrease by if they don't play for a day. The answer is that your deviation changes (approximately) according to this formula: new deviation ≈ sqrt((old deviation)^2 + 30178 * (volatility)^2) Strictly speaking, your deviation increases by this much each day whether you play or not, but if you do play, the decrease from the games you play offsets it. However, after you play your first battle in a day, your CRE may sometimes decrease, and that's because you have not offset the daily deviation increase yet, which is applied in the rating window after your first battle in a rating period. After you play more in the same rating period, you will always experience a net increase in CRE. Here is a fully worked out example. Suppose a user has these stats: CRE: 1449 Rating: 1521-1569 Volatility: 0.051068226... What will his CRE be if he doesn't play for a day? First work out his deviation and mean rating: mean rating = (1521+1569)/2 = 1545 rating deviation = (1569-1521)/2 = 24 Now apply the formula above: new deviation = sqrt(24^2 + 30178 * 0.051068226^2) => new deviation ≈ 25.587167266882 The mean rating is unaffected, so we use the formula for the CRE to find his new CRE. new CRE = 1545 - 4 * 25.587167266882 ≈ 1443 So by not playing for a day, this user's CRE will decrease from 1449 to 1443. This should give you a tangible idea of what to expect with the daily deviation increases. Last edited by Colin; Jun 9th, 2009 at 12:10:53 PM.

Aug 7th, 2009, 11:46:06 PM	#24
PI-Dimension Join Date: Jul 2009 Posts: 7	I just have to say one thing, i'm disturbed that its not in LaTeX! Other than that, this is amazing! I always wondered what CRE really meant. It took me a while to get the percentage stuff, but it makes sense now.