1. Welcome to Smogon Forums! Please take a minute to read the rules.
  2. New to the forums? Check out our Mentorship Program!
    Our mentors will answer your questions and help you become a part of the community!

Data Official Smogon University Usage Statistics Discussion Thread

Discussion in 'Competitive Discussion' started by Antar, Apr 3, 2014.

Thread Status:
Not open for further replies.
  1. Antar

    Antar That's Dr. Antar to you
    is a Battle Server Administratoris a Programmeris a Super Moderatoris a Community Contributor
    Official Data Miner

    Joined:
    Feb 17, 2010
    Messages:
    3,036

    In recent months, there has been a massive explosion of activity on Pokemon Showdown. While that's great and all, it has led to some problems. First, it takes a lot longer to process the stats than it did when we were dealing with a Pokemon Online server running off my desktop computer (Comcast did not care for that at all, lol). Secondly, there are now waay more active metagames than ever before. This is a good thing, by and large, but it means that my statistics threads have grown so large that it takes my browser a full minute to load the thread sometimes.

    Finally, we're attracting a bunch more players who are new to competitive Pokemon. Considering we use usage stats for tiering, this is kind of a big problem, and what we've decided to do about that is to use weighted stats for tiering, with baselines significantly above that of a "new" or "average" player. Read more about that here. But what that means is that now I need to not only prepare "regular" stats for each metagame, I need to prepare stats at a variety of baseline levels, and putting that all into a single thread would be a *nightmare.*

    So instead, i'm doing away with the monthly stats threads, and we're going to have a single continuous thread for statistics discussion. The previous rules apply, namely:

    and I'll announce each month when the stats are "up."

    Feel free to ask any questions you have about how things are calculated, but be sure to first check the FAQ directly below this post.

    Enjoy, data junkies!
    ROMaster2 and DineshThePoet like this.
  2. Antar

    Antar That's Dr. Antar to you
    is a Battle Server Administratoris a Programmeris a Super Moderatoris a Community Contributor
    Official Data Miner

    Joined:
    Feb 17, 2010
    Messages:
    3,036
    Frequently Asked Questions
    1. I can't load that link!
      That stinks! You must be on a network that blocks port 8080. Unfortunately, there's nothing I can do about that right now. Eventually, we might move the stats to their own dedicated web server, and that will probably fix your problem.
    2. What's this business with "Raw" and "Real?"
    3. How are usage stats weighted?
      Every player on Pokemon Showdown has a skill rating for each metagame they participate in. This rating--which is different from your ladder score--is calculated using an algorithm called Glicko and consists of an estimated skill value R and an uncertainty in that estimate RD. Based on these two values, we calculate the likelihood that a given player has a "true" skill value above a certain baseline (the conventional baseline was 1500, corresponding to the "average" player). For more about ratings, read here. For more about weightings, read here.

    4. How are tiers determined from usage?
      Tiers are based off a predictive algorithm designed to estimate how often a Pokemon will appear in the next month's usage statistics, based on the usage stats for the past three months (we update our standard tiers every three months). So we start by weighting the last three months' stats like this:
      Code:
      Three month usage= (20x last month + 3x month before that + 1x month before that)/24
      then the "OU" list for that metagame consists of all the Pokemon who appear on at least ~3.41% of teams, which is not as random a number as it might seem [citation needed]. Note that suspect tests are designed to move Pokemon into the Borderline ("BL") teams, which, like Ubers, are not based on usage statistics.

    5. Why does "Illuminate" sometimes show up in the abilities section of the moveset stats for Pokemon that can't have Illuminate as an ability?
      "Illuminate" is my placeholder for "no ability," or an ability that simply isn't recognized. This kind of situation happens when Showdown glitches out and (should be) exceedingly rare. Note that the nature equivalent is Hardy (though all five neutral natures are also aliased to Hardy) and the item equivalent is "nothing" (though that could also correspond to no item).

    6. What's the deal with the file names?
      You'll notice that for each tier and type of analysis, there are a bunch of of different files, most with names like uu-1695.0.txt. The first part of the filename is the tier, the second part is the weighting baseline (see 3). If there's no number following the tier name, then the baseline is 1500. Also note that a baseline of 0.0 means that the stats are basically unweighted.

    7. Can I perform my own analyses?
      Due to privacy concerns, I can't give you access to the raw logs, but if you have background with a programming language that can parse json, take a look in the "chaos" folder of each month's stats. Those files contain all the information used to generate the moveset statistics and include a lot more data than I could feasibly put into a file.
    More to come!
    Last edited: May 2, 2014
  3. Antar

    Antar That's Dr. Antar to you
    is a Battle Server Administratoris a Programmeris a Super Moderatoris a Community Contributor
    Official Data Miner

    Joined:
    Feb 17, 2010
    Messages:
    3,036
  4. Pyritie

    Pyritie

    Joined:
    Sep 20, 2010
    Messages:
    533
    What's the difference between files like "ou.txt" and "ou-0.0.txt"? I get that other numbers like "ou-1760.0.txt" is usage statistics high up the ladder, but the first two confuse me
  5. Antar

    Antar That's Dr. Antar to you
    is a Battle Server Administratoris a Programmeris a Super Moderatoris a Community Contributor
    Official Data Miner

    Joined:
    Feb 17, 2010
    Messages:
    3,036
    Pyritie

    Oh, yeah. Sorry. No suffix means the baseline is 1500.
    Pyritie likes this.
  6. SHUCKLE MAN

    SHUCKLE MAN

    Joined:
    Apr 26, 2006
    Messages:
    1,182
    Thanks for the stats as always!

    RU is set to start this month apparently. What stats will be used for making RU? There's UU stats and UU beta stats for this month, will just the UU stats be used, or both of them? Also, there isn't 3 months worth of 1760 stats, so will just this month's be used this time?
  7. Antar

    Antar That's Dr. Antar to you
    is a Battle Server Administratoris a Programmeris a Super Moderatoris a Community Contributor
    Official Data Miner

    Joined:
    Feb 17, 2010
    Messages:
    3,036
    SHUCKLE MAN, that's a topic for another thread, but expect an announcement in the next few hours.
    Seismitoad and SHUCKLE MAN like this.
  8. Salt

    Salt

    Joined:
    Nov 19, 2013
    Messages:
    320
    When the time comes for 3 month cumulative stats to be posted, not just for UU but for all tiers, is there going to be a place where we could find them?
  9. Antar

    Antar That's Dr. Antar to you
    is a Battle Server Administratoris a Programmeris a Super Moderatoris a Community Contributor
    Official Data Miner

    Joined:
    Feb 17, 2010
    Messages:
    3,036
    Yes, I'll post three-month usage stats and tier update announcements separate from this thread.
    Salt likes this.
  10. Flare Blitzle

    Flare Blitzle

    Joined:
    Nov 21, 2010
    Messages:
    237
    I don't know if this will be considered ok to ask here, but I'm not sure where else to ask: given that UU went official and out of beta, does that mean the tiers won't update for another three months now? I just noticed that a few things moved up and down (if using OU 1760 like last month). If this is the wrong place to ask this then feel free to delete this or whatever action is necessary.
  11. The Immortal

    The Immortal Administrator of Showdown!
    is a Battle Server Administratoris a Programmeris a Forum Moderatoris a Tiering Contributor
    Moderator

    Joined:
    Sep 27, 2010
    Messages:
    1,373
    Balanced Hackmons seems to be missing.
    asterat and Seismitoad like this.
  12. Antar

    Antar That's Dr. Antar to you
    is a Battle Server Administratoris a Programmeris a Super Moderatoris a Community Contributor
    Official Data Miner

    Joined:
    Feb 17, 2010
    Messages:
    3,036
    The Immortal, yeah, you know why BH is missing? Because you assholes allow shit like Mega Evolutions without Mega Stones and defaulting Darmanitan to Zen Mode. My scripts spit out so many errors I almost cried.

    Bottom line: no BH or Hackmons stats for... let's go with "a while."
  13. Arcticblast

    Arcticblast Winner of the Biggest Dork Competition 2014
    is a Forum Moderatoris a Community Contributoris a Tiering Contributoris a Battle Server Moderator Alumnusis a SPL Winner
    Moderator

    Joined:
    Nov 29, 2008
    Messages:
    5,253
    We can just say BH is so good not even a machine can analyze it!
    Arhops, Seismitoad, asterat and 6 others like this.
  14. HNA

    HNA

    Joined:
    Aug 28, 2010
    Messages:
    29
    I'm sorry if I didn't search hard enough but where can I see the mega usage stats ?
    And what the hell is seasonalfabulousfebruary ?
  15. Antar

    Antar That's Dr. Antar to you
    is a Battle Server Administratoris a Programmeris a Super Moderatoris a Community Contributor
    Official Data Miner

    Joined:
    Feb 17, 2010
    Messages:
    3,036
    Last edited: Apr 3, 2014
  16. nkhlshnd99

    nkhlshnd99

    Joined:
    Jan 6, 2014
    Messages:
    83
    When is ru becoming an official tier
  17. Magnemite

    Magnemite nvm i did it
    is a Contributor to Smogon
    Mentor

    Joined:
    Aug 20, 2013
    Messages:
    874
    I apologize if I'm just missing something, but will the changes since last month still be viewable? Doing them for every cutoff level for each metagame would be pretty unnecessary, but it would be nice to see them for the only stats used for tiering at the very least.
  18. shadowyoshi64

    shadowyoshi64

    Joined:
    Jun 7, 2012
    Messages:
    64
    I might be wrong, but looks like OU gained Hippowdon, Manectric, and Gardevoir, while losing Goodra, Volcarona, and Lucario. (based on 1760 stats)
    blaziken1337 likes this.
  19. Magnemite

    Magnemite nvm i did it
    is a Contributor to Smogon
    Mentor

    Joined:
    Aug 20, 2013
    Messages:
    874
    Since UU is now official, changes between OU and UU will now happen every three months. The changes you listed (along with Quagsire possibly moving to OU) will likely happen in June.
    Antar and Aragorn the King like this.
  20. Petrico94

    Petrico94

    Joined:
    Nov 13, 2013
    Messages:
    549
    Its stuff like this that makes BL so populated
    Aragorn the King likes this.
  21. GoldenSandslash15

    GoldenSandslash15

    Joined:
    May 13, 2013
    Messages:
    127
    This may be a stupid question, or it may even be in the wrong place (or both), but why do we do it this way? Why not have them update every month?
  22. Petrico94

    Petrico94

    Joined:
    Nov 13, 2013
    Messages:
    549
    Trends happen, sometimes someone gets popular for 40 days and goes back down. It would be dumb to move something manageable like Manectrike to OU when it was only very used for a while so theres a 3 month period before it's considered OverUsed.
    Antar likes this.
  23. Profesor Rod

    Profesor Rod

    Joined:
    Mar 15, 2005
    Messages:
    5
    I might had missed it before, but why Smogon Doubles doesn't have Leads stats?
  24. Antar

    Antar That's Dr. Antar to you
    is a Battle Server Administratoris a Programmeris a Super Moderatoris a Community Contributor
    Official Data Miner

    Joined:
    Feb 17, 2010
    Messages:
    3,036
    Because there are twice as many leads, and my stats currently can't handle that. It's also why there's no Checks & Counters data, because figuring out who killed what in a 2v2 situation is a hot mess.
  25. Profesor Rod

    Profesor Rod

    Joined:
    Mar 15, 2005
    Messages:
    5
    I understand the Checks & Counters situation. Not to jinx it and see them gone (because they're insightful), but then why VGC, XYDoubles and DoublesSuspect have them?
Thread Status:
Not open for further replies.

Users Viewing Thread (Users: 0, Guests: 0)