• The moderator of this forum is jetou.
  • Welcome to Smogon! Take a moment to read the Introduction to Smogon for a run-down on everything Smogon, and make sure you take some time to read the global rules.

Programming Porydex - Showdown usage stats viewer, with trend charts

Drk Pwnr

Drk Pwnr




(Ignore the security warning; I don't have an SSL certificate yet because I don't have a domain name yet.)

For the last year and a half I've been working on a database/website that aims to show Pokémon Showdown usage stats data in a user friendly, easily navigable way that makes more in-depth analysis possible. I envisioned a site that combined the best parts of veekun - its comprehensive relational database of Pokédex data - with the best parts of the smogon.com/stats data archives - its shit tons of usage data. To that end, I've created Porydex (name probably not final) - the (probably not yet, but hopefully soon) best way to search through and analyze Showdown usage stats.

I started this project long before azelf.info or pikalytics.com were public, but I think my goals are different enough that I can still carve a niche for myself. For one thing, my main motivation was to make it easier to compare data. This month vs last month, this rating level vs that rating level, and so on. I wanted to be able to do things like see a list of the most common users of Trick Room or Defog or Choice Scarf. Also, charts. Lots and lots of charts.

Features

Planned Features
  • Trend charts with multiple data sets! Look at the usage history of Landorus-Therian vs Tapu Koko! Or Landorus-Therian with Choice Scarf vs Tapu Koko with Hidden Power Ice! Or Magearna's usage of Assault Vest vs Trick Room vs Shift Gear! Technically all these are already possible, but not accessible from the UI. I don't know how to handle the UI for that yet, and I'm open to suggestions. (I also need to finish the code that dynamically generates chart titles and line labels based on which trends are being requested.)
  • A Pokédex component to the website, with all the things people expect from Pokédex websites like serebii or veekun. Technically the website's database already has most of that data already, but a UI for any of it has been very low on my priorities list compared to the stats half of the site.
  • Whatever enough other people want - I'm taking requests.

This database has a LOT of potential, and I don't want to keep it all to myself. Even beyond simple usage lists, some real hardcore analysis is possible with the right queries. For example, did you know that the current most common Eeveelution trio in OU teams is Jolteon/Umbreon/Espeon? Or that the most used nature by Cresselia for high rating players across all formats in the history of gen 6 and 7 is Bold? That's just a taste of what this data makes possible. So if any stats nerds like myself have any esoteric usage stats related requests, ask and I will try to find you an answer.

Also, it's all open source on GitHub. The site was built with a PHP/MySQL back-end. If anyone wants to contribute, go right ahead! Especially anyone who's better than me at UI/UX (so, anyone at all).

Enjoy!
 

Attachments

  • porydex.png
    porydex.png
    77.8 KB · Views: 5,056
Last edited:
I like the site! This information is largely already available, but it saves a lot of time in tracking usage over an extended period. I typically have multiple tabs open when trying to accomplish the same. To that end...

On the Pokemon's individual usage pages (example), would it be possible to include an option to combine the usage data of multiple months at once? Having more stats to draw upon would make the figures shown less influenced by the uptake of "fads", which can skew impressions of the most common sets.
 
Good idea! I can do that for abilities, items, and moves. For spreads, teammates, and counters, there's too much variability from one month to the next for such averages to be accurate (which is why I haven't implemented graphing over time for those sections).

There are some other pitfalls though, specifically with gaps in the data. I get these stats from the human-readable stats files, not the "chaos" stats files (because they are too big to be parsed by my tools), so low-usage items/moves will often fall into the "Other: X%" category and not be imported to my database. In all the charts, I assume gaps in the data are 0% because it's harmless enough there. In taking averages over time, those 0%'s will lower all averages a little bit from what they really should be... Which might not be a problem, if you're specifically looking to avoid usage fads.

How flexible do you want this feature to be? "Show me the combined usage data from <arbitrary month 1> to <arbitrary month 2>"? Or just "Show me the combined usage data from all time"?
 
I see what you're saying with respect to the flaws in the truncated stat files, although I think the feature could still be useful in getting a general overview of the most common moves. In my head I originally imagined a "from X to Y" toggle, but now that I think about it a toggle for "all time" would cover the majority of usage scenarios, and would therefore likely be sufficient.
 
Drk Pwnr said:
There are some other pitfalls though, specifically with gaps in the data. I get these stats from the human-readable stats files, not the "chaos" stats files (because they are too big to be parsed by my tools), so low-usage items/moves will often fall into the "Other: X%" category and not be imported to my database.
Click to expand...
I'll make no promises about it but since I've done this exact work (parse chaos to something usable) if you can explain the troubles you have I may be able to help you solve them. Send me a PM or something if you're interested and we can continue this without filling the thread with it.
 
Thanks! Having those sprites and redirect links really makes the whole "journey" more relaxing. Incidentally, is there a possibility that you'll add in the future Older Gens? I'm aware that some tiers are essentially dead, but tiers like ADV OU, DPP OU, BW OU and ORAS OU are still played by a good chunk of players on the ladder.
Also, how do I access the chart/trends page in order to check other things? http://159.89.92.65/stats/trends/chart doesn't seem to work nor http://159.89.92.65/stats/trends/
 
Last edited:
Carbonific: I'm working on this feature now. It might take a week or two, because technical reasons. (In my database I stored all usage data with separate year/month numeric columns, rather than a single date column that would have made this feature trivial to implement. I'm changing that first.) I think I'll do it as a "from X to Y" anyway, for maximum versatility.

Quite Quiet: The trouble I'm having is that I built this in PHP, and PHP is woefully under-equipped to parse big files. Not much I can do about that, nor much I especially want to do about it right now. The current system works well enough for me (for now?)

100 Hoes Kaido:
1) Yes! My plan was always to add more formats on request, because I had no idea what non-current-gen formats people would be interested in.
2) The chart/trends pages can be accessed through the usage change "+X%" or "-X%" table cells. Those table cells are actually links. (I'm sorry I'm so bad at UI)

Other updates:
- For the last month I've been adding move learnset data from the games to the database. Eventually that data will be web-accessible.
- April usage data has been up for a week or so.
 
Last edited:
Ooo I like what I see so far! Took a peek at the repo, do you mind including a quick read me/getting started? I'm a UI/UX designer with some front end exp and I'd enjoy playing with this in my free time. Reminds me of champion.gg (for league stats).

PM me if you want to chat more technically w/o spamming here!
 
very minor thing but in case it's of help, zygarde is listed as Aura Break in the Anything Goes ranks when it should be Power Construct.
 
New stuff:
- May usage data has been added.
- I wrote a brief readme for anyone looking at the project code.
- I'm in the process of adding a handful of missing Pokémon gifs.
- I created the averaged stats over multiple months pages (direct link). I haven't yet added any kind of UI navigation to those pages, though. Until I do, you can manually change the month range in the url.

Other comments:
MacChaeger: What ^he said. But specifically, it's because the site has a form (the change language button at the bottom of every page) plus no SSL certificate. Browsers these days are very security conscious, so that combination gets flagged as dangerous.

Tiksi: Yeah, I'm aware of that bug. The source usage files don't separate the Zygarde formes into separate Pokémon the way the games do, so I import all instances of 50% Zygarde as the form with Aura Break. The best fix I could do for this is to always import Zygarde in Ubers/AG as the Power Construct forme. It wouldn't be a perfect fix though, because some small percent of Zygardes in those formats use Aura Break.
 
Last edited:
New stuff:
- I have a domain name: https://www.porydex.com Goodbye, security warnings
- Usage data for the last several months, obviously. I'm waiting on importing October data because it went online suspiciously early in the month and it hasn't been officially posted on Smogon yet, so I want to wait for confirmation that the data is accurate.

Future stuff:
- I'm currently working on in depth dex pages for Pokémon/abilities/moves/types/etc. None of it is online yet, but it'll be coming soon-ish.
- Just for fun, after I realized this database had enough data to do it, I wrote an egg move breeding chain calculator. That's going to be integrated with the Pokémon move learnset dex page as soon as one exists. I'm quite proud of this breeding chain calculator. It covers everything from XD-exclusive moves to Smeargle. It should be a useful resource for anyone playing on cartridge.
- Since a lot of this dex stuff doesn't matter outside cartridge play, I'm wondering if I should make VGC the 'default' format, as far as the UI goes. A quick link on the homepage, "View Current Stats," that takes you to the latest VGC data? It'll either be VGC or OU, for sure.
 
Last edited:
Oh man, this is super helpful. It's a step up from a lot of forms of usage stats and all that, that's for sure.

I'm pretty sure you could implement links to SmogonDex Analysis's with their respective formats, do you think it's a good idea?

On the note of the RoA stuff, think that adding in the RoA Metagames would be helpful for legacy purposes. Competitive History is interesting, y'know? I personally play a lot of Gens 1+2 and there isn't really a lot of resources helping them from Smogon itself. You tend to have to look through buried (and possibly outdated) threads or something. Just general data in a more readable format would be really helpful. If you do go implementing legacy formats, I think arranging them by generation would be the best way to go about it without cluttering it all.

As a VGC Player, I think this is phenomenal, as there isn't really many accessible resources around either. The graphs are perfect. I noticed you didn't have the Checks/Counters stuff down though, is this something to come or is it bugged? It's not anything bad since I know another website that gives it, but is there a fix on the horizon?

Are there plans for an installer or something for the downloadable version? I'm not really...great with Github. I kind of suck at it, actually. Outside of installing Pokered or something I'm kind of like "How do I computer?". Sorry if I sound demanding or anything ^^;
 
I absolutely love this tool and have started using it a lot in my team building and meta trend research. Its one of my favorite bookmarks now. Keep doing what you're doing!!
 
New stuff:
- Per Chippy2001's suggestion, I added Smogon Dex Analysis links to the Pokémon stat pages. This was a good idea I doubt I ever would have thought of on my own, thanks!
- November and December stats are up. Sorry for the longer-than-usual wait time for December; I was on vacation when the stats were released.
- Dex pages for abilities, natures, and types are up. Dex pages for moves and Pokémon are pending, so links on Pokémon/move names from those pages are all placeholders and broken right now.
- The dex page for egg move breeding chains is up, but inaccessible through the UI until I make the Pokémon dex page. This was a fun side project, though.

Other comments:
- RE including stats from older gens: The main reasons I haven't been doing that to begin with are 1) older gens don't have as many games played as current formats, so their usage stats from month to month could be a lot less trustworthy; and 2) i ain't made of database server money, yo. That said... I'd still love to someday include all the 'canonical' formats (Uber through PU, for gens 1 through 7+).
- RE checks and counters not showing up: I include checks and counters for all the formats that have them, aka all the singles formats. Checks and counters aren't calculated for the VGC/doubles formats, I guess because there's no way to calculate them.
- RE installer or something for the downloadable version: Sorry, the only 'downloadable' version for this is if you download the website code and manually run it in your own web server.
- RE when will [month] stats be up: usually, within a few days of when the stats are released here, which is usually a week or two into the month.

Future stuff:
- The next thing on my todo list is to get evolution data into the database, but that data won't be useful/visible to anyone for a while.
- The next useful thing on my todo list is to do the dex pages for moves and Pokémon. Then I'll at least have all the basic dex data publicly accessible, and I can declare my childhood dream of creating a Pokédex website complete.
- Also, I need to add all these new dex pages to the menu. As in, I need to make a menu lol

EDIT 2019-02-28 because I don't feel like double posting this time:
The site is going to be 'down' for a couple days, because I've decided to import the data for all the old generations' Ubers/OU/UU/etc formats. So, those of you who wanted old gen data, you won't have to wait much longer! In the meantime, azelf.info and pikalytics are still online, so y'all can use those for now.

EDIT 2019-03-01 yes i'm aware that gen 1 moveset information is currently broken. it's because i didn't program the gen 1 stat calculation into the page yet. that darn Special stat always messing things up

EDIT 2019-03-05 okay all the new data is in now. I still have to fix the gen 1/2 stat calculation stuff, but other than that there is still more data now than there was before
 
Last edited:
New stuff:
  • Stats through March are up.
  • See that new "Save as my default format" button on the usage page? Click it. Then add https://www.porydex.com/stats/current to your bookmarks, and never have to worry about updating your bookmark again :) (And then go save your actual preferred format as your default, haha)
  • If you missed my many edits to my last post, Showdown stats are now being imported for old generations. Also, LC and Monotype, so now Porydex includes all Core and Official Smogon metagames.
  • Created the dex pages for specific Pokémon and specific moves. It's just move learnset data for now (all the moves that each Pokémon can learn, all the Pokémon that can learn each move, respectively) but it's better than nothing.
  • See those egg icons in the learnset tables? The ones that show which egg moves a Pokémon can learn? Click one. It'll give you the breeding chains to get that egg move.
Future stuff:
  • Next I'm going to make a dex page for all Pokémon (as I have with abilities), and a dex page for all moves. Then I'll finally have all the important top-level dex pages, so I'll make an actual nav menu.
  • I said in my last update that I would be working on evolution data next. I decided to put that off until gen 8, because I want to see how Rockruff/Cosmoem's version-specific evolution triggers are handled.
As always, if anyone has any feature requests, please share them! Otherwise the next several months are probably going to be behind-the-scenes code rewrites and adding more dex data to the UI.
 
Last edited:
This is good stuff! Have you thought about putting OM and unofficial metagame data on porydex too? This has been really helpful for teambuilding, so thank you for working so hard on all of this!
 
Porydex is *not* discontinued. I'm currently working on importing dex data for Sword/Shield. (I put off playing them for a couple months because of Dexit bitterness. That's the main reason I didn't import the data months ago.) It's a slow process because I'm taking data dumps from the game (via pkNX, et al) and converting the relevant data for use in a relational database, one data object type at a time (moves, then Pokémon, then Pokémon forms, then learnsets, etc).

I'm going to import the October 2019+ Showdown usage data as soon as I have the bare minimum for the site to be able to handle gen 8 usage stats.
 
Porydex has been updated with gen 8 usage stats and gen 8 game data - minus learnsets, which I decided I may as well wait for the DLC before I add those at this point.

A lot of stuff is broken (especially the National Dex formats, because of how everything is implemented). I'm going to fix those.

Give me your bug reports, your other requests, etc.

EDIT 6/23: While waiting for various third parties to dump data/compile assets for the DLC, I'm trying to enhance the functionality of the dex pages on the site. Updates so far:
- On the types page, you can now toggle the visibility of rows/columns of the type chart. Because when I was trying to come up with an in-game moveset for Kubfu with good coverage, I realized this feature would help.
- On the abilities page, I now list all Pokémon that each ability has. And also I added a search bar I guess.
 
Last edited:
Lots of updates over the last few months. Here's a brief list:

All of the usage stats pages are now powered by an internal json API, so if for some reason you want to consume Porydex's data without having to scrape the web pages themselves, you can. The following API routes are currently exposed:

Also, a lot of UI/UX enhancements on all of the above pages. Table sorting and filtering, and hopefully the chart links are more apparent. On that note, the charts are now fully integrated with the usage stats pages. The chart links open up the chart in a sidebar on the same page. Which means, we're only one small step away from charting multiple things at the same time (aka, Clefable vs Dragapult usage over time).

For a teaser of that functionality, I've added a "Chart this in all Rating levels" button to the chart sidebar, which will create lovely charts like this that let you compare pro usage numbers against non-pro usage numbers. Because it's fun to see how the trends differ between casual and hardcore play.

Also, I know I'm behind again on recent usage data. I keep putting off adding the DLC to my database in favor of doing fun things like everything else I listed in this post. Oops.
 
Last edited:
You must log in or register to reply here.
Back
Top