• The moderator of this forum is jetou.
  • Welcome to Smogon! Take a moment to read the Introduction to Smogon for a run-down on everything Smogon, and make sure you take some time to read the global rules.

Programming Smogon / RMT Team Dump

FullLifeGames

FullLifeGames

Bene
is a Programmer
Hey guys,

I admit it, I'm one of the guys who always went to the forums for teams. The forums provide a great resource for someone like me, if you just want to hop on Showdown and play some fast games!

Lately however, I was getting kind of bored, trying to search for teams, as it seemed to be getting more difficult. Because of this problem, I thought about making my life a little bit easier and try to think of an automatic solution.

This is what I'm presenting you today!


So what am I talking about?

I created a program which goes through the "Smogon Metagames" and "Rate My Team" forums and all their subforums. In each thread of these subforums the program analyzes each post and if it finds a team hidden in a spoiler or in a pokepast.es link, it saves the team. For each team some metadata like likes and time of posts will be saved so each team can be ranked by the following formula:
this.Likes *
(1.0 /
(
(DateTime.Now.Subtract(this.PostDate).TotalMinutes / 60.0 / 24.0 / 365.0) * koeffScale + 1
)
);
Where koeffScale = 5, so that newer teams get way more attention
With this, I got over 12000 teams from Smogon and exported them into an importable Showdown format.


Show it! (Link: https://fulllifegames.com/Tools/ReplayScouter/#/smogonDump)

1688844447524.png


arAfLzE.jpg


tr384Fe.jpg


VAdN59r.jpg


Is it perfect?
No, of course not! There are many mistakes and problems, but most of the teams will work. If you encounter a problem, let me know and I will try and have a look at it!


Advice

Since there are so many teams, the filesize of the dump is 12 MB. Because of that, I advice you to download the dump and then import it into an incognito window of your browser, if you really want to have a look at it, since it otherwise will probably just crash your browser.


Enough of it, where do I get it?

Url for dumps: https://fulllifegames.com/Tools/ReplayScouter/#/smogonDump

The source: https://github.com/FullLifeGames/Smogon-Team-Crawler

Download the paste: https://drive.google.com/uc?authuser=0&id=1rjUVdzZWw8dqBi3Yt7CLLfyVe1tSE7Sc&export=download
The paste: https://drive.google.com/open?id=1rjUVdzZWw8dqBi3Yt7CLLfyVe1tSE7Sc


What if I just want a dump for a single tier?

Don't worry, I created a paste for each tier and you can access it from here:
https://drive.google.com/open?id=18gIrfuAXcxrEVm3FKj72r4BKG8l5DcRc
Or look here at the visualization: https://fulllifegames.com/Tools/ReplayScouter/#/smogonDump

8E5L0uT.png
 

Attachments

  • gen8outeams.PNG
    gen8outeams.PNG
    114.4 KB · Views: 4,533
Last edited:
Yeah, so I enhanced some things and hit the limit:

OHoP8k2.png


https://github.com/FullLifeGames/Smogon-Team-Crawler/commit/56fd79270bae4b47fc14294f83af418e741c7dc6

What I did:

- enhanced team detection
- basically add gen 1 and gen 2 support
- fixed some bugs with the new design
- added pastebin / hastebin support
- extend on multiple team pastes

This brought us from ~7000 teams to 12055 teams, which is a big improvement.
I'm glad it now works again and I'm amazed by the better results to be honest.
(Downloadlinks in the main post are updated)
 
Last edited:
What could be great is a way to search for teams given 6 pokemons and a tier, to find the opponent's team directly from preview if it was ever posted somewhere.

Edit: Filtering by date would also be useful to find teams built with the latest bans of the tiers.

This is great !
 
Last edited:
Very cool stuff, I'm a newb on this but would it be possible in your Smogondump's tools the ability to ctrl+f some 'mons redirecting at their sprites? For Example I typed Hippowdon, and while there are some teams with it on the Gen5ou section it doesn't give me a correlation, since there is only the sprite, I believe it could be very useful if you could type more than one 'mon in order (ex: Tyranitar Garchomp Skarmory Jellicent etc) on your search bar and find the team, kinda like you can do it on Showdown, especially on something like Usum who has too many teams.
Still, thank you for your hard work, I can't imagine how tiring this has to be, kudos to you, really.
 
100 Hoes Kaido said:
Very cool stuff, I'm a newb on this but would it be possible in your Smogondump's tools the ability to ctrl+f some 'mons redirecting at their sprites? For Example I typed Hippowdon, and while there are some teams with it on the Gen5ou section it doesn't give me a correlation, since there is only the sprite, I believe it could be very useful if you could type more than one 'mon in order (ex: Tyranitar Garchomp Skarmory Jellicent etc) on your search bar and find the team, kinda like you can do it on Showdown, especially on something like Usum who has too many teams.
Still, thank you for your hard work, I can't imagine how tiring this has to be, kudos to you, really.
Click to expand...
Try the search again => I fixed the search bar I think.
Type in "Tyranitar, Garchomp, Skarmory, Jellicent" and press Enter.
 
I wanted to talk about the "Statistics" tab which I added to the website!
Basically it shows how many times a Pokémon has been used and how many times a move has been used (and their relative percentage).
Here a little example how it looks:

MVgWE3T.png


(Also I worked a little bit on the sprites, so better representation of the Pokémon here)
 
Hey, thank you for your great work on this resource. I share it pretty frequently in the OU room and its a great solution for finding new teams.

Could you update the google docs to include gen8 txts? The visualizer still seems to work and its in the json too but I find that the fully importable txt format is by far the best.

Thanks.
 
UBERPYRO said:
Hey, thank you for your great work on this resource. I share it pretty frequently in the OU room and its a great solution for finding new teams.

Could you update the google docs to include gen8 txts? The visualizer still seems to work and its in the json too but I find that the fully importable txt format is by far the best.

Thanks.
Click to expand...
Glad to hear that people are using it!

So apparently due to a previous error I was only automatically updating previous pastes in the Google Drive.
Now I'm also creating new ones if new .txt files were to show up.

This means that Gen8 formats and some nieche formats are also now integrated in the Google Drive folder!
 
Hey !
Wow congratulations for the awesome work !
I'm not into programming so I don't know if it's doable, but would it be possible to be able to sort the teams from the older to the most recent ? That would allow us to see the evolution of the metagame, and find recent teams (so more adapted to the meta) more easily. Would make it even more magical.
 
Hey there, big fan of this resource. One thing I've started noticing is that a number of Pre-Home Gen 8 OU teams are bleeding into the Gen 7 OU text dumps. And this problem is as recent as the last update as far as I can tell. It's super useful, but I just want to make sure I'm not trying to use a Corviknight in SM OU!
 
Hi! I'm having trouble dumping the teams. They either all gather in one team slot and become a huuuuuge long line of teams or when I put them on the backup all teams space they create a new folder for each team, which is still annoying to mess with. Can anybody help me? Figured i'd post here instead of dm'ing FullLifeGames in case someone else later has the same question. or maybe im just dumb lol

Big thanks in advance
 
Sorry for the late answers!

Atha said:
Hey !
Wow congratulations for the awesome work !
I'm not into programming so I don't know if it's doable, but would it be possible to be able to sort the teams from the older to the most recent ? That would allow us to see the evolution of the metagame, and find recent teams (so more adapted to the meta) more easily. Would make it even more magical.
Click to expand...
So basically https://drive.google.com/file/d/1wA6q9a7uOqDBcMeRfMx67Bi-Lrl-0jey/view?usp=sharing and https://drive.google.com/file/d/1QrXa8-1oRdQNrN8HqzCLZ-bGbtwziinM/view?usp=sharing include the post timestamp, so from that you could make such a representation. No plans are currently however made for this.

Voltage said:
Hey there, big fan of this resource. One thing I've started noticing is that a number of Pre-Home Gen 8 OU teams are bleeding into the Gen 7 OU text dumps. And this problem is as recent as the last update as far as I can tell. It's super useful, but I just want to make sure I'm not trying to use a Corviknight in SM OU!
Click to expand...
I think I either resolved this, or this resolved itself, but sorry for the late reply.

Bandkrook said:
Hi! I'm having trouble dumping the teams. They either all gather in one team slot and become a huuuuuge long line of teams or when I put them on the backup all teams space they create a new folder for each team, which is still annoying to mess with. Can anybody help me? Figured i'd post here instead of dm'ing FullLifeGames in case someone else later has the same question. or maybe im just dumb lol

Big thanks in advance
Click to expand...
It has been a long time, when I used the dumped .txt files. I would recommend using the website and including the teams you like from there!
 
hey just wanted to know if its not an error if every tier doesnt have his own .txt in the google drive directory because tier like gen8uu, gen8ru ect.. are in the FinalJson.txt file but it tooks ages for my computer to fetch this :( so is it possible to make more txt files for gen8 tiers please :heart::) ?
 
Last edited:
Maxouille said:
hey just wanted to know if its not an error if every tier doesnt have his own .txt is the google drive directory because tier like gen8uu, gen8ru ect.. are in the FinalJson.txt file but it tooks ages for my computer to fetch this :( so is it possible to make more txt files for gen8 tiers please :heart::) ?
Click to expand...
So apparently my upload to Google Drive script was broken at some point, when I tried to remove the upload for empty files.

Fixed it and now included the missing files:

"gen8nationaldexmonotype.txt
gen8battlespotsingles.txt
gen1nfe.txt
gen8nationaldexag.txt
gen52v2doubles.txt
gen2pu.txt
gen8almostanyability.txt
gen8nationaldexbh.txt
gen3zu.txt
ou.txt
gen8nationaldex.txt
gen2zu.txt
gen1stadium.txt
gen8nationaldexuu.txt
gen8pu.txt
gen1rby898.txt
gen8mixandmega.txt
gen8camomons.txt
gen5 pu.txt
gen8nfe.txt
uu.txt
gen5pu.txt
gen82v2doubles.txt
gen8battlestadiumsingles.txt
gen4zu.txt
gen1lc.txt
gen3nu.txt
gen7anythinggoesmixandmega.txt
gen6zu.txt
gen8ru.txt
gen8.txt
gen8doublesuu.txt
gen6monotype.txt
ORAS OU.txt
gen1nu.txt
gen8stabmons.txt
gen8balancedhackmons.txt
gen4lc.txt
gen7metronomebattle.txt
gen8nu.txt
gen8uu.txt
gen7trademarked.txt
gen8zu.txt"

(Remember me / or someone to rework this mess of a codebase at some point)
 
FullLifeGames said:
(Remember me / or someone to rework this mess of a codebase at some point)
Click to expand...

Yeah, so I got quite annoyed that the dump apparently did not get updated, so I straight up did a full refactor of the code base! Commit

There are still some very rough edges, but I think the code base is now much more maintainable.

As a bonus: It is now testable and comes with a library, so you can use it in your own project!

Enjoy!
 
You must log in or register to reply here.
Back
Top