Availability of battle logs/replays

#1
I'm trying to get a data set for pokemon battles to apply some machine learning techniques to. I'm specifically looking for turn by turn player actions. The larger the dataset the better.

I've found the "Replays" section on pokemonshowdown.com, but the page appears non-functional, and the only data shown is from 2015.
I've found the statistics page, https://www.smogon.com/stats, which shows in depth team composition and usage.
From the source code, https://github.com/Zarel/Pokemon-Showdown/blob/master/logs/logging.md, there are server battle logs in modlog_battle.txt.
Privacy policy says "We reserve the right (but not the obligation) to keep text logs of all chatting in chat rooms, battles, and private messages, including secret rooms. We will keep private message logs as private as possible, and will only look at them if a participant gives us permission (such as through a "Report" button), or to help enforce laws."
I haven't found any other resources yet.

I have written a client for the showdown server for another project. I could set up a client bot to start joining and logging battles, but I don't know how appropriate that would be as far as site policy. I can use the parser and models I have to do the data processing I need given the raw messages for a battle room. I could make other formats work - like the replays. Any of this data publically available?
Thanks!
 
#2
I'm trying to get a data set for pokemon battles to apply some machine learning techniques to. I'm specifically looking for turn by turn player actions. The larger the dataset the better.

I've found the "Replays" section on pokemonshowdown.com, but the page appears non-functional, and the only data shown is from 2015.
I've found the statistics page, https://www.smogon.com/stats, which shows in depth team composition and usage.
From the source code, https://github.com/Zarel/Pokemon-Showdown/blob/master/logs/logging.md, there are server battle logs in modlog_battle.txt.
Privacy policy says "We reserve the right (but not the obligation) to keep text logs of all chatting in chat rooms, battles, and private messages, including secret rooms. We will keep private message logs as private as possible, and will only look at them if a participant gives us permission (such as through a "Report" button), or to help enforce laws."
I haven't found any other resources yet.

I have written a client for the showdown server for another project. I could set up a client bot to start joining and logging battles, but I don't know how appropriate that would be as far as site policy. I can use the parser and models I have to do the data processing I need given the raw messages for a battle room. I could make other formats work - like the replays. Any of this data publically available?
Thanks!
I don't exactly understand what you're trying to do, but for the first part, replays are just simply down atm
 

HoeenHero

The Misspelled Hero!
is a Battle Server Administratoris a Community Leaderis a Programmer
Community Leader
#3
there are server battle logs in modlog_battle.txt.
Actually its stored in the logs/ directory in various folders sorted by month, then format, then day. Each battle is a file containing all info about the match including player ratings, teams, what happened in the battle, who won, ect. This info is not accessible unless you have access to the server itself (so zarel for example).

Replay server is having some issues at the moment, standby. You can get some basic info from replays in JSON format by adding .json to the end of the replay link, this will include the format, players, battle id, and log. You may be able to parse and use the log for this.

I could set up a client bot to start joining and logging battles, but I don't know how appropriate that would be as far as site policy.
I think thats a bad idea, someone in the past was banned for using a setup like this to scout their opponents in tournaments while they practiced. But public replays should be fine.
 
#4
Actually its stored in the logs/ directory in various folders sorted by month, then format, then day.
Not the first slightly out of date documentation I've found :-D.

Replay server is having some issues at the moment, standby.
Is this just temporary downtime at the moment?

You can get some basic info from replays in JSON format by adding .json to the end of the replay link, this will include the format, players, battle id, and log. You may be able to parse and use the log for this.
This looks good, it's got the full logs, which should work well. Once the replay server is accessible I can see how large the available data set is.
 

HoeenHero

The Misspelled Hero!
is a Battle Server Administratoris a Community Leaderis a Programmer
Community Leader
#5
Is this just temporary downtime at the moment?
yes

Not the first slightly out of date documentation I've found :-D.
Actually, I think you just missunderstood what it said.

logging.md said:
Battle logs are placed under a subdirectory for each month (e.g. 2013-02).

Moderator actions are logged in the subdirectory modlog. Each chat room has a separate log file (e.g. modlog_lobby.txt). Battle rooms share a single log file, which is named modlog_battle.txt.
The first part explains where battle logs are stored, the second is refering to modloging for battles (and rooms). The modlog stores moderator actions (ex: If I lock someone in a battle its stored in modlog_battle.txt).

An example of a modlog entry is: [12:00am] (lobby) LOCK [guest1] alts: [guest2] [127.0.0.1] by hoeenhero (reason)
 
#6
Played around with the data from the replays this morning since the server is back up. It's got a good amount of data, but there are some limitations as a spectator, primarily limits of team info. For example, I can only infer movesets by what is used during the battle. Here's an extraction from one battle.
Code:
==> gen7ou-772180155.moves <==
p1a: Landorus|Earthquake
p1a: Landorus|Stealth Rock
p1a: Tangrowth|Hidden Power
p1a: Tapu Koko|Gigavolt Havoc
p1a: Tapu Koko|Hidden Power
p1a: Tapu Koko|Thunderbolt
p1a: Zeraora|Knock Off
p1a: Zeraora|Plasma Fists
p1a: Zygarde|Thousand Arrows
p2a: Blacephalon|Never-Ending Nightmare
p2a: Blacephalon|Shadow Ball
p2a: Blacephalon|Substitute
p2a: Greninja|Ice Beam
p2a: Greninja|Spikes
p2a: Gyarados|Dragon Dance
p2a: Magearna|Calm Mind
p2a: Magearna|Fleur Cannon
p2a: Magearna|Shift Gear
p2a: Zygarde|Coil
p2a: Zygarde|Extreme Speed
p2a: Zygarde|Substitute
p2a: Zygarde|Thousand Arrows
The |request| message is the most detailed, but won't show up for a spectator. I don't see any additional team information in the logs or in the html for the viewer.
I'm going to try spinning up a server and see if the logs have more information.
 
#7
Checked the format of the log files for a battle, this has everything that I need. There's team information, input info, and the log. I'll have to get in touch with a server operator and see if they are willing to share the logs.
 

Users Who Are Viewing This Thread (Users: 1, Guests: 0)

Top