Technical Machine: A Pokemon bot

[david stone]

Technical Machine's home page, Mercurial repository, and old thread.

I am working on an AI to play Pokemon called Technical Machine. I am starting work on generation 5.

Here's a quick overview of where it is now:

It can load a Pokemon Online or Pokemon Lab team file and connect to a Pokemon Lab server. it currently cannot connect to Pokemon Online, but I do have plans to add that soon. It can battle on its own with most generation 4 teams.

The major set of moves it does not understand is moves like Sleep Talk (all moves that call other moves). The major set of items it doesn't understand is a foe's Life Orb. It can search up to three turns into the future (meaning it sees every possible outcome at most three turns ahead and acts based on that information).

Technical Machine only understands 1v1 battles.

Technical Machine cannot yet create its own teams; they must be supplied to it. After I add support for generation 5, I will work on having it make its own teams.

Technical Machine is written in C++. There are no plans to switch to a different language.

There is no pre-compiled Windows executable for you to run. If you want to run the program, you have to compile it yourself. I'll be putting up more detailed build instructions some time this month.

[deluge]

Cool, I remember you had posted about cost models for this sometime ago (as obi).
I'm going to check it out.

[MajorGambit]

I didn't quite get it? Could you provide us further clarification?

[verbatim]

Its an artificial A.I., sort of like a chess playing computer, except Obi (now David Stone) is treading relatively new grounds (i.e., he can't copy a code from something the smartest people in the world already put together, i.e., chess). To my knowlege he made a working one for gen four and is now trying to convert it over to gen five.

Obi, beyond connecting to Pokemon Online with the A.I. and uploading the data on all of the fifth gen pokemon, what else do you need to accomplish?

[Box]

You are a genius. What do you mean by it not understanding Life Orb?

[Togelover]

Originally Posted by Fat Box

You are a genius. What do you mean by it not understanding Life Orb?

My guess is that it doesn't understand that the opponent takes recoil from LO.

[Dr. Eyepatch]

^Likely, it does not factor any residual damage by an opponent's Life Orb when perceiving its own future options. That is what I assumed, anyway. (Looks like someone beat me to it.)

I remember seeing something about this several months ago and it interests me. I have been curious about one thing since then, which was whether or not Technical Machine will develop its own teams, but you seemed to have mostly answered it in the first post. Tell me, will it be gathering data from the opponents it faces or has already faced in order to create a team? If so, how will it go about this? Will it use what it found to be most effective from opposing teams, what could be used to gain the advantage over previously-seen pokemon, or some combination of the two?

In any case, no need for me to mention that this is an impressive feat, as many have probably already told you as much.

[Box]

As a rough estimation, when will you be done with Generation 5? Is this a program that anybody can use?

[Abacus]

"The second time, Hippowdon was attempting to use Slack Off, which seems to be an obvious bad move, but apparently not one so bad that TM sees why. Gyarados shows that it has a Life Orb, but unfortunately, I do not have support for Life Orb in my log analysis function, (a shortfall that will come back to bite TM later far worse than this). This means that even though Technical Machine's Hippowdon just took nearly 50% damage from Waterfall, meaning Slack Off only wins with Leftovers recovery and risks a KO from a CH, TM still thinks Waterfall will do about 38%, making Slack Off seem like a clear winner."

If this is still the case, then it appears that it doesn't take into account the damage increase of the foe's life orb.

Anyway, very exciting to see this project advance

[Togelover]

If items like Choice Band are factored in, I think it can see the damage boost, just not the recoil (again, just my assumption)

[david stone]

Originally Posted by Fat MajorGambit

I didn't quite get it? Could you provide us further clarification?

Technical Machine is a program / AI / bot that can play Pokemon. It connects to a server and without any assistance it determines which moves to use and then uses them.

Originally Posted by Fat verbatim

I(i.e., he can't copy a code from something the smartest people in the world already put together, i.e., chess)

Fortunately, some of the work already done is still applicable. My central algorithm is what is used in chess, but modified for games of chance. I've also gotten some ideas from backgammon AI (another game of chance). The biggest differences between this and any other good AI game is all the hidden pre-battle customization and simultaneous moves. In most games, players take turns making moves, but in Pokemon, players take moves at the same time. Combine that with hidden information about teams and you have a lot of unknowns that TM has to just kind of guess at.

Obi, beyond connecting to Pokemon Online with the A.I. and uploading the data on all of the fifth gen pokemon, what else do you need to accomplish?

I'm going to write a team creator, add in "yomi layers" (outlined at the bottom of the page at http://doublewise.net/pokemon/), and work on applying an evolutionary algorithm to Technical Machine's evaluation of the game. Right now it's a text file with a bunch of values where I just said "This looks like maybe the number is close to the real value" for things like Stealth Rock and status. I would have the program try out millions of combinations of data and use the one that seems to work best. It would battle against itself to weed out the bad ones, and take the values that seem to work and battle against humans to finish it up.

And obviously, I want to make it faster. The faster it runs, the stronger of a player it is. Right now it can search up to three turns ahead. My goal is five. Doesn't sound like a big difference, but consider this: looking one turn ahead is almost instant. Two turns ahead is barely noticeable. Three turns ahead is 10-30 seconds. Four turns ahead is probably about 10 minutes. Five turns would currently take more than a day at my estimation. I want to get 5 turns down to ~20 seconds per turn.

Originally Posted by Fat Box

You are a genius. What do you mean by it not understanding Life Orb?

In a battle, when it sees "Infernape lost 10% HP!" it doesn't understand that means it has Life Orb, so it will continue on with whatever its estimate of the item is. That estimate could be Leftovers, Choice Band, or even Life Orb (whatever is the most used item on that Pokemon). I have code built in that corrects HP if Technical Machine thinks it is different than the server does, but TM doesn't correct its estimate of the item, so it never thinks that they have a Life Orb boost. I don't have my reverse damage calculator built into TM yet, so it doesn't adjust items based on damage, either.

Originally Posted by Fat Dr. Eyepatch

I have been curious about one thing since then, which was whether or not Technical Machine will develop its own teams, but you seemed to have mostly answered it in the first post. Tell me, will it be gathering data from the opponents it faces or has already faced in order to create a team? If so, how will it go about this? Will it use what it found to be most effective from opposing teams, what could be used to gain the advantage over previously-seen pokemon, or some combination of the two?

My current plan is to take TM's prediction of the foe's team, and if the foe wins, save that team for later use. I'll also add in some ability to remove teams that do not perform well so TM isn't bogged down with bad teams that it happened to lose to. I'd also like to create a sort of "team optimizer" algorithm. It would run this on all of its teams and set EVs to optimal values, maybe switch some moves around, etc.

Originally Posted by Fat Box

As a rough estimation, when will you be done with Generation 5? Is this a program that anybody can use?

I have no idea when it will be done. Probably early next year, but maybe sooner or later.

Anybody can use if they can compile it. I've only compiled it under Linux, not Windows or Mac. I'll be posting detailed instructions on how to compile under Linux in a couple of weeks. I already have a pretty good set of instructions with the source code right now, though.

[Innocent Criminal]

Wow, I looked at the log and I must say I'm impressed. It might not be saying much, but this already seems much better than in-game AI (unless 5th gen had some drastic improvement, which I doubt). I'm currently working on a minimax AI for school, so I thought a bit of how one would make such a program... How exactly do you handle randomness ? Creating a new branch for every 10% burn chance would be way too expensive with negligible benefits, but evaluating both outcomes seems like an absolute necessity for moves like WoW or Focus Blast.

As for team prediction, I just made for Blue Kirby a slightly modified PO server that saves, for each battle, a complete log of the movesets used. Thanks to 5th Gen's team preview, such detailed data should help you make better guesses of what the opponent's sets are.

[420]

Why not just use a variation of the team predictor algorithm to have TM build its own team? For example, let it choose a top 15 pokemon at random and then use usage statistics to build the rest of the team for him, with maybe a few extra parameters to make sure it doesn't accidentally build a team with a glaring weakness to ice or something.

[Penance]

You sir have intrigued me. So will TM use only popular metagame teams or more a mix of gimmick teams and your everyday common weather team?

[yaya0]

Right now, what needs to be worked on is the prediction algorithm (and the Life Orb problem) more than anything else.

[STEELDRAGON]

oh wow, I knew I'd see that day when a Pokemon AI, is being developed.

I bet we are only a few years away, If not sooner, To having a AI, being capable of beating the best of the best easily. I mean In tournaments, I think there should be a "No AI rule", but that seems really difficult to implement.

[min min]

Hmm, PO has their very own Shanai, maintained by nixeagle. It's not that developed yet though, since it can only battle in a tailor-made metagame with everything deliberately kept simple.

[Vix]

This looks like a blast to eventually face on the smogon server.
As the above person has said PO does have shanai and it is fun to face despite its unfortunate limits.

Edit:ok I looked at the battle that you give a link to and all that I can say is WOW! O_o this thing looks like it would blow shanai to the moon and back! also luvdic'ed

[david stone]

Originally Posted by Fat Innocent Criminal

How exactly do you handle randomness ? Creating a new branch for every 10% burn chance would be way too expensive with negligible benefits, but evaluating both outcomes seems like an absolute necessity for moves like WoW or Focus Blast.

I actually do branch at every random event, and it is quite expensive, which is why I can only get to a depth of 3 currently. I was considering testing out a version where I "sample" the randomness by running the algorithm a bunch of times and just picking a random variable at each point and acting like that's what will happen. Right now that doesn't seem very useful, but maybe when my depth of search gets to 4-5 without it, that might provide enough of a speed up while still providing enough sampling opportunities to justify it.

What am I considering doing but haven't done is a way to cut off the random search when I know it doesn't matter. For instance, if I evaluate Fire Punch from Electivire against Celebi, then if I were to evaluate CH Earthquake before I evaluate non-CH Earthquake, I could notice that even with a CH on Earthquake, Fire Punch is better than Earthquake, then I don't even bother looking at how much worse a non-CH EQ is. Currently I only do cut-offs like this at the move selection level (if I evaluate Fire Punching their Abomasnow first and I get a score, then I evaluate switching Dragonite into their Abomasnow and I immediately spot that Blizzard would be bad for me, I don't have to keep evaluating all the other moves Abomasnow could possibly make because I assume the opponent uses the best move, so I can stop there).

Originally Posted by Fat 420

Why not just use a variation of the team predictor algorithm to have TM build its own team? For example, let it choose a top 15 pokemon at random and then use usage statistics to build the rest of the team for him, with maybe a few extra parameters to make sure it doesn't accidentally build a team with a glaring weakness to ice or something.

This is another idea I've thought of, and what will probably be version 1 of the "I did the entire battle myself" AI. Currently it needs a little bit of help with move set synergy, though.

Originally Posted by Fat dregstr

You sir have intrigued me. So will TM use only popular metagame teams or more a mix of gimmick teams and your everyday common weather team?

It can use really whatever teams it wants. The teams it currently uses have recently expanded. It has a variant of my stall team, a minor variant of my paralysis / Machampions team, a U-turn / Choice item team, and a Baton Pass team.

Using my ultimate vision of the team thief AI, it would use an improved version of any teams that can beat it (and probably delete teams that it uses that lose frequently). One idea would be for Technical Machine to re-evaluate teams every 100 battles (or something along those lines). It would delete every team from its memory that has lost more often than its overall score (or something like that). In other words, if it has an overall win rate of 70%, it would delete any teams it used that lost more than 30% of the games it was used in.

Obviously that's just a rough sketch (since it's just an idea I'm having right now), and it would need some tweaking like "don't delete teams that have been in one battle" or something, and don't delete so many teams that you have fewer than 10 teams, and various rules like that. Perhaps it would even be best to give teams a little wiggle room like "If you perform one standard deviation below the mean, you're cut", after determining the distribution of team wins and using an appropriate statistical measure of standard deviation.

As far as what it has now, the gen 4 teams I have it using are for the most part standard, and I'll probably stick with that for the early pre-built teams in Gen 5. However, I also want TM to have a diverse set of teams to pick from, so I'll probably craft teams representing the top 5 styles or so of play. As an example, I'll probably start out with something like a Rain team, Sun team, Sandstorm team, a stall team, a choice item team, a LO offense team, a BP team, etc. Now, a bunch of those teams could overlap (maybe a stall rain team or something), but I'd probably shoot for about 5 teams that show some variety in teams and cover most of OU.

Originally Posted by Fat yaya0

Right now, what needs to be worked on is the prediction algorithm (and the Life Orb problem) more than anything else.

Well, the Life Orb thing isn't such a big deal. It's probably a 10 minute fix. I'm just working on bigger picture stuff right now in hopes that I'll come up with a better solution by the time I get around to fixing it. As far as prediction goes, I do have a pseudo-prediction algorithm in mind (I outline it on my site at the bottom, under the Yomi Layers section: http://doublewise.net/pokemon/). However, I don't consider this to be as high priority as implementing Gen 5 and connecting to Pokemon Online, and even once I can do that, I'll probably fix up some other things before I try and add any "prediction" to it. It actually battles pretty well right now as long as it correctly understands the game. It plays more in a style of "I'm going to make it so no matter how well you can predict what I'll do, I'll try and get ahead anyway" instead of a high-risk, high-reward style of "I know what you're going to do and I have the best response".


What I'm currently working on is getting Technical Machine to connect to Pokemon Online. This would make it what I believe to be the first cross-server client. However, I'm having great difficulty deciphering the organization of Pokemon Online's source code, and I took a look at Shanai, but it's written in Lisp which I'm not finding very readable, either. However, I will persevere, and I'm hoping that I'll be connected within a week.

This has also given me an idea for writing a normal client that users would use to connect to Pokemon servers. However, the client would not be limited to any one type of server (any more than Firefox is limited to sites where the author uses Firefox, and Internet Explorer is limited to sites where the author uses IE), and it would possibly have some cool tools built in to it. However, that is much later in the future, after Technical Machine has all gens supported and can connect to all popular servers.

[david stone]

Technical Machine just made its first connection with Pokemon Online. I should hopefully have the rest of the protocol implemented tonight, or at the latest, tomorrow evening, after which TM should be able to have its first battle on PO.

[WinstonShnozwick]

Wow, this looks amazing. Great work so far. I can't wait to play with it.

Just to clarify; This is a battling AI for pokemon online correct? So I can have single player battles?

-Does it work offline?
-Are there random vs set options for the enemy teams? I.E. you can load a team for them or they have their own?
-What kind of strategies can it develop? Does the AI know how to sacrifice a nearly dead pokemon for revenge kills? Can it use a baton pass team? Can it play weather/trick room teams with emphasis on keeping the effect(weather, tr) active to use abusers?

[breh]

if you can finally get it to work on PO, it'll be a fantastic resource for new players!

[Unbreakable]

Originally Posted by Fat obi

Currently it needs a little bit of help with move set synergy, though.

Sounds like where I'm at. Also, +1 to Breludicolo's statement in that it could possibly teach new players how to play without having them throwing fits in the chat about losing streaks. Still, there will be no substitute for the tutor program.

[Box]

Originally Posted by Fat Unbreakable

Sounds like where I'm at. Also, +1 to Breludicolo's statement in that it could possibly teach new players how to play without having them throwing fits in the chat about losing streaks. Still, there will be no substitute for the tutor program.

Not really. A bot would have a vastly different playstyle from a human player which is somewhat static, and will eventually become predictable to an extent.

The bot would serve better as practice for new players rather than an actual learning program.

[crobats]

I agree it could help with some basics on the game but there still is no substitute for real humans. However this looks like a brilliant program and could be fun to play against or use.

Gratz on creating this though.