1. Welcome to Smogon! Check out the Smogon Starters Hangout for everything you need to know about starting out in the community. Don't forget to introduce yourself in the Introduction and Hangout Thread, too!
  2. Welcome to Smogon Forums! Please take a minute to read the rules.

Technical Machine: A Pokemon bot

Discussion in 'BW Competitive Discussion' started by david stone, Oct 2, 2011.

  1. PK Gaming

    PK Gaming Pursuing My True Self
    is a Site Staff Alumnusis a Forum Moderator Alumnusis a Community Contributor Alumnusis a Tiering Contributor Alumnusis a Contributor Alumnusis a Past SPL Winner

    Joined:
    Aug 18, 2009
    Messages:
    5,158
    Wow you've moved onto gen V already?

    You never cease to amaze me obi.
  2. david stone

    david stone Fast-moving, smart, sexy and alarming.
    is a Site Staff Alumnusis a Smogon IRC AOp Alumnusis a Programmer Alumnusis a Super Moderator Alumnusis a Researcher Alumnusis a Contributor Alumnusis a Battle Server Moderator Alumnus

    Joined:
    Aug 3, 2005
    Messages:
    5,150
    After some grueling programming, including trying to read source code that doesn't exist until the moment right before it's compiled, I finally have Technical Machine working with passwords and fully able to log in to the server and do everything except battling. Now that I have a strong foundation, I should be able to get battling working somewhat quickly.
  3. gookie

    gookie Administrator
    is a Forum Moderator Alumnusis a Contributor Alumnusis a Battle Server Moderator Alumnus

    Joined:
    Jul 29, 2010
    Messages:
    2,947
    Looking forward to this. A fantastic idea that could only be carried out by one user...
  4. Nonary Game

    Nonary Game

    Joined:
    Sep 11, 2007
    Messages:
    247
    Loving the concept, but I'm curious as to one thing: How predictable the bot will be. In an earlier post Obi you stated that it'd use minimax to determine the best moves to make, but would you perhaps allow some variance so it doesn't become too predictable (i.e. always switching to the absolute best counter, and on occasion making a different move that the AI would consider 'safe'.)

    Otherwise, upon looking through the source code, not bad at all. Nice work.
  5. david stone

    david stone Fast-moving, smart, sexy and alarming.
    is a Site Staff Alumnusis a Smogon IRC AOp Alumnusis a Programmer Alumnusis a Super Moderator Alumnusis a Researcher Alumnusis a Contributor Alumnusis a Battle Server Moderator Alumnus

    Joined:
    Aug 3, 2005
    Messages:
    5,150
    My original plan was to use the best move with some probability, the next best move with some probability, etc. However, what I'm thinking now is making use of yomi layers (http://doublewise.net/pokemon/ down at the bottom), and first estimating the probability my opponent is at a particular yomi layer, and then playing the move that beats that layer with that probability. However, this seems like a somewhat difficult prospect (especially the "what level are they thinking at" bit).
  6. david stone

    david stone Fast-moving, smart, sexy and alarming.
    is a Site Staff Alumnusis a Smogon IRC AOp Alumnusis a Programmer Alumnusis a Super Moderator Alumnusis a Researcher Alumnusis a Contributor Alumnusis a Battle Server Moderator Alumnus

    Joined:
    Aug 3, 2005
    Messages:
    5,150
    Update!

    It's been a while since I've posted an update here, but rest assured, I have been working hard. This is the state of Technical Machine:

    Technical Machine can log into Pokemon Online and Pokemon Lab.

    It has almost all standard features supported for non-battling activity in both. It can chat in the main chat (currently only if you tell it what to say) and watch the chat for certain key phrases and highlight them for me. It can read and respond to private messages (trusted users can send TM commands via PM). It can join channels. It automatically reconnects to the server if it gets disconnected.

    Technical Machine can also battle on both types of servers. It is not yet aware of timer options. It cannot play in Challenge Cup or any generation other than 4. However, it can be in any number of battles on either server simultaneously (as many as it can be in and still move fast enough not to lose by time, which is quite a large number if you set the depth of search to 1). It can either challenge a particular user or accept challenges (which can be filtered to only accept challenges from trusted users or from anyone). It cannot yet use the ladder. It has successfully used a U-turn, stall, and bulky paralysis team (performing best with the stall team, probably). It has a Baton Pass team that a bit buggy, but it's very close to using that properly, as well.

    One cool new feature of Technical Machine is its team selection process. You can give it either a team file for it to use, or a directory containing only team files and other directories. If you give it a particular file, it will always use that team, but if you give it a particular directory, it will use every team file in that directory with equal probability. It searches all subdirectories for team files as well.

    When it challenges a user, it randomly loads one of its allowed teams (so if you said a particular team, it just loads that one particular team, but if you said a directory, it loads a file from that directory) and uses that team to challenge. This way, when Technical Machine is the challenger, it cannot be counter-teamed, unless someone finds a way to counter-team all of its teams.

    On Pokemon Lab, when Technical Machine is challenged, it sees the user who is challenging it and loads a random team and then accepts their challenge (if it wants to accept).

    On Pokemon Online, due to a major flaw in their team system, you have to 'register' a team with the server. If you load a new team after being challenged, you automatically reject the other user's challenge with a "Technical Machine is busy" notice. I considered many ways to work around this problem, but ultimately I decided that the only reasonable solution is to reload a new team at the start of a battle. In other words, the process goes something like this:

    1. Fred challenges Technical Machine.
    2. Technical Machine looks at the battle settings and decides they are acceptable.
    3. Technical Machine accepts the challenge with whatever team it had loaded previously, since it cannot change teams.
    4. The battle begins, and right at the start of the battle, Technical Machine sends the team it would like to use in its next battle to the server. This way, if Technical Machine is challenged again, it will not necessarily be using the same team as the previous challenge.

    The one down side of this implementation is that it means that Technical Machine can only have one pending challenge at a time (because you cannot change teams while a challenge is pending). However, Technical Machine instantly makes decisions about whether to accept an incoming challenge, so the only time there is really a potential for lots of pending challenges is when Technical Machine sends out a challenge to someone else, because silly humans take time to decide whether to accept.

    The reason that all of this has taken me longer than expected is this: Rather than cobbling together something that mostly works for Pokemon Online that consists primarily of copy and pasting my old PL code and then modifying it, I carefully considered how best to support multiple servers simultaneously. The end result is code that is easy to maintain because I only have to update most things in one place, and both client implementations benefit, the code is potentially smaller and easier for the compiler to optimize because I'm saying exactly what I mean, less bug prone because I don't have implementations going out of sync, and easier to port to other servers. This means that when Pokemon Showdown comes out and I add support for that, it should be much easier and faster to get that up and going, because I already have a solid "generic client" framework laid down.

    However, that still doesn't explain all of the extra time it took me (my initial estimate was that I would be done with PO-specific stuff about 3 weeks ago). The remainder of the time came from me adding a few new features and securing my networking code. I cannot trust that the server I am connecting to is both bug-free and the exact code that is in the repository. There is nothing stopping someone from creating their own custom server and claiming it is a PO / PL server. This means that if there is a security flaw in my design, and I connect to such a server, they can send me whatever kind of data they want to exploit that flaw. Therefore, I spent extra time validating a lot of information the server sends me (primarily things like the server claiming that the message it's sending is, say, 5 bytes, but then actually sending me a subset of that message that is 9 bytes, which if not coded properly leads to a buffer overflow). I still have work to do here, but my code now has a little bit of error correction and a little bit more of error detection. If it detects an unrecoverable error in transmission, rather than trying to just keep going with data that is definitely wrong, Technical Machine just closes its connection to the server and reconnects.

    My plans for the immediate future are fairly simple. I'm going to finish up the last few bugs in my PO battle implementation first. Then I will add ladder support to both PO and PL. After that, I'll add something cool that I think should have been in Shoddy Battle, Pokemon Lab, and Pokemon Online for a while now. Based on both players ratings, Technical Machine will calculate the odds of each player winning. So if a player on PO has a rating of 1250 and they are fighting a player with a rating of 1370, then if the PO rating system accurately measures players ratings, it should be possible to calculate the expected outcome. Technical Machine could use this for its initial estimation of player skill if I decide to implement my idea for yomi layered battling.

    Finally, I'll probably work on a few bugs in my gen 4 implementation and add generation 5 support.
  7. kd24

    kd24 yo daddy musta been a drug dealer? why, cuz you're dope
    is a Pokemon Researcheris a Site Staff Alumnusis a Forum Moderator Alumnusis a Smogon IRC SOp Alumnusis a Tiering Contributor Alumnusis a Contributor Alumnusis a Smogon Media Contributor Alumnusis a Battle Server Moderator Alumnus

    Joined:
    Dec 21, 2007
    Messages:
    5,325
    dont these 2 statements conflict? the very first time its challenged, it should be able to randomly pick a new team for the next game, so in actuality, it should never be forced to use the same team?

    regardless, very impressive work
  8. david stone

    david stone Fast-moving, smart, sexy and alarming.
    is a Site Staff Alumnusis a Smogon IRC AOp Alumnusis a Programmer Alumnusis a Super Moderator Alumnusis a Researcher Alumnusis a Contributor Alumnusis a Battle Server Moderator Alumnus

    Joined:
    Aug 3, 2005
    Messages:
    5,150
    Let's see if I can explain it a little more clearly.

    You cannot change teams while you have a pending challenge. This means that whatever team I have loaded at the moment I am sent a challenge is the team that I must use for that challenge (unless I want to cancel their challenge, load a new team, and then send a new challenge). Therefore, if someone were to watch TM battle and then challenge it immediately after that, then with no 'extra' machinery, TM would be forced to use that same team if it accepts.

    To combat this, Technical Machine registers a new team with the server at the start of each battle (as soon as it receives the first battle message, it sends a new team to the server). This means that Technical Machine has a new team ready to go for the next battle at the start of its current battle. That way, if someone sends TM a challenge, it will be using a randomly selected team instead of the same team it is using in this battle.
  9. Unbreakable

    Unbreakable

    Joined:
    May 21, 2010
    Messages:
    348
    It's no surprise to me that TM does best with stall. After all, such teams are fairly straightforward to play: Set up hazards, keep them around as long as possible, phaze the shit out of the other team, wear them down. (Also, there was an article in The Smog a while back about how all the best ladder players usually used stall teams. I find it interesting how control-based strategies are so dominant at the highest levels of most games, though.)

    However, the part where you give it a team to play with is the most intriguing aspect by far. You can literally give it a team, have it run a few (dozen) battles, then read through the logs and see what is and isn't working on the team without ever having to jump into the action yourself, with all the butthurt that can entail. In this respect, it could very well be the ultimate team rater. ME GUSTA.
  10. david stone

    david stone Fast-moving, smart, sexy and alarming.
    is a Site Staff Alumnusis a Smogon IRC AOp Alumnusis a Programmer Alumnusis a Super Moderator Alumnusis a Researcher Alumnusis a Contributor Alumnusis a Battle Server Moderator Alumnus

    Joined:
    Aug 3, 2005
    Messages:
    5,150
    One other thing that I've mentioned before and will be adding soon (probably after or during Gen 5 support) is the ability to battle itself several hundred / thousand times and update its views of the importance of various things. If it has a wide variety of teams of its own, then even at low levels of play, it could do a self-play where one instance of TM always uses the team being tested, and the other instance goes through loading its teams randomly. Any major flaws / weaknesses could very well be discovered by this method. As a way to further test any major flaws that is mostly removed from any issues that TM itself has, I could run a "depth = 0" version of TM that just uses moves at random. This set-up could theoretically run about twice as fast as a real TM vs. TM battle, but is pretty much guaranteed to catch any truly glaring flaws after several battles.
  11. Vemane

    Vemane

    Joined:
    Nov 26, 2010
    Messages:
    398
    I would assume that this would have some utility for helping to team build, indirectly of course.

    Once the team builder is implemented, an algorithm could possibly be used to choose/create a team that nearly counters/would fare well against the opposing team.

    Then if a human player played against the AI using this algorithm, they could see what gives their team trouble, thus helping them to fix the holes.
  12. david stone

    david stone Fast-moving, smart, sexy and alarming.
    is a Site Staff Alumnusis a Smogon IRC AOp Alumnusis a Programmer Alumnusis a Super Moderator Alumnusis a Researcher Alumnusis a Contributor Alumnusis a Battle Server Moderator Alumnus

    Joined:
    Aug 3, 2005
    Messages:
    5,150
    Technical Machine is now able to read and write teams in PL or PO format. It could previously just read PL teams and mostly read PO teams, but I believe I have the bugs worked out to where it can read and write either format perfectly.

    The ability to write teams makes the "team thief" idea very easy to implement. If TM loses, have it save the foe's team to its own team directory. It is then able to select the team for future use. If I decide to add this, it would be just a couple of minutes to code. Much harder would be a 'team optimizer' function. Rather than just taking its estimation of the foe's team, it would try to improve the team. This would probably be scheduled to run on new teams as they are saved, in addition to being provided as a stand-alone script. The stand-alone version would allow anyone to get their teams optimized. The benefit to TM would be that I could set it to optimize all teams in a directory, so that as I improve the team optimizer, I can run it on TM's older teams so they can still get the optimizing benefit.

    Other formats I am considering supporting for this are various plain text formats (the result of Export to text from ShoddyBattle, Pokemon Lab, and Pokemon Online, as well as NetBattle) and ShoddyBattle1 teams, as well as NetBattle teams. I have many NB teams that I would like to able to easy give TM to use for when I get generation 3 support working.

    Obviously, it makes no sense for me to work on NB teams until after I have at least generation 3 support working in terms of progressing TM's abilities.

    Since the last update, I have also fixed many, many bugs. The team thing is the only major feature change. Now that I have better team file support written, I can write some automated tests that will catch several more bugs I'm sure, as well as prevent the addition of many new ones.

    I will simultaneously be improving my PO support. Once I have my tests written, I will move on to Generation 5 (much later than expected). I know exactly how I will write generation 5 support. However, it is important for me to put in some tests now for things that I will likely break before I go mucking about in the deepest parts of TM's code.
  13. BattleStar

    BattleStar

    Joined:
    Aug 29, 2010
    Messages:
    637
    This sounds awesome!
    This certainly would be a valuable tool to use in team building!
  14. WinstonShnozwick

    WinstonShnozwick

    Joined:
    Jul 13, 2011
    Messages:
    486
    With all of this being able to adapt to enemy teams and improve it's own teams, TM is starting to sound like "The very best that no one ever was"
  15. Treadshot A1

    Treadshot A1

    Joined:
    Jun 30, 2010
    Messages:
    1,253
    You really got something cool here, Obi. I'm wondering one thing, though. Since this will be implemented in the current simulator (or whichever is current at the time you finish), will it affect rankings? Or can i just practice against it a million times and not have to worry about dropping many points for losing?

    Also, the team optimising section. How exactly does the program define 'optimising'? I mean, if it's relatively simple, then it would become easy to predict, but if it were complex enough to analyse the meta as it went along, then...that's one hell of a program. I'm just curious as to how it'd work, is all. :)
  16. david stone

    david stone Fast-moving, smart, sexy and alarming.
    is a Site Staff Alumnusis a Smogon IRC AOp Alumnusis a Programmer Alumnusis a Super Moderator Alumnusis a Researcher Alumnusis a Contributor Alumnusis a Battle Server Moderator Alumnus

    Joined:
    Aug 3, 2005
    Messages:
    5,150
    I am actually keeping Technical Machine current with multiple simulators. It mostly works on PL and PO. When PS is publicly released, it will support that as well as the previous simulators. Some day, I'll check and see if there is still an active Shoddy Battle community and support that, too.

    Battles against Technical Machine will affect rankings in the same way as any other user. My bot will appear on the server as a normal user (unless the server owner makes some sort of special exception for TM). This means that rated battles against TM will count toward the ratings.

    I have mentioned in the past that I will have TM battle itself as part of its training program. For technical reasons, this will be taking place on a private server that I host, not a public server. There will therefore be no ratings change because of this part. The primary technical reason that TM will play on a private server is due to network latency. If Technical Machine can decide what move to use in .05 seconds, but it takes .5 seconds for the server to receive a message, then the game will take 10-20 times longer to complete (.5 seconds there and .5 seconds back), which means that TM will be able to play 1/10th - 1/20th as many training games.

    For the team optimizing:

    The first thing I would do is write an EV optimizing algorithm. You may have seen X-Act's app that does something similar, but this would be more like Sarenji's. X-Act's applet works the opposite of how I would write it. His has you determine what you want your bulk to be and then tells you where to invest your EVs. Mine would begin with the EVs you have already invested and rearrange them to give identical defensiveness (and offensive stats when I factor in natures), but have EVs left over. It would then apply a second pass to determine where to put those EVs. For instance, if someone gave Shuckle 0 HP / 4 Atk / 252 Def / 252 SpD, it would replace that with something along the lines of 92 HP / 4 Atk / 0 Def / 0 SpD. Shuckle would take most hits exactly the same, but now I have an extra 412 EVs.

    In other words, the optimizer would keep the EV structure as similar as possible to what it was before, but just make incremental improvements.

    More advanced versions of the optimizer might replace Flamethrower with Fire Blast, or vice versa, depending on which it determines to be more useful to the team. Even more advanced would decide that Manectric sucks and Raikou can do its job better (assuming Gen 3).

    As you can tell, the optimizer would not change the team structure at all, just make what it already has work better. The basis for this all would be saving teams that TM loses to. This would allow it to keep up with current trends (it instantly knows about a good team as soon as that good team can beat TM) while potentially improving on them.
  17. WinstonShnozwick

    WinstonShnozwick

    Joined:
    Jul 13, 2011
    Messages:
    486
    Not sure if I saw this, but will TM be able to interact offline for solo battling? If internet connection is sketchy or I want to be able to test out my team and refine it.
  18. AasTmO

    AasTmO

    Joined:
    Aug 18, 2010
    Messages:
    154
    Is TM in Open Beta yet? Because I'd very much like to help to give some human input to the program, but I don't see a possibility to do that yet.
  19. david stone

    david stone Fast-moving, smart, sexy and alarming.
    is a Site Staff Alumnusis a Smogon IRC AOp Alumnusis a Programmer Alumnusis a Super Moderator Alumnusis a Researcher Alumnusis a Contributor Alumnusis a Battle Server Moderator Alumnus

    Joined:
    Aug 3, 2005
    Messages:
    5,150
    The easiest way to interact with TM offline would probably be to install a server that is supported by TM, such as Pokemon Online. It has a straight-forward Windows installer, I believe.

    TM's source code is publicly available, so anyone who wants to test it is free to do so. However, they must build TM from source. This is very easy to do on a GNU / Linux system (you install some widely available things from your repository and then type scons), however, it is more difficult on non-Windows systems. The reason is that TM requires gcc version 4.6 or higher to compile (4.6 is the newest released version). This is the reason that other compilers do not work:

    Even Microsoft Visual C++ 11 Developer Preview will not compile Technical Machine. It lacks the following features from C++11: constexpr, initializer lists, and range-based for loops. I may eventually use delegating constructors, which is also not supported (but that does not appear in my code yet).

    Clang will not work because it lacks the following C++11 features: initializer lists. I may also add forward declared enums, which is also not supported. I'm having a hard time determining if it supports the integer types defined in cstdint, but I believe it does.

    Intel's C++ compiler will not work because it lacks the following features: constexpr, initializer lists, range-based for loops. It also does not support delegating constructors, which as I said I do not use but may some day add.

    g++ 4.6 and up support all of this. g++ 4.7 (still experimental) would be required to be released before I would add delegating constructors, which I am pretty excited to add. I wouldn't change the support status of any other listed compiler, because VC++ and Intel C++ lack support for constexpr, initializer lists, and range-based for loops, all of which are used extensively.

    Sources:
    Visual C++
    Clang
    Intel

    Unfortunately, the standard Cygwin and Mingwin solutions would not work because they only have very old versions of gcc available (and I don't intend to spend time compiling gcc from source, as this is not an easy process). Fortunately, I discovered mingw64, which supports even the experimental gcc 4.7. This means that I could fairly easily create a Windows executable that people could download and just run.

    However, Technical Machine is not at the point where this would be beneficial to development. I already know about the obvious bugs in Technical Machine, so I wouldn't get a benefit from people using it.

    Pretty soon I will be running TM on a public server, however, where people can "test" it by just playing against it.
  20. WinstonShnozwick

    WinstonShnozwick

    Joined:
    Jul 13, 2011
    Messages:
    486
    I use Pokemon Online as my simulator and I'm very pleased that TM can run offline. It would be invaluble in team building and testing for me, and even might replace online battling when I don't have internet connection. I hope it can be implemented as such in the future, if not even a plugin for PO. Sounds great, good work :D
  21. david stone

    david stone Fast-moving, smart, sexy and alarming.
    is a Site Staff Alumnusis a Smogon IRC AOp Alumnusis a Programmer Alumnusis a Super Moderator Alumnusis a Researcher Alumnusis a Contributor Alumnusis a Battle Server Moderator Alumnus

    Joined:
    Aug 3, 2005
    Messages:
    5,150
    Note that you would have to download the PO server, not just the client. The client is what you use to connect to the server. I'll post instructions on how to do this when I feel Technical Machine is at a state where that would be useful.
  22. Treadshot A1

    Treadshot A1

    Joined:
    Jun 30, 2010
    Messages:
    1,253
    That sounds great Obi. I've always wanted a way to competitively battle without having to fight someone else's pokemon. A computer is far less stressful. I cannot wait to find out how to make it work.

    Just one little question, though. I've studied a bit of basic coding, and read up on a bit of C++, but i couldn't write a line of code to save my life. I'm assuming that won't be necessary to set up TM (when it's ready)?
  23. Unbreakable

    Unbreakable

    Joined:
    May 21, 2010
    Messages:
    348
    Treadshot A1: Finally, someone that feels the way I do about fighting computers vs. fighting people.

    I would think that having a whole damn server necessary to play against TM would mean that the sort of person willing to run this is the sort that overclocks their CPUs to hell and back again and buys a shitload of memory expansion cards. In other words, not something your average basement-dwelling mama's boy can set up on a rickety 98 box.
  24. david stone

    david stone Fast-moving, smart, sexy and alarming.
    is a Site Staff Alumnusis a Smogon IRC AOp Alumnusis a Programmer Alumnusis a Super Moderator Alumnusis a Researcher Alumnusis a Contributor Alumnusis a Battle Server Moderator Alumnus

    Joined:
    Aug 3, 2005
    Messages:
    5,150
    Here's the complete process to run a PO server on Windows:

    Go here
    Unzip and install (for most users, I think the process is "double click this, then double click the result")
    Double click the icon that comes after that

    Congratulations, you now have your own server running! You may want to set it to "Private" if it's only for you to test on, so that it does not show up in the registry.

    You will always need a server to play Pokemon. In most single-player games, the server code and the client code are mixed together, but a "server" is still there. There is no way around needing "a whole damn server". If you are not playing online, then you need to have that server on your own computer.
  25. Max212

    Max212

    Joined:
    Jun 9, 2011
    Messages:
    245
    This looks very cool! I cant wait to see this project reach its full potential.

Users Viewing Thread (Users: 0, Guests: 0)