1. New to the forums? Check out our Mentorship Program!
    Our mentors will answer your questions and help you become a part of the community!
  2. Welcome to Smogon Forums! Please take a minute to read the rules.

Programming [WIP] Setting up the Smogon development environment

Discussion in 'Technical Projects' started by ryubahamut, Jan 30, 2009.

  1. ryubahamut

    ryubahamut
    is a Site Staff Alumnusis a Programmer Alumnusis a Forum Moderator Alumnusis a Contributor Alumnus

    Joined:
    Jan 18, 2007
    Messages:
    999
    I figured I'd post it here before posting it in the Programming the Site forum.

    Setting up the Smogon development environment

    This tutorial will help you install the dependencies needed for running the SVN checkout on your computer.

    Step 0a. Choosing an IDE.

    Worthy candidates are SciTE, which is pretty lightweight, Komodo Edit, somewhat heavy, based around the Mozilla code and with support for debugging and syntax autocompletion, and emacs.

    Step 0b. An informal introduction to the Windows Command Shell.

    This will be really important so read this if you do not know how to use the Windows Command prompt. A non-issue for Linux users since they universally know how to use their shell anyway.

    -> How to start:
    Start > Run, enter cmd and hit Enter or click OK.

    -> Commands you'll need to know for this tutorial:
    cd: changes the directory.

    Code:
    C:\Programming>cd smogon
    
    C:\Programming\smogon>_
    
    Instead of the directory name, a complete path can be entered. For instance, entering cd programming\smogon on C:\ will take you to C:\Programming\smogon.

    To go to the parent directory, enter cd ... To go to the drive root, enter cd \.

    An important note, File and directory names are case-insensitive.

    Another important note, You can use TAB to auto-complete directory names, e.g. cd pro<TAB> completes pro to programming. If there are more than one files or directories that match, then TAB cycles through them in alphabetical order. You can also hit TAB if there is nothing to autocomplete, in which case it simply cycles through the files and directories in an alphabetical order.

    Finally, you can copy output from the command prompt. To do that, right-click in the black region and select 'Mark'. Select the region you want to copy with the mouse and hit Enter to copy it to the clipboard.

    Step 1. Installing Python and MySQL

    Download Python 2.6 from http://www.python.org/download/ and install. The installation is pretty straightforward.

    Download MySQL 5.1 from http://dev.mysql.com/downloads/mysql/5.1.html and install. Please note that there is no need to register as a user; in fact, there's a rather concealed "No thanks, just take me to the downloads!" link just below the login/registration forms. Also note that you must click the link in order to see the list of mirrors. The installation is pretty straightforward, but you'll be presented with a configuration screen at the end of it. Just proceed with the default selected options, and enter a password you like for the root account (setting it to root is a bad idea, even if you have "Enable remote users to log on as root" checked off).

    Start MySQL by the shortcut in the start menu. Log on as root, then invoke the following commands:

    Code:
    mysql> CREATE USER smogon IDENTIFIED BY 'password';
    mysql> GRANT ALL ON *.* TO 'smogon'@'localhost';
    
    Replace 'password' with your password of choice. You could modify line 2 to GRANT ALL ON db TO 'smogon'@'localhost'; where db is the name of the database to which the access is to be provided (for instance dex or user, etc).

    Exit MySQL by typing exit and hitting enter.

    Step 2. Setting up the PATH.

    First of all, we need to set up the PATH variable to include the Python directory and the Scripts subdirectory, along with the directory where you will place the smogon checkout.

    Windows users:

    Press WinKey + PauseBreak or right click My Computer and select Properties to bring up the System Properties box. Under the Advanced tab, click Environment Variables to bring up another dialog box with two lists. Search the lists for an entry called PATH. If none exists, create one by clicking New and enter the following as the value. Else, select it and click Edit, and append a semicolon to the end of the value and then append the following to it.

    Code:
    C:\Python26;C:\Python26\Scripts;C:\Programming
    Similarly create another entry by the name of PYTHONPATH with the value C:\Programming

    (Replace C:\Python26 with the path where Python is installed. Replace C:\Programming with wherever you want to store the site code. Note that the site code will ultimately go in a folder called smogon under this folder. Do not replace C:\Programming with C:\)

    Linux users:

    Any good package manager should have placed python and all related paraphernalia in /usr/bin so the only change we should need to make to the system path is to add the Smogon specific stuff.

    To add the Smogon directory for the current session only simply run the following command:

    Code:
    export PATH=$PATH:/home/tim/Projects
    export PYTHONPATH=/home/tim/Projects
    Where /home/tim/Projects is the path to the directory where the Smogon folder will reside.

    To make the change permanent you need to add that line to the .bash_profile file in your home directory or adapt any existing declaration you might find there (it's a hidden file so you will need to press Ctrl+H in nautilus to make it appear) or do the same with /etc/profile to make the change systemwide.

    Step 3. Installing setuptools/easy_install

    To begin with, install easy_install since it will greatly aid the process of installing the other dependencies. The package page on PyPI doesn't have a Windows version for Python 2.6 yet, so we are going to manually install it. Download ez_setup.py. Start the command prompt and cd to where you downloaded the file, then invoke it by executing the following command:

    Code:
    C:\Programming>ez_setup.py
    
    This will produce a lot of output. If you get "Setuptools version 0.6c9 or greater has been installed." or anything that denotes success at the end of it, the process was successful. If not, post the log in this thread.

    Now enter 'easy_install' on the command prompt. You should get an output similar to the following:

    Code:
    C:\Programming>easy_install
    error: No urls, filenames, or requirements specified (see --help)
    
    C:\Programming>
    
    If you get something like this:

    Code:
    C:\Programming>easy_install
    'easy_install' is not recognized as an internal or external command,
    operable program or batch file.
    
    Then the PATH was not properly set up. Restart your computer and try executing the command, if it still doesn't work, you have set up the path incorrectly.

    Step 4. Installing the dependencies

    By this point, you should have a working copy of easy_install. Enter the following commands one-by-one at the command prompt:

    easy_install colubrid
    easy_install mako
    easy_install formencode
    easy_install sqlalchemy
    easy_install PIL
    easy_install paste

    That finishes up the list of dependencies that can be installed automatically with the help of easy_install, leaving us with two others, namely MySQLdb and advas.

    MySQldb: The project page on SourceForge still doesn't have a binary for Python 2.6, so we are going to get a custom-compiled one. Get the installer here, it'll require libguide40.dll and libmmd.dll, download them and place them in the lib\site-packages folder of your Python installation.

    advas: The process to install advas is relatively shorter. Download advas-0.2.3.tar.gz and unpack it. Then start the command prompt, cd to where you unpacked it, and run the following command:

    Code:
    C:\Programming\advas-0.2.3>setup.py install
    C:\Python26\lib\distutils\dist.py:266: UserWarning: Unknown distribution option:
     'release'
      warnings.warn(msg)
    running install
    running build
    running build_py
    creating build
    creating build\lib
    creating build\lib\advas
    ...
    byte-compiling C:\Python26\Lib\site-packages\advas\successorVariety.py to succes
    sorVariety.pyc
    byte-compiling C:\Python26\Lib\site-packages\advas\synonym.py to synonym.pyc
    byte-compiling C:\Python26\Lib\site-packages\advas\__init__.py to __init__.pyc
    running install_egg_info
    Writing C:\Python26\Lib\site-packages\advas-0.2.3-py2.6.egg-info
    
    C:\Programming\advas-0.2.3>_
    
    If you get an output like that, the process was successful.

    Only one last dependency now remains, lightorm. SVN checkout the following url:

    svn://linear.cc/lightorm/tags/0.02a

    Installation process is similar to advas.

    Step 5. Getting the SVN checkout.

    PM chaos with a username/password combo to get yourself an account. Then create a directory called 'smogon' under the Programming folder.

    Windows users:
    Get TortoiseSVN, install and restart your PC. Then open the Programming folder, right-click the smogon folder and select 'SVN Checkout...'. Enter svn://smogon.com/smogon/trunk under 'URL of repository' and click OK. The process will take some time.

    Linux users:
    Run 'svn co svn://smogon.com/smogon/trunk' under the smogon directory.

    Step 6. Last steps

    Create a file by the name of config.py under smogon with the following contents:

    Code:
    password="aaa"
    Where aaa is the password for the user "smogon".

    Run test.py and access localhost:9999 in your browser. If you see a layout similar to smogon's, the setup was successful. Post any problems here.
  2. Blue Kirby

    Blue Kirby Never back down.
    is a Tutor Alumnusis a Site Staff Alumnusis a Battle Server Admin Alumnusis a Programmer Alumnusis a Smogon IRC SOp Alumnusis a Smogon Media Contributor Alumnusis a Contributor Alumnusis an Administrator Alumnusis a Past SPL + WCoP Winner

    Joined:
    Nov 1, 2007
    Messages:
    2,841
    This is awesome - I remember the hiccups with my dependency list that caused me endless problems when trying to set up my environment.

    I'll give it a bit of a once over to see if there's anything that I found helpful that should be added, but knowing you, I doubt there will be!
  3. Shiv

    Shiv mostly harmless
    is a Site Staff Alumnusis a Smogon IRC AOp Alumnusis a Forum Moderator Alumnusis a Battle Server Moderator Alumnusis a Past WCoP Winner

    Joined:
    Apr 7, 2005
    Messages:
    5,870
    the problem with bk's dependency list was that 'bk is damb'.

    anyway, i'm glad you did this man, i remember how much of a pain this was a couple of times.

    good job :)
  4. chaos

    chaos
    is a member of the Site Staffis a Battle Server Administratoris a Programmeris a Smogon IRC SOPis a Contributor to Smogonis an Administratoris a Tournament Director Alumnusis a Researcher Alumnus
    Owner

    Joined:
    Dec 18, 2004
    Messages:
    9,658
    Huh? You should NOT change db.py, its under subversion control and you might commit the wrong stuff. Create a user account "smogon" in MySQL and grant it access to all of the dbs it needs

    also under windows the best choice of "ide" is pyscripter imo... other python ides here http://wiki.python.org/moin/IntegratedDevelopmentEnvironments
  5. ryubahamut

    ryubahamut
    is a Site Staff Alumnusis a Programmer Alumnusis a Forum Moderator Alumnusis a Contributor Alumnus

    Joined:
    Jan 18, 2007
    Messages:
    999
    Done, please check. PyScripter is good but I always experienced a queer problem with the software -- each time I opened the parenthesis for a function, the program would temporarily stop responding for about a second before presenting the tooltip with the descriptions for the arguments. If you can confirm you didn't face the problem, I'll edit the OP accordingly.
  6. chaos

    chaos
    is a member of the Site Staffis a Battle Server Administratoris a Programmeris a Smogon IRC SOPis a Contributor to Smogonis an Administratoris a Tournament Director Alumnusis a Researcher Alumnus
    Owner

    Joined:
    Dec 18, 2004
    Messages:
    9,658
    perhaps you should explain how to create the smogon user
  7. ryubahamut

    ryubahamut
    is a Site Staff Alumnusis a Programmer Alumnusis a Forum Moderator Alumnusis a Contributor Alumnus

    Joined:
    Jan 18, 2007
    Messages:
    999
    i did!

  8. chaos

    chaos
    is a member of the Site Staffis a Battle Server Administratoris a Programmeris a Smogon IRC SOPis a Contributor to Smogonis an Administratoris a Tournament Director Alumnusis a Researcher Alumnus
    Owner

    Joined:
    Dec 18, 2004
    Messages:
    9,658
    oh sorry, looks good then
  9. timw06

    timw06

    Joined:
    Jun 27, 2007
    Messages:
    543
    Any good package manager should have placed python and all related paraphernalia in /usr/bin so the only change we should need to make to the system path is to add the Smogon specific stuff.

    To add the Smogon directory for the current session only simply run the following command
    Code:
    export PATH=$PATH:/home/tim/Projects/Smogon
    To make the change permanent you need to add that line to the .bash_profile file in your home directory or adapt any existing declaration you might find there (it's a hidden file so you will need to press Ctrl+H in nautilus to make it appear) or do the same with /etc/profile to make the change systemwide.
  10. ryubahamut

    ryubahamut
    is a Site Staff Alumnusis a Programmer Alumnusis a Forum Moderator Alumnusis a Contributor Alumnus

    Joined:
    Jan 18, 2007
    Messages:
    999
    Thanks timw06 :)
  11. Blue Kirby

    Blue Kirby Never back down.
    is a Tutor Alumnusis a Site Staff Alumnusis a Battle Server Admin Alumnusis a Programmer Alumnusis a Smogon IRC SOp Alumnusis a Smogon Media Contributor Alumnusis a Contributor Alumnusis an Administrator Alumnusis a Past SPL + WCoP Winner

    Joined:
    Nov 1, 2007
    Messages:
    2,841
    It's a pretty small nitpick, but it might be worth mentioning why Eclipse isn't recommended (I haven't actually used it myself).
  12. chaos

    chaos
    is a member of the Site Staffis a Battle Server Administratoris a Programmeris a Smogon IRC SOPis a Contributor to Smogonis an Administratoris a Tournament Director Alumnusis a Researcher Alumnus
    Owner

    Joined:
    Dec 18, 2004
    Messages:
    9,658
    especially since doug uses it
  13. ryubahamut

    ryubahamut
    is a Site Staff Alumnusis a Programmer Alumnusis a Forum Moderator Alumnusis a Contributor Alumnus

    Joined:
    Jan 18, 2007
    Messages:
    999
    removed the bit about eclipse (my qualm was that it is too memory-intensive). finished the pythonpath bit, i was checking out incorrectly. should be complete now; is it ready to be moved to the programming subforum?
  14. X-Act

    X-Act np: Biffy Clyro - Shock Shock
    is a Site Staff Alumnusis a Programmer Alumnusis a Smogon IRC SOp Alumnusis a Researcher Alumnusis a CAP Contributor Alumnusis a Tiering Contributor Alumnusis a Contributor Alumnusis an Administrator Alumnus

    Joined:
    Feb 17, 2006
    Messages:
    4,675
    Thanks for this very useful resource!

    Also as an aside, this is my first post from my new mobile device. Hopefully it goes through correctly...
  15. Cathy

    Cathy

    Joined:
    Jul 11, 2007
    Messages:
    1,062
    Even though this is still sort of a work in progress, I've moved it to this forum so that everybody can benefit from it.
  16. mingot

    mingot free agent
    is a Site Staff Alumnusis a Battle Server Admin Alumnusis a Researcher Alumnusis a Smogon Media Contributor Alumnusis a Contributor Alumnusis an Administrator Alumnus

    Joined:
    Jun 15, 2008
    Messages:
    3,050
    A couple of things right off the bat (sorry, been meaning to post this for the last few days) --

    It might be helpful to know which version of Python should be installed.

    The MySql download would really like you to sign up and it can be hard to spot the "just take me to the download" link. A mention that such a thing exists would be helpful.

    More to come as I get further through the process.
  17. bearzly

    bearzly Shoddy Battle dev
    is a Battle Server Admin Alumnusis a Programmer Alumnusis a Forum Moderator Alumnus

    Joined:
    Apr 25, 2009
    Messages:
    230
    For any Mac OS X users:

    After a some mysterious crashes of the test server, I discovered that Python 2.6.2 was the problem. I had to install 2.6.1 to make it work. Alternatively, you could try using Python 2.5
  18. cjt107

    cjt107

    Joined:
    Aug 29, 2008
    Messages:
    14
    Colin told me to post here.

    how do you access this:
    ssh://monsan.to/lightorm/tags/0.02a
    ? I've done everything else but I can get to that.
  19. ryubahamut

    ryubahamut
    is a Site Staff Alumnusis a Programmer Alumnusis a Forum Moderator Alumnusis a Contributor Alumnus

    Joined:
    Jan 18, 2007
    Messages:
    999
    Sorry, the correct URL is svn://linear.cc/lightorm/tags/0.02a . Corrected, thanks :)
  20. cjt107

    cjt107

    Joined:
    Aug 29, 2008
    Messages:
    14
    when I run test.py it comes up with an error:
    Code:
    Traceback (most recent call last):
      File "C:\Users\Public\programming\smogon\trunk\test.py", line 15, in <module>
        from smogon.helpers import server
    ImportError: No module named smogon.helpers
    
    anyone know why this happened?
  21. ryubahamut

    ryubahamut
    is a Site Staff Alumnusis a Programmer Alumnusis a Forum Moderator Alumnusis a Contributor Alumnus

    Joined:
    Jan 18, 2007
    Messages:
    999
    Did you set up the PYTHONPATH variable? To check that, start the Python interpreter and enter
    Code:
    >>> os.getenv('PYTHONPATH')
    if the response doesn't resemble
    Code:
    'C:\\Users\\Public\\programming'
    then you need to set it up, consult the documentation for details.

    Alternatively, you could try doing a simple 'import smogon' at the interpreter and see if it succeeds.
  22. cjt107

    cjt107

    Joined:
    Aug 29, 2008
    Messages:
    14
    i set up PYTHONPATH as a system variable on Vista, does it need to be system or user?

    edit: when windows users svn "svn://smogon.com/smogon" it download the stuff into branches and trunk, which is why the site wasn't working. after moving the stuff into the folder which I svn checkout(ed) to, it worked, so surely you should svn "svn://smogon.com/smogon/trunk"?
  23. ryubahamut

    ryubahamut
    is a Site Staff Alumnusis a Programmer Alumnusis a Forum Moderator Alumnusis a Contributor Alumnus

    Joined:
    Jan 18, 2007
    Messages:
    999
  24. rory

    rory T^T
    is a Site Staff Alumnusis a Battle Server Moderator Alumnus

    Joined:
    Oct 13, 2007
    Messages:
    210
    This installs sqlalchemy 0.5.x which will not work with lightorm. You need to use 0.4.x which can be downloaded here

    This should be changed to svn://monsan.to/lightorm

    The default port is 9990
  25. jumpluff

    jumpluff princess
    is a Forum Moderatoris a Site Staff Alumnusis a Super Moderator Alumnusis a Smogon IRC SOp Alumnusis a Researcher Alumnusis a Smogon Media Contributor Alumnusis a Contributor Alumnusis a Battle Server Moderator Alumnus
    Moderator

    Joined:
    Aug 22, 2008
    Messages:
    6,171
    okay some things I noticed on vista:

    You can't change environment variables the normal way on Vista, so you'll need to open the User Accounts window (Control Panel) and click Change environment variables in the task pane.

    also:

    For those having inexplicable trouble with PATH when setting up easy install, use this .exe installer.

    (linear.cc -> monsan.to in everything)

Users Viewing Thread (Users: 0, Guests: 0)