1. Welcome to Smogon! Check out the Smogon Starters Hangout for everything you need to know about starting out in the community. Don't forget to introduce yourself in the Introduction and Hangout Thread, too!
  2. Welcome to Smogon Forums! Please take a minute to read the rules.

Programming [WIP] Using the Smogon Framework

Discussion in 'Technical Projects' started by ryubahamut, Jun 21, 2009.

  1. ryubahamut

    ryubahamut
    is a Site Staff Alumnusis a Programmer Alumnusis a Forum Moderator Alumnusis a Contributor Alumnus

    Joined:
    Jan 18, 2007
    Messages:
    999
    Posting this here for scrutiny. Please help me correct this draft for factual inaccuracies and cryptic tones!

    This is really, really, REALLY rough at the moment. I have marked some of the suspects in red, where assistance would be really welcome.

    TODO:

    • Add the use of __default__ in a Mapped class, under models.py. Need help with this.
    • What does request_uri=req.environ["PATH_INFO"] really do in a render call?!
    • Expand util.py.
    • Under standard library: cover smogon.helpers.util, smogon.helpers.validators, smogon.tools.*
    • Templating.
    • Make a test project!

    Using the Smogon Framework

    This tutorial will help you familiarize yourself with the Smogon development framework.

    1. Projects
    Smogon is organized into several "projects", which are essentially directories created on the site root level, for instance dex, or scms, or site. Note that these projects might or might not be accessible using their directory names. We'll address issue in the next section.

    Each project comprises of the following:
    • A directory in which the files of the project will be stored, at the Smogon root directory.
    • A directory which would store the template files for the project. It is created under the templates directory. It is advisable to name this directory the same as the project directory, though technically that is not a requirement.
    • __init__.py, under the project directory. This file (initially blank) is required so that the directory is treated as a module, that is, so that it can be imported using the import statement.
    • controllers.py, under the project directory. This file contains handlers that are responsible for dispatching every file in the project.

    Apart from these files, the following files can be created, depending on the project requirement:
    • models.py, if your project uses database functionality. This file contains all the classes required to interact with the database using the ORM.
    • auth.py, if your project requires the user to log in.
    • validators.py, if you need to deal with form data (submissions from a user, etc).
    • util.py, for storing miscellaneous functions that you might need.

    Note that these file names are conventional and hence it is recommended that they be used to ensure uniformity. We will deal with each of these files in detail later on.

    Apart from the files, a reference to the project must be made in app.py at the site root, by adding an entry to the following list:

    Code:
    projects = ["dex", "scms", "site", "shoddybattle", "qdb"]
    2. Components of a project

    a. __init__.py
    This file basically serves as a marker for modules. It is usually blank, except in special cases mentioned later. Any directory containing this file can be imported as a module. For instance consider a directory structure as shown:

    Code:
    /foo (folder)
    /foo/__init__.py
    /foo/bar.py
    
    Without __init__.py, statements like "from foo import bar" or "import foo" would yield an ImportError.

    Along with this, the file is also useful for storing code that needs to be present in all files of the module. For instance, check out the file __init__.py stored at the site root where it is used to "Override global namespace with some useful stuff."

    b. controllers.py
    Basically, this file contains functions to help generate pages, and mappings between URLs and functions. These will be made clear with the help of sample code:

    Code:
    
    from __future__ import absolute_import
    
    from colubrid.exceptions import *
    from colubrid import HttpResponse
    
    from smogon.helpers.template import render
    
    #*******************************************************************************
    # page display
    #*******************************************************************************
    
    def index(req):
        return render("blah/index.html",
            request_uri=req.environ["PATH_INFO"])
    
    def foo(req):
        return render("blah/foo.html",
            request_uri=req.environ["PATH_INFO"])
    
    #*******************************************************************************
    # mappings
    #*******************************************************************************
    
    from smogon.helpers.controllers import prefix
    
    MAPPINGS = prefix("blah", 
        ("", index),
        ("foo", foo),
    )
    
    Let's deal with MAPPINGS first. prefix is a function to make our lives easier (and the code prettier), by automatically suffixing the first argument before the first member of every following tuple (along with a slash; "blah/" in this case).

    Code:
    prefix("blah",
        ("", index),
        ("foo", foo),
    )
    
    is equivalent to
    Code:
    (
        ("blah/", index),
        ("blah/foo", foo),
    )
    
    By now you must have guessed what the second member of every tuple is: It is the function that is called when the corresponding URL regular expression is matched. Consider:

    Code:
    def foo(req):
        return render("blah/foo.html",
            request_uri=req.environ["PATH_INFO"])
    
    req is the request object passed to the function by Colubrid. This contains several important things, such as form data, URL parameters, cookies, and uploaded files.

    render is a function that generates an HTTP response object containing the rendered template code, i. e. template code executed and converted to HTML.

    When the page http://www.smogon.com/blah/foo is accessed, Colubrid sends the return value of the "foo" function to the client as an HTTP response.

    c. models.py
    We use this file to store all the classes used by the ORM if our project requires database functionality. Consider an example file from a fictitious project where we have a table containing buyable items and their costs.

    Code:
    from sqlalchemy.schema import *
    from sqlalchemy.types import *
    
    from lightorm import *
    
    from smogon.helpers.db import *
    
    ################################################################################
    
    __all__ = ["Item"]
    
    ################################################################################
    
    item_table = Table("items", 
        Column('id', Integer, primary_key=True, key="id"),
        Column('name', String),
        Column('cost', Integer),
        schema="dummydatabase")
    
    class Item(Mapped):
        __table__ = item_table
        __default__ = "id", "name", "cost"
        
        def __repr__(self):
            return """<Item #%s, %s, cost %s>""" % (self.id, self.name, self.cost)
    
    As simple as that. Let's deal with it, one section at a time.

    __all__ is used to list the objects to be imported, whenever a "from xyz import *" statement is made. It can be used outside of models.py too, but it is particularly helpful here because let's face it,
    Code:
    from smogon.qdb.models import *
    is better than
    Code:
    from smogon.qdb.models import Basher, Quote, Submission, Rating, Flagged
    item_table is a reference to a table in the database. The first argument is the name of the table as it appears in the database; every other unnamed argument is a reference to every column in the table, and the last, named argument "schema" is the database in which the table is present.

    Item is the object corresponding to any row from the table. __table__ is the corresponding Table object.
    __default does something I keep forgetting. Each column automatically becomes an attribute of the table, and thus the item cost can be retrieved by calling, for instance, tm50.cost.

    __repr__ is the function which is called whenever we try to perform repr(obj). The return value should ideally be a string containing the more salient attributes of the object.

    Once all this is done, the Item object is ready to be used. For instance, Item.q.order_by("cost DESC").limit("0, 10") returns the 10 most expensive items.

    d. auth.py
    This file is created if login functionality is required in the project. A detailed overview of the login API was given by chaos in another post, which is quoted here:

    e. validators.py
    Validators are needed whenever form data processing is involved. An example of such a scenario would be the SCMS, where changes to analyses are made and submitted. As the name suggests, they validate user input, and redirect the user back to the form page (then marked with error signs) if the input is incorrect.

    Validation is done with the help of validators. Validators match a given form field for a set of defined rules, and raise errors if the match doesn't succeed. A lot of such validators are defined under smogon.helpers.validators, I recommend you check them out.

    Each validator has a _to_python and a _from_python function. The former converts form data to Python objects, and the latter works the other way round. They do not need to be defined in your forms, however, if no out-of-the-ordinary behavior is to be implemented.

    Some of the more commonly used validators are:

    • String: Matches unicode strings.
    • HTMLString: Converts a string to HTML (i. e. replaces line breaks with <br> and the like)
    • Int: Matches numbers.
    • Regex: Matches regular expressions.

    A sample file would look thus (for the sake of the example, let's assume a form where a user enters the name and cost of a Pokéitem):

    Code:
    
    from sqlalchemy.sql import *
    
    from smogon.helpers.validators import *
    
    ################################################################################
    
    __all__ = ["AddItemForm"]
    
    ################################################################################
    
    class AddItemForm(Form):
        name = String(not_empty=True)
        cost = Int()
    
    Here, we have defined a validator for a form with two fields namely name and cost, with values matching string and number respectively. We can now use this validator to validate form data in the following way:

    Code:
    from smogon.helpers import validators
    from smogon.blah.models import Item
    
    if req.form:
        try:
            data = AddItemForm.to_python(req.form)
            form = validators.FormData()
            
            item = Item()
            
            item.name = data['name']
            item.cost = data['cost']
            
            item.save()
            
            render("blah/add_successful.html", 
                request_uri=req.environ["PATH_INFO"])
            
        except validators.FormError, e:
            form = e
            render("blah/errors.html",
                request_uri=req.environ["PATH_INFO"])
    else:
        render("blah/no_form.html",
            request_uri=req.environ["PATH_INFO"])
    
    The line if req.form verifies if form data has been submitted. If not, the page "blah/no_form" is rendered. Else, an attempt to validate the form is made. If the form passes validation, an item object is created, populated and saved, and a success notification is rendered. Else, an error page is thrown.

    Realistically, instead of 3 template pages as shown here, you'll likely require only one. Nonetheless, this should serve as a good enough demonstration of how you would deal with form data.

    f. util.py
    Not much to be said here. If you have any functions to define that don't fit anywhere else, you can dump them in here.

    3. The "Standard Library"

    a. Cache (smogon.helpers.cache)
    INCOMPLETE!!!!
    You will most likely not need them, as the cache is an expensive commodity and should be used with discretion. We'll go through this anyway.

    getset(key, fn): Checks if an item with the identifier key exists in the cache. If not, adds an entry to the cache with the identifier key containing the return value of fn.

    getset_multi(seq): Works similar to getset, except it accepts a list containing (key, fn) pairs.

    b. Controller (smogon.helpers.controllers)
    Just one function, prefix; check the controllers.py section for details.

    c. Database (smogon.helpers.db)
    INCOMPLETE!!!!
    Contains the framework for interfacing with the database.

    Table(name, *args, **kwargs): Constructor that returns a sqlalchemy.schema.Table object. name is the name of the table, *args is a list of Column objects, **kwargs include schema, which is the name of the database in which the table is present.

    d. Server (smogon.helpers.server)
    Essentially contains path returning functions.

    path: Absolute path to the Smogon directory.
    join_path(*args): Returns items of args joined with path using the OS-specific path separator ("/" for Linux, "\\" for Windows).
    join_public(*args): Equivalent to join_path("public", *args).
    join_cache(*args): Equivalent to join_path("cache", *args).

    e. Templates (smogon.helpers.template)
    Contains render, which is THE templating function.

    render(filename, content_type, status_code, **kwargs): Renders a template specified by filename. Note that filename is relative to the Smogon root. content_type is by default text/html, you might want to set it to text/plain or something for AJAX return values. status_code is 200 by default (HTTP OK), it can be set to implement status pages (as is done in smogon.site.controllers). **kwargs are named arguments that are directly passed to the page being rendered. More on that under templating.
  2. chaos

    chaos
    is a member of the Site Staffis a Battle Server Administratoris a Programmeris a Smogon IRC SOPis a Contributor to Smogonis an Administratoris a Tournament Director Alumnusis a Researcher Alumnus
    Owner

    Joined:
    Dec 18, 2004
    Messages:
    10,120
    ssh://monsan.to/lightorm should have stuff for lightorm, tags/0.02a is the latest version i think
  3. Shiv

    Shiv mostly harmless
    is a Site Staff Alumnusis a Smogon IRC AOp Alumnusis a Forum Moderator Alumnusis a Battle Server Moderator Alumnusis a Past WCoP Winner

    Joined:
    Apr 7, 2005
    Messages:
    5,874
    it was there even via svn btw:

    svn://monsan.to/lightorm/tags/0.02a
  4. ryubahamut

    ryubahamut
    is a Site Staff Alumnusis a Programmer Alumnusis a Forum Moderator Alumnusis a Contributor Alumnus

    Joined:
    Jan 18, 2007
    Messages:
    999
  5. Faltzer

    Faltzer

    Joined:
    May 5, 2006
    Messages:
    62
  6. Cathy

    Cathy

    Joined:
    Jul 11, 2007
    Messages:
    1,061
    I edited the contents of that post into the original post in the auth.py section.

Users Viewing Thread (Users: 0, Guests: 0)