Programming [WIP] Using the Smogon Framework

#1
Posting this here for scrutiny. Please help me correct this draft for factual inaccuracies and cryptic tones!

This is really, really, REALLY rough at the moment. I have marked some of the suspects in red, where assistance would be really welcome.

TODO:

  • Add the use of __default__ in a Mapped class, under models.py. Need help with this.
  • What does request_uri=req.environ["PATH_INFO"] really do in a render call?!
  • Expand util.py.
  • Under standard library: cover smogon.helpers.util, smogon.helpers.validators, smogon.tools.*
  • Templating.
  • Make a test project!

Using the Smogon Framework

This tutorial will help you familiarize yourself with the Smogon development framework.

1. Projects
Smogon is organized into several "projects", which are essentially directories created on the site root level, for instance dex, or scms, or site. Note that these projects might or might not be accessible using their directory names. We'll address issue in the next section.

Each project comprises of the following:
  • A directory in which the files of the project will be stored, at the Smogon root directory.
  • A directory which would store the template files for the project. It is created under the templates directory. It is advisable to name this directory the same as the project directory, though technically that is not a requirement.
  • __init__.py, under the project directory. This file (initially blank) is required so that the directory is treated as a module, that is, so that it can be imported using the import statement.
  • controllers.py, under the project directory. This file contains handlers that are responsible for dispatching every file in the project.

Apart from these files, the following files can be created, depending on the project requirement:
  • models.py, if your project uses database functionality. This file contains all the classes required to interact with the database using the ORM.
  • auth.py, if your project requires the user to log in.
  • validators.py, if you need to deal with form data (submissions from a user, etc).
  • util.py, for storing miscellaneous functions that you might need.

Note that these file names are conventional and hence it is recommended that they be used to ensure uniformity. We will deal with each of these files in detail later on.

Apart from the files, a reference to the project must be made in app.py at the site root, by adding an entry to the following list:

Code:
projects = ["dex", "scms", "site", "shoddybattle", "qdb"]
2. Components of a project

a. __init__.py
This file basically serves as a marker for modules. It is usually blank, except in special cases mentioned later. Any directory containing this file can be imported as a module. For instance consider a directory structure as shown:

Code:
/foo (folder)
/foo/__init__.py
/foo/bar.py
Without __init__.py, statements like "from foo import bar" or "import foo" would yield an ImportError.

Along with this, the file is also useful for storing code that needs to be present in all files of the module. For instance, check out the file __init__.py stored at the site root where it is used to "Override global namespace with some useful stuff."

b. controllers.py
Basically, this file contains functions to help generate pages, and mappings between URLs and functions. These will be made clear with the help of sample code:

Code:
from __future__ import absolute_import

from colubrid.exceptions import *
from colubrid import HttpResponse

from smogon.helpers.template import render

#*******************************************************************************
# page display
#*******************************************************************************

def index(req):
    return render("blah/index.html",
        request_uri=req.environ["PATH_INFO"])

def foo(req):
    return render("blah/foo.html",
        request_uri=req.environ["PATH_INFO"])

#*******************************************************************************
# mappings
#*******************************************************************************

from smogon.helpers.controllers import prefix

MAPPINGS = prefix("blah", 
    ("", index),
    ("foo", foo),
)
Let's deal with MAPPINGS first. prefix is a function to make our lives easier (and the code prettier), by automatically suffixing the first argument before the first member of every following tuple (along with a slash; "blah/" in this case).

Code:
prefix("blah",
    ("", index),
    ("foo", foo),
)
is equivalent to
Code:
(
    ("blah/", index),
    ("blah/foo", foo),
)
By now you must have guessed what the second member of every tuple is: It is the function that is called when the corresponding URL regular expression is matched. Consider:

Code:
def foo(req):
    return render("blah/foo.html",
        request_uri=req.environ["PATH_INFO"])
req is the request object passed to the function by Colubrid. This contains several important things, such as form data, URL parameters, cookies, and uploaded files.

render is a function that generates an HTTP response object containing the rendered template code, i. e. template code executed and converted to HTML.

When the page http://www.smogon.com/blah/foo is accessed, Colubrid sends the return value of the "foo" function to the client as an HTTP response.

c. models.py
We use this file to store all the classes used by the ORM if our project requires database functionality. Consider an example file from a fictitious project where we have a table containing buyable items and their costs.

Code:
from sqlalchemy.schema import *
from sqlalchemy.types import *

from lightorm import *

from smogon.helpers.db import *

################################################################################

__all__ = ["Item"]

################################################################################

item_table = Table("items", 
    Column('id', Integer, primary_key=True, key="id"),
    Column('name', String),
    Column('cost', Integer),
    schema="dummydatabase")

class Item(Mapped):
    __table__ = item_table
    __default__ = "id", "name", "cost"
    
    def __repr__(self):
        return """<Item #%s, %s, cost %s>""" % (self.id, self.name, self.cost)
As simple as that. Let's deal with it, one section at a time.

__all__ is used to list the objects to be imported, whenever a "from xyz import *" statement is made. It can be used outside of models.py too, but it is particularly helpful here because let's face it,
Code:
from smogon.qdb.models import *
is better than
Code:
from smogon.qdb.models import Basher, Quote, Submission, Rating, Flagged
item_table is a reference to a table in the database. The first argument is the name of the table as it appears in the database; every other unnamed argument is a reference to every column in the table, and the last, named argument "schema" is the database in which the table is present.

Item is the object corresponding to any row from the table. __table__ is the corresponding Table object.
__default does something I keep forgetting. Each column automatically becomes an attribute of the table, and thus the item cost can be retrieved by calling, for instance, tm50.cost.

__repr__ is the function which is called whenever we try to perform repr(obj). The return value should ideally be a string containing the more salient attributes of the object.

Once all this is done, the Item object is ready to be used. For instance, Item.q.order_by("cost DESC").limit("0, 10") returns the 10 most expensive items.

d. auth.py
This file is created if login functionality is required in the project. A detailed overview of the login API was given by chaos in another post, which is quoted here:

The Smogon API Part 2: Handling logins

[...]The new Smogon site has a very
comprehensive API to make programming as absolutely painless as possible. The
first of a long series of documents detailing this API is contained... within this post!
I'll try to do a new one every once in a while, as eventually when I'm done I'd like it
if other programmers helped me without the site :-) This documentation is available
via Python help system as well, but I figure it'd be even better to provide a
bonafide tutorial on this shit.

[...]

A quick, and dirty example to start things off!

Code:
from smogon.helpers.user import cond_login

def index(req):
    login = cond_login(req)
    print login.user_username
A surprising amount of functionality for 4 lines of code! What this sample does
is print the logged in user's username. For the more attentive, you may be wondering...
"what if they aren't logged in?" "what if they are banned?" and other questions.
That's the beauty of the login API; it ALWAYS returns a valid login object. If the user
can't login for whatever reason then an exception is raised and we never reach the
print statement. Pretty awesome huh? The defaults for helpers.user.cond_login take
the user to a login page that redirects to the current page if a session can't be found.

Customizing cases!

Sometimes you don't want the default functionality every single time. What if you don't give a
crap about whether they can login or not? Try the uncond_login function. uncond_login, despite the name,
CAN raise exceptions, they are just of a different nature. It returns None if the user can't login
but has the ability to raise exceptions if a user is banned, for instance.

Code:
    login = uncond_login(req)
    print login
This function is useful for, say, when you never really use the login object but just want
to pass the information to a template so it can say who is currently logged in.

It's possible to give custom logic to both cond_login and uncond_login. Let's take a look
at their function signatures first though:

Code:
def uncond_login(req, **kwargs)
def cond_login(req, **kwargs)
req is an instance of the colubrid Request object. It's passed as the first parameter
to every HTTP request function, so just blindly pass it on.

Each function can also take a series of keyword arguments. These arguments are functions
that return exceptions. Any keyword argument that is left out resorts to the defaults
for that function. You can pass True if you want the function to raise a default exception.

Some examples:

Code:
from colubrid.exceptions import HttpFound, BadRequest
from smogon.helpers.user import cond_login, uncond_login, NotLoggedIn, Banned

# Redirect all banned users to banned.html, print login object of logged in and unlogged in users.
def example1(req):
    login = uncond_login(req, banned=lambda req, login: HttpFound("/banned.html"))
    print login

# Hmm... redirect users to a login screen if they aren't logged in, but print a banned msg if they are banned.  
def example2(req):
    try:
        login = cond_login(req, banned=True)
        print "Welcome to the club, non banned user."
    except Banned, e:
        print e.message

# Make sure the user knows that he is new. 
def example3(req):
    try:
        login = cond_login(req, not_logged_in=True)
        print "Hey buddy :D"
    except NotLoggedIn, e:
        print "WHATS UP YOU'RE NEW, GO LOGIN $_$"
The main reason for passing True to these arguments is when you want to render a different
template depending on a condition, since you can't raise that as an exception.
Other logic should be handled via functions that return colubrid exceptions.
e. validators.py
Validators are needed whenever form data processing is involved. An example of such a scenario would be the SCMS, where changes to analyses are made and submitted. As the name suggests, they validate user input, and redirect the user back to the form page (then marked with error signs) if the input is incorrect.

Validation is done with the help of validators. Validators match a given form field for a set of defined rules, and raise errors if the match doesn't succeed. A lot of such validators are defined under smogon.helpers.validators, I recommend you check them out.

Each validator has a _to_python and a _from_python function. The former converts form data to Python objects, and the latter works the other way round. They do not need to be defined in your forms, however, if no out-of-the-ordinary behavior is to be implemented.

Some of the more commonly used validators are:

  • String: Matches unicode strings.
  • HTMLString: Converts a string to HTML (i. e. replaces line breaks with <br> and the like)
  • Int: Matches numbers.
  • Regex: Matches regular expressions.

A sample file would look thus (for the sake of the example, let's assume a form where a user enters the name and cost of a Pokéitem):

Code:
from sqlalchemy.sql import *

from smogon.helpers.validators import *

################################################################################

__all__ = ["AddItemForm"]

################################################################################

class AddItemForm(Form):
    name = String(not_empty=True)
    cost = Int()
Here, we have defined a validator for a form with two fields namely name and cost, with values matching string and number respectively. We can now use this validator to validate form data in the following way:

Code:
from smogon.helpers import validators
from smogon.blah.models import Item

if req.form:
    try:
        data = AddItemForm.to_python(req.form)
        form = validators.FormData()
        
        item = Item()
        
        item.name = data['name']
        item.cost = data['cost']
        
        item.save()
        
        render("blah/add_successful.html", 
            request_uri=req.environ["PATH_INFO"])
        
    except validators.FormError, e:
        form = e
        render("blah/errors.html",
            request_uri=req.environ["PATH_INFO"])
else:
    render("blah/no_form.html",
        request_uri=req.environ["PATH_INFO"])
The line if req.form verifies if form data has been submitted. If not, the page "blah/no_form" is rendered. Else, an attempt to validate the form is made. If the form passes validation, an item object is created, populated and saved, and a success notification is rendered. Else, an error page is thrown.

Realistically, instead of 3 template pages as shown here, you'll likely require only one. Nonetheless, this should serve as a good enough demonstration of how you would deal with form data.

f. util.py
Not much to be said here. If you have any functions to define that don't fit anywhere else, you can dump them in here.

3. The "Standard Library"

a. Cache (smogon.helpers.cache)
INCOMPLETE!!!!
You will most likely not need them, as the cache is an expensive commodity and should be used with discretion. We'll go through this anyway.

getset(key, fn): Checks if an item with the identifier key exists in the cache. If not, adds an entry to the cache with the identifier key containing the return value of fn.

getset_multi(seq): Works similar to getset, except it accepts a list containing (key, fn) pairs.

b. Controller (smogon.helpers.controllers)
Just one function, prefix; check the controllers.py section for details.

c. Database (smogon.helpers.db)
INCOMPLETE!!!!
Contains the framework for interfacing with the database.

Table(name, *args, **kwargs): Constructor that returns a sqlalchemy.schema.Table object. name is the name of the table, *args is a list of Column objects, **kwargs include schema, which is the name of the database in which the table is present.

d. Server (smogon.helpers.server)
Essentially contains path returning functions.

path: Absolute path to the Smogon directory.
join_path(*args): Returns items of args joined with path using the OS-specific path separator ("/" for Linux, "\\" for Windows).
join_public(*args): Equivalent to join_path("public", *args).
join_cache(*args): Equivalent to join_path("cache", *args).

e. Templates (smogon.helpers.template)
Contains render, which is THE templating function.

render(filename, content_type, status_code, **kwargs): Renders a template specified by filename. Note that filename is relative to the Smogon root. content_type is by default text/html, you might want to set it to text/plain or something for AJAX return values. status_code is 200 by default (HTTP OK), it can be set to implement status pages (as is done in smogon.site.controllers). **kwargs are named arguments that are directly passed to the page being rendered. More on that under templating.