Tutorial
Interested in trying out WebCore or want to see how it works? This tutorial will step you through the process of getting started with a simple one-function application, then walk you through the process of evolving your application to make use of cinje templates, custom views, database connectivity through MongoDB, and JSON serialization. The end result will be a fully functional (if basic) wiki, and you with the knowledge of exactly how WebCore services your requests.
Don't worry if you've not used MongoDB, the cinje template engine, or even Python before; we'll introduce the basics and show how easy web development can be so long as you are already familiar with the basic precepts of programming such as functions, classes, flow control, variables, etc.
First, you'll need a copy of Python 3 in order to follow this guide. Getting Python itself running is beyond the scope of this documentation; you may find your Linux or Mac already comes with it pre-installed. Once you do and are able to successfully invoke the interpreter in a terminal we can get going. Python 2 will mostly work, but only if the examples are adapted to the earlier language conventions. You may also wish to prepare a local copy of MongoDB for use later.
The next thing you'll need is a place to put your project. For the sake of simplicity we'll use the virtual environment support built-in to Python 3 in order to isolate the packages we'll be installing and provide a convenient "project folder" to contain the code we'll be writing. We will assume a UNIX-like computer for now, but you Windows users should be able to follow along without too much difficulty; let us know how it goes!
Opening a terminal, which should place you in the "home" (~) folder associated with your user account, let's make sure we have a place for projects in general then tell Python to create a new environment within, and a “src” folder for our application code:
mkdir -p Projects
python3 -m venv Projects/wiki
mkdir Projects/wiki/src
Now each time you wish to begin working on this project in a new terminal session you'll want to "activate" the environment; this will update your shell's PATH and set PYTHONHOME and friends to inform Python of where environment-installed packages are. Enter the project directory Python just created and "source" the activation script for your shell. The default will work in most cases:
cd Projects/wiki
source bin/activate
You might be able to abbreviate the second line as . bin/activate, or use shell integration to automatically activate and de-activate environments for you, such as virtualenvwrapper or something similar to this Zsh configuration, though these are entirely optional.
Now that we have a project we can install packages into, let's get the package manager updated and WebCore installed. The following will install the current version of WebCore and enable installation of a default set of development dependencies; packages useful in development that are less useful in production. Also installed are the cinje template engine and the Python MongoDB driver which we'll use a little later.
pip install -U setuptools pip
pip install "WebCore[development]" cinje pymongo
pip install git+https://github.com/marrow/web.dispatch.resource.git
pip install git+https://github.com/marrow/web.db.git
pip install git+https://github.com/marrow/mongo.git
Quotes are needed because of the brackets, which most shells will attempt to interpret as an expansion (like a wildcard), which won't work. See the README for a list of the tags that can be used there. What might seem to be "a lot" of packages will have been installed. (This includes two packages installed directly from git; these are unreleased as of the time of writing, providing generalized database access to WebCore applications.) The highlights include:
-
WebOb which WebCore uses to provide
context.requestandcontext.responseobjects, as well as the library implementing HTTP status code exceptions. Its documentation will be generally useful to keep handy. -
Backlash, a web-based interactive debugger and REPL shell.
-
Ipython and Ptpython improved terminal REPL shells. To use them together, start your REPL by running
ptipythoninstead of justpython. -
pudb which provides a full-screen console-based visual debugger, similar in styling to the debugger present in Borland IDEs under DOS. Where Backlash lets you find out what happened when something has already gone wrong, pudb lets you find out what your code is doing by stepping through it, setting break points, etc.
-
Testing tools including the pytest runner, pyflakes static analyzer / "linting" tool, and coverage calculator letting you know if all of your code is tested.
We'll touch on or use each of these as we go... and we're ready to begin!
The entry point of WebCore is its Application class; it conducts the orchestra that is your application. You instantiate this class to configure the framework, and it is called by the web server to service each request. It speaks Python's Web Server Gateway Interface protocol in the role of an application, allowing applications you develop to leverage the large existing ecosystem of severs and server bridges. WebCore's self-organizing extension architecture can manage and integrate WSGI "middleware" components, and you can even embed other WSGI applications, too.
We're first going to need an Application instance, then. Create a new file in the src folder (changing your terminal's working folder to that directory, too) within the environment, name it run.py, and populate it like this:
# Get a reference to the Application class.
from web.core import Application
# This is our WSGI application instance.
app = Application("Hi.")
# If we're run as the "main script", serve our application over HTTP.
if __name__ == "__main__":
app.serve('wsgiref')The above is basically a long-form version of the minimal example from the README splitting the process into three clear steps: import, instantiate, serve. Currently we pass in "Hi." as our "website root", but we'll expand on this in the next section. For now, save the file out and run it in a shell:
python run.py
This will invoke the Python interpreter from our project environment and execute the above script. The resulting output should read something like:
serving on http://127.0.0.1:8080
Now you can open that link up in a browser and see your first live application saying hello. Before adding much to this file, however, we need to decide on how to structure the project.
Small utility apps might be able to get away with storing all their code in a single file, but if you ever plan on coming back to your code to update it later it’s a substantially better idea to structure your code into logical components. A basic wiki, luckily, has very few components.
Because WebCore is so absolutely minimal it doesn't offer much in the way of restrictions on how you structure your application code. WebCore considers anything it executes within your code to be just that, "application code", excluding extensions, which are extensions. Internally some terms used may seem to relate to "model, view, controller" separation, and while WebCore can be used in that way, our example here will be a bit simpler. WebCore's concept of a view, for example, never directly communicates with application code, nor do extensions.
With that understanding, which is elaborated on in the next section, create a wiki folder under the src folder, with a public folder under the wiki one, and touch (create empty) the following files under that wiki folder:
-
article.py— code relating to individual wiki pages -
extension.py— our custom application extension -
template.py— a place for our template code -
wiki.py— this will contain our "root" class
This can be quickly accomplished with two terminal commands after stopping the current server by pressing ⌃C (that is, the c key while holding the control key):
mkdir -p wiki/public
touch wiki/{article,extension,template,wiki}.py
And that's it. We'll fill out these files as we go.
This might be considered the controller in MVC terminology as typically used by web frameworks. Some, such as Pyramid, refer to this as view due to how tenuous it is applying the MVC pattern to the web. Django is a MVT (model-view-template) framework for similar reasons. We leave this distinction up to you, as not all applications benefit from a rigid structure.
WebCore uses dispatch, defaulting to object dispatch, to look up an endpoint to use for a given request, using the root object you pass to Application as the starting point for the search. An endpoint may either be a callable, such as a plain function, instance method, or executable instance (a class with __call__ method), or even a static value such as the "Hi." we're currently passing in. These endpoints represent web-accessible resources.
Update wiki.py to contain the following:
# HTTP status code exception for "302 Found" redirection.
from webob.exc import HTTPFound
class Wiki:
"""Basic multi-article editable wiki."""
def __init__(self, context=None):
pass # We don't use the context at all.
def __call__(self):
"""Called to handle direct requests to the web root."""
return HTTPFound(location='/Home') # Redirect.This illustrates the basic protocol your endpoints will need to understand. First, we import an HTTP status code exception used later. Next, we define a class we'll use as our root object, exposed to the web. Python supports the idea of a "doc string", which is a string created without assignment as the first instruction in a new module, class, or function scope. We're using a triple-quoted string here, which lets the "documentation" easily flow across multiple lines without having to worry about line continuations, joining strings, etc. Documentation text like this is visible in IDEs, can be used by automatic documentation generators, and can be seen in a REPL shell by running help(Wiki).
At the start of a request the class constructor (__init__) is called and is passed the current RequestContext as its first positional argument. As noted, we don't need it right now, but if you did you might assign it to self._ctx. Python doesn't like passing unknown arguments to functions, and since we don't even use it we default the value to None to make testing easier. If you forget to include an initializer accepting that argument your application might seem to start, but you'll encounter a TypeError exception and 500 Internal Server Error when attempting to access it.
The __call__ method is the callable endpoint used when the class itself is the target of the request; because we'll be using this as our root object, __call__ here will be executed for any request to /.
Save this file and run ptipython in your shell. Each time you run it you will be presented with an interface similar to the ordinary python REPL, but with 100% more color, a summary of some command shortcuts, and a status bar with additional keyboard shortcuts, similar to this:
We should be able to import our newly constructed Wiki class, instantiate it, then call it like WebCore would in order to inspect the result. Transcribe this one command at a time and watch for lines with red Out prefixes:
from wiki.wiki import Wiki
Wiki()
_()
_.locationThis is exploiting a feature of the REPL shell where the variable _ refers to the result of the previous expression, so when calling _() as a function, it's pointing at the instance of Wiki constructed on the line prior. Similarly, _.location is referring to the location attribute (where to redirect to) of the returned HTTPFound object. Your output should be virtually identical to the following, barring a different memory address for our instance:
To exit the interactive shell press ⌃D and confirm. Now let's wire this into our run.py file and see if we're redirected in a real browser. While we're at it, we can set up a few of the extensions we'll be using. Open run.py up and add the following imports to the top:
# Get references to web framework extensions.
from web.ext.annotation import AnnotationExtension
from web.ext.debug import DebugExtension
# Get a reference to our Wiki root.
from wiki.wiki import WikiOkay, admittedly, that last line is a little silly. Now update the Application() instantiation to reference our new root and extensions:
# This is our WSGI application instance and extension configuration.
app = Application(Wiki, extensions=[
AnnotationExtension(),
DebugExtension(),
])Save the file, then run it as a Python script to start the development server back up:
python run.py
Open up your browser again (or refresh the existing tab you had open saying "Hi.") and... it's a blank page. If you look at the logs...
127.0.0.1 - - [...] "GET / HTTP/1.1" 302 0
ERROR:web.core.application:__call__() takes 1 positional argument but 2 were given
127.0.0.1 - - [...] "GET /Home HTTP/1.1" 404 0
We can see the redirection (302 response) on the request to the root, then WebCore tried calling Wiki.__call__ with an unknown argument, finally returning 404 Not Found for /Home. Seems we haven't explicitly defined what Home is yet, and __call__ doesn't know, either!
This is a potentially confusing feature for new users, is very important to understand, and is best described as WebCore specifically searching for the deepest object, and calling the deepest object found if it is callable. This is doubly important if your __call__ method accepts unlimited positional arguments by declaring *args (or similar single-star prefixed label; a catch-all) in your argument list, as suddenly there can be nothing "missing" below that object. You would want to manually perform validation, in that case, and return HTTPNotFound if the unknown path is actually unknown.
With that out of the way, it's time to introduce...
Now that we have a "web root", and it's trying to point the user at a default "Home" page, we need to inform WebCore how we want it to resolve child paths. With the note from the end of the last section, you might think "hey, I can just add a 'page' argument to __call__ and it'll be passed in, right?" You'd be correct, that would potentially work, but remember the terminology we've used: __call__ is an endpoint representing the final destination for the request, and in the end you'll be able to do more than just read articles.
Open up article.py and add the following, our initial basic Article object:
class Article:
"""A wiki article."""
def __init__(self, context, name):
self._name = name # Save this for later.
def __call__(self, _METHOD='GET'):
return "I'm an article named " + self._nameThen update the wiki.py file to import it at the top:
# Get a reference to our Artlce handler.
from .article import ArticleThis uses a package-relative import to avoid needing to reference the full path for every import within the same package, the package being the src/wiki directory in our case. Next, add this method to the Wiki class, remembering to match the class indentation level:
def __getattr__(self, name):
"""Look up an otherwise unknown child path."""
return Article(self._ctx, name)Looks like we're going to need that context object after all! Remove the pass from the __init__ method and assign the context to self to satisfy the fact we're trying to pass it to the Article constructor:
def __init__(self, context=None):
self._ctx = contextThe __getattr__ method is a standard part of Python's class interface. It is called any time the language attempts to look up an attribute on the object that doesn't actually exist, giving your code a chance to intervene and redirect the lookup. In the event the attribute really, really doesn't exist, your code should raise AttributeError() to declare that attribute lookup failed; this will let WebCore continue and try other approaches to resolving the request. You can also raise an HTTPException subclass such as HTTPNotFound, but this approach will prevent the framework from attempting fallback or alternate dispatch approaches.
If you refresh your browser, you'll notice that nothing has changed. This is normal; Python aggressively caches your code and actually makes true module reloading nearly impossible. To apply the changes, press ⌃C (control and c), then up arrow to recall the last command, and enter to re-run it. Refreshing now, you should see a very plain page with the sentence:
I'm an article named Home
Success! You can even try out other paths to get similar results. It's a little bit spartan, though. Let's try getting a little HTML in there.
Now that we plan on using templates, we're going to dig into the cinje template engine package a bit. According to its README now that it's installed, all that should be nessicary is to import cinje prior to any attempt to import a template function from one of our cinje-encoded template modules. The easiest place to accomplish this is in the run.py file. Add this import prior to the wiki imports:
# Register the cinje template encoding.
import cinjeWith that out of the way, let's open up template.py and add our page template, which our other templates will use:
# encoding: cinje
# Get the standard HTML5 page template.
: from cinje.std.html import page as _page, default_header, default_footer
: def page title, header=default_header, footer=default_footer, metadata=[], styles=[], scripts=[], **attributes
: """The outermost Wiki page template."""
: using _page title, header=header, footer=footer, metadata=metadata, styles=styles, scripts=scripts, **attributes
: yield
: end
: endAs the comments describe, we wrap the call to the standard HTML5 page template. This will allow us to add various bits of metadata, JavaScript, and CSS to our pages later. The template for a single page might be:
: def article title, content
: """The presentation for a single article."""
: using page "Wiki: " + title, lang="en"
<article>
# The least secure approach possible.
#{content}
</article>
: end
: endThis defines a template function accepting a title and chunk of content, which passes a modified title along to the overall page template, defines that the content is in English, then emits the content as raw HTML within an <article> tag. That's it. Now to make use of this we will need to import the function we just defined.
Open up article.py once more and add the following import after the rest:
# Get a reference to our template function.
from .template import articleNow we can update the handler to use the template. It can be replaced with:
def __call__(self):
return article(self._name, "<h1>I'm an article named " + self._name + "!</h1>")Restart the development server and refresh to see the awesomeness that is HTML with a title tag. It's still a bit spartan, and certainly lacking in interactivity, so let's get the ball rolling on making the page editable.
Now that we want to add interactivity to our application we'll need a place to serve static assets from. We can add to the Wiki class to allow our files to be served in development. In production you would tell Nginx or other front-end server to serve files from this directory or distribute them on a CDN. For more complicated setups you can use WebAssets to build, combine, and optimize your files. Add the following import to the top of wiki.py:
# Static file serving for development.
if __debug__:
from web.app.static import staticThen add the following to the Wiki class, above the __init__ method:
if __debug__:
try:
public = static(join(dirname(__file__), 'public'))
except:
passAnd you're done.
First you'll want to download the following files into the public folder, with the given names:
Add the following to template.py above the : def page function declaration:
: default_scripts = ['/public/{}.min.js'.format(i) for i in ('medium-editor', 'autolist', 'RIP')]
: default_styles = ['/public/styles.css']This will include a Vanilla JS WYSIWYG editor styled after Medium.com's editor. Now we need to add our own custom script code. To do this we're going to provide our own "footer" to override default_footer, since our <script> tag should come at the end of the page, just prior to </body>.
Add the following function above our : def page page template:
: def wiki_footer styles=[], scripts=[]
: use default_footer styles, scripts
<script>
var lastEdit = null,
editor = new MediumEditor('article', {
placeholder: {text: "Enter your article text here.", hideOnClick: true},
extensions: {
autolist: new AutoList(),
}});
editor.subscribe('editableInput', function(event, editable) {
// Clear any existing pending save.
if ( lastEdit ) window.clearTimeout(lastEdit);
// Wait 500ms before actually saving.
lastEdit = window.setTimeout(function() {
// Save the change with the server.
RIP.POST(window.location.pathname, {content: editable.getContent()});
}, 500);
});
}});
</script>
: endThen update the footer=default_footer reference on the page template to point instead at our wiki_footer.
Now restart the application and refresh again; behold, a beautifually editable page. Click, start typing, select some text. Try typing a list naturally (starting with 1. for a numeric list, or * / - to start a bulleted list), with placeholder for page title and body if missing.
The mystery as to the earlier _METHOD argument defined for our Article class' __call__ is resolved: the RIP library (0.4kb) we are using uses this to pass the REST verb.
Now we need somewhere to save the data.