PageMaker » History » Version 10
Version 9 (Elmer de Looff, 2013-02-06 17:40) → Version 10/17 (Arjen Pander, 2015-03-10 15:33)
PageMaker is the _Controller_ of the MVC approach in µWeb. After a request is received by the web server (either [[Standalone]] or [[Apache]]) and wrapped inside a [[Request]] object, it is [[Request Router|routed]] here to be answered.
In the PageMaker, there might be database lookups done through the [[Model|data abstraction layer (model)]] and likely output is sent back making use of the [[TemplateParser]].
{{toc}}
h1. A very minimal PageMaker
In the simplest form, a PageMaker for a project subclasses from µWeb's default @PageMaker@ class and provides its own methods to handle requests. The full source for this would look something like this:
<pre><code class="python">
#!/usr/bin/python
"""PageMaker demonstration module"""
# uWeb framework
import uweb
class Minimalist(uweb.PageMaker):
def Index(self):
return 'Welcome to our website, it is still very much under construction.'
def Catchall(self, path):
return 'The requested page %r does not exist yet' % path
</code></pre>
h1. DebuggingPageMaker
Before we do anything else, during development you are _strongly advised_ to use µWeb's @DebuggingPageMaker@. This has a lot of additional features for when something goes wrong on the server side. When the regular PageMaker runs into a server side error, it returns a very plain HTTP 500 response:
<pre>
INTERNAL SERVER ERROR (HTTP 500) DURING PROCESSING OF '/'
</pre>
Where @'/'@ is the path requested by the client. When running @DebuggingPageMaker@ there is a significantly more helpful (for the developer at least) page whenever an internal server error is encountered. It will show a full stack trace, the local variables on each stack level (typically at the point of calling another function), which helps to arrive to the point of failure more quickly.
To use, just subclass your PageMaker from DebuggingPageMaker:
<pre><code class="python">
class Minimalist(uweb.DebuggingPageMaker)
</code></pre>
Example Internal Server Error response "as image":http://bugs.underdark.nl/attachments/download/185/http500_full.png or in the "µWeb demo project":http://info.underdark.nl/haltandcatchfire?debug
In all cases, an internal server error will cause a full stacktrace to be logged in the log file database.
h1. Templateparser
The µWeb *[[TemplateParser]]* is available on the standard PageMaker instance. When using PageMaker, an instantiated TemplateParser instance is available through the @parser@ member of PageMaker. Basic usage looks like this:
<pre><code class="python">
import uweb
import time
class TemplateDemo(uweb.PageMaker):
def VersionPage(self):
return self.parser.Parse(
'version.utp', year=time.strftime('%Y'), version=uweb.__version__)
</code></pre>
The example template for the above file could look something like this:
<pre><code class="html">
<!DOCTYPE html>
<html>
<head>
<title>µWeb version info</title>
</head>
<body>
<p>µWeb version [version] - Copyright 2010-[year] Underdark</p>
</body>
</html>
</code></pre>
And would result in the following output:
<pre><code class="html">
<!DOCTYPE html>
<html>
<head>
<title>µWeb version info</title>
</head>
<body>
<p>µWeb version 0.12 - Copyright 2010-2012 Underdark</p>
</body>
</html>
</code></pre>
Full documentation, with plenty of example template uses can be found on the [[TemplateParser|TemplateParser wiki-entry]].
h2. Template directory configuration
By default, template are loaded from the 'templates' directory that is expected to be on the same path as the pagemaker module. If your pagemaker is located on @/var/www/uweb_project/project.py@, then templates will be automatically loaded from @/var/www/uweb_project/templates/@.
To change the default template loading path, define a new path in the class variable @TEMPLATE_DIR@. This should be a relative path (and defaults to @'templates'@).
h1. Serving static content
Your website most likely has a few static files that need to be served up. If you have a large website you would run many of these from a separate domain (to reduce the amount of overhead from a heavy web server and complex processes), but often there are at least some files that need to be served up from the local disk.
µWeb has built-in facilities to serve static files, which prevent filesystem traversal by those lesser-behaved browsers. A browser requesting http://example.com/../secret_configuration.txt should *not* get the keys to your database server. The static handler has a base (configurable) directory from which all static content is served. For a client it is impossible to 'browse' up from that directory, preventing these leaks of information.
The MIME-Type for the content will be determined using the @mimetypes@ module (available by default in Python), based on the file's extension.
h2. Static handler demonstration
In your router configuration, add a route the directs to the static handler:
<pre><code class="python">
ROUTES = (
...
('/images/(.*)', 'Static'),
...
)
</code></pre>
This will cause the following behaviour:
* A browser requesting http://example.com/images/fish.jpg will be presented with the content from @'static/fish.jpg'@
* A browser requesting http://example.com/images/../../secret.txt will be presented with the content from @'static/secret.txt'@ (if it exists)
h2. Static directory configuration
As can be seen in the previous example, the content is served from the @'static'@ directory (this is not dependent on the selected route. This path is relative to the absolute path of the PageMaker itself. If the PageMaker module exists on @'/var/www/project/controller.py'@ then the default static directory is @'/var/www/project/static/'@.
This default path can be changed by setting the @PUBLIC_DIR@ class variable of PageMaker. The path can be made absolute simply by providing one:
<pre><code class="python">
import uweb
class DifferentPublic(uweb.PageMaker):
PUBLIC_DIR = '/var/www/project/public_http'
</code></pre>
h2. 404 on static content
Whenever a request for static content (through @Static@) cannot be fulfilled, the method @_StaticNotFound@ is called, with the requested relative path as the sole argument. The default response for which is a simple plain text:
<pre><code class="python">
def _StaticNotFound(self, _path):
message = 'This is not the path you\'re looking for. No such file %r' % (
self.req.env['PATH_INFO'])
return response.Response(message, content_type='text/plain', httpcode=404)
</code></pre>
Override this page if you want to provide your user with a more informative or styled response.
h1. Model admin
µWeb comes with a minimal admin interface based on the [[model]]
h2. Add the Admin mixin class to PageMaker
This is as simple as adding the Admin mixin class as one of the ancestors of your PageMaker. It is generally advisable to place mixin classes before the base class):
<pre><code class="python">
# Package imports
from . import model
# uWeb imports
import uweb
from uweb.pagemaker import admin
class Pages(admin.AdminMixin, uweb.PageMaker):
</code></pre>
After this, you will want to tell the admin code where to find your model, this happens inside your PageMaker class:
<pre><code class="python">
ADMIN_MODEL = model
</code></pre>
h2. Router details
To setup a route to the admin pages, you will need to add a specific route to your router.
<pre><code class="python">
ROUTES = (
...
# Admin routes
('/admin(/.*)?', '_Admin'),
...
)
</code></pre>
After this, you can navigate to http://localhost:8082/admin to see the database's contents, manipulate records and view the documentation stored inside your model's methods.
h1. Login and sessions
h2. OpenID
To enable users of your website to log in using OpenID, there are only a few steps that need to be taken:
h3. Add the OpenID mixin class to PageMaker
This is as simple as adding the OpenID mixin class as one of the ancestors of your PageMaker. It is generally advisable to place mixin classes before the base class):
<pre><code class="python">
import uweb
from uweb.pagemaker import login
class Pages(login.OpenIdMixin, uweb.PageMaker):
</code></pre>
h3. Set up routes to the OpenID validator
The following routes (or similar ones) should be added to the [[router]]:
<pre><code class="python">
ROUTES = (
...
('/OpenIDLogin/?(\w+)?', '_OpenIdInitiate')
('/OpenIDValidate', '_OpenIdValidate')
...
)
</code></pre>
The optional capture after @'OpenIDLogin'@ here is to provide the optional provider URL (instead of through the POST field @'openid_provider'@).
h3. Add handlers to PageMaker for success, sucess, failure, etc:
<pre><code class="python">
def OpenIdAuthCancel(self, message):
return 'OpenID Authentication canceled by user: %s' % message
def OpenIdAuthFailure(self, message):
return 'Authentication failed: %s' % message
def OpenIdAuthSuccess(self, auth_dict):
# Authentication succeeded, the auth_dict contains the information we received from the provider
#
# Next: Retrieve user information from database, or create a new user
# Store the user's session (in the database, cookie, or both)
session_id = base64.urlsafe_b64encode(os.urandom(30))
self.req.AddCookie('OpenIDSession', session_id, max_age=86400)
return 'OpenID Authentication successful!'
def OpenIdProviderBadLink(self, message):
return 'Bad OpenID Provider URL: %s' % message
def OpenIdProviderError(self, message):
return 'The OpenID provider did not respond as expected: %r' % message
</code></pre>
h2. Underdark Login Framework
Using the Underdark Login Framework requires steps comparable to using OpenID, but comes with a little more default setup. The system has two modes for logging in, one that is a straightforward plaintext form submit. This is slightly easier to implement, but when used without SSL it is highly vulnerable to man-in-the-middle (MITM) attacks. The second mode is a Javascript enabled mode where the password is hashed (using the SHA-1 algorithm), and to prevent replay attacks (from a MITM), a random 'challenge' is provided for each login attempt, which is also hashed with the result. This prevents a MITM from learning the password value, or using it to log in later (though the plaintext communication remains visible).
h3. Add the ULF mixin class to PageMaker
<pre><code class="python">
import uweb
from uweb.pagemaker import login
class Pages(login.LoginMixin, uweb.PageMaker):
</code></pre>
h3. Set up routes to the OpenID validator
The following routes (or similar ones) should be added to the [[router]]:
<pre><code class="python">
ROUTES = (
...
('/ULF-Challenge', '_ULF_Challenge'),
('/ULF-Login', '_ULF_Verify'),
...
)
</code></pre>
h3. Add handlers to PageMaker for success sucess and failure:
<pre><code class="python">
def _ULF_Failure(self, secure):
reutrn 'ULF authentication failed (secure mode: %d)' % secure
def _ULF_Success(self, secure):
return 'ULF authentication successful! (secure mode: %d)' % secure
</code></pre>
h1. Persistent storage between requests
µWeb allows you to store objects in a process-persistent storage. This means that the storage will be properly persistent and available when µWeb is in [[standalone]] mode. When running on top of [[apache]], this persistence is only as good as the apache process, which is typically a couple hundred to a few thousand requests.
h2. Default users of the persistent storage
By default, the [[TemplateParser]] and the various database connectors are stored in the persistent storage. This has the benefit that pre-parsed templates will not need to be read from disk on subsequent requests. For databases the benefit is that connections need not be made on-the-fly, but can mostly be retrieved from the storage.
h2. Storing persistent values
Storing persistent values is done with the @Set@ method, as follows:
<pre><code class="python">
def _PostInit(self):
if 'connection' not in self.persistent:
self.persistent.Set('connection', self._MakeConnection())
</code></pre>
In the example above, the database connection is only created, and added to the persistent storage, if it's not already present. This way expensive but reusable actions can be optimized by performing them only once (or once every few so many requests, if running on Apache).
h2. Retrieving persistent values
Retrieving stored values works much like this, but uses the @Get@ method:
<pre><code class="python">
def DatabaseAccess(self):
with self.persistent.Get('connection') as cursor:
cursor.Execute('INSERT INTO `message` SET `text` = "success!"')
</code></pre>
This uses the connection we created (or still had) during @_PostInit@, and uses it to update the database.
In case a key has is not present in the persistent storage (because it wasn't set in the process' lifetime or because it was exlicitly dropped), the @Get@ method has an optional second argument, that is returned when the key is not present:
<pre><code class="python">
def FirstVisit(self):
when = self.persistent.Get('first_visit_time', 'just now')
return 'Your first visit was %s.' % when
</code></pre>
This will return the stored date and time when there was a previously recorded visit, or the text _just now_ if there was no previous time logged.
Finally, the persistent storage has a @SetDefault@ method, that acts much like the similarly named dictionary method. It returns the value for the given key, but if it's not present, it will set the key to the provided value, and return it as well. With this, we can improve on our first-visit tracker, and in one call retrieve or store the first time someone visited:
<pre><code class="python">
def FirstVisit(self):
when = self.persistent.SetDefault('first_visit_time', datetime.datetime.now())
return 'Your first visit was %s.' % when
</code></pre>
h2. Deleting persistent values
If for any reason you need to delete a value from the persistent storage, this can be done using the @Del@ method. The given key name is removed from the storage. *N.B.:* If the key was already removed from the storage (this can happen if the delete code runs more than once, or the key was not defined in the process' lifetime), no error is raised. It is assumed that removing the key is the only desired action.
<pre><code class="python">
def DeletePersistentKey(self, key):
self.persistent.Del(key)
</code></pre>
In the PageMaker, there might be database lookups done through the [[Model|data abstraction layer (model)]] and likely output is sent back making use of the [[TemplateParser]].
{{toc}}
h1. A very minimal PageMaker
In the simplest form, a PageMaker for a project subclasses from µWeb's default @PageMaker@ class and provides its own methods to handle requests. The full source for this would look something like this:
<pre><code class="python">
#!/usr/bin/python
"""PageMaker demonstration module"""
# uWeb framework
import uweb
class Minimalist(uweb.PageMaker):
def Index(self):
return 'Welcome to our website, it is still very much under construction.'
def Catchall(self, path):
return 'The requested page %r does not exist yet' % path
</code></pre>
h1. DebuggingPageMaker
Before we do anything else, during development you are _strongly advised_ to use µWeb's @DebuggingPageMaker@. This has a lot of additional features for when something goes wrong on the server side. When the regular PageMaker runs into a server side error, it returns a very plain HTTP 500 response:
<pre>
INTERNAL SERVER ERROR (HTTP 500) DURING PROCESSING OF '/'
</pre>
Where @'/'@ is the path requested by the client. When running @DebuggingPageMaker@ there is a significantly more helpful (for the developer at least) page whenever an internal server error is encountered. It will show a full stack trace, the local variables on each stack level (typically at the point of calling another function), which helps to arrive to the point of failure more quickly.
To use, just subclass your PageMaker from DebuggingPageMaker:
<pre><code class="python">
class Minimalist(uweb.DebuggingPageMaker)
</code></pre>
Example Internal Server Error response "as image":http://bugs.underdark.nl/attachments/download/185/http500_full.png or in the "µWeb demo project":http://info.underdark.nl/haltandcatchfire?debug
In all cases, an internal server error will cause a full stacktrace to be logged in the log file database.
h1. Templateparser
The µWeb *[[TemplateParser]]* is available on the standard PageMaker instance. When using PageMaker, an instantiated TemplateParser instance is available through the @parser@ member of PageMaker. Basic usage looks like this:
<pre><code class="python">
import uweb
import time
class TemplateDemo(uweb.PageMaker):
def VersionPage(self):
return self.parser.Parse(
'version.utp', year=time.strftime('%Y'), version=uweb.__version__)
</code></pre>
The example template for the above file could look something like this:
<pre><code class="html">
<!DOCTYPE html>
<html>
<head>
<title>µWeb version info</title>
</head>
<body>
<p>µWeb version [version] - Copyright 2010-[year] Underdark</p>
</body>
</html>
</code></pre>
And would result in the following output:
<pre><code class="html">
<!DOCTYPE html>
<html>
<head>
<title>µWeb version info</title>
</head>
<body>
<p>µWeb version 0.12 - Copyright 2010-2012 Underdark</p>
</body>
</html>
</code></pre>
Full documentation, with plenty of example template uses can be found on the [[TemplateParser|TemplateParser wiki-entry]].
h2. Template directory configuration
By default, template are loaded from the 'templates' directory that is expected to be on the same path as the pagemaker module. If your pagemaker is located on @/var/www/uweb_project/project.py@, then templates will be automatically loaded from @/var/www/uweb_project/templates/@.
To change the default template loading path, define a new path in the class variable @TEMPLATE_DIR@. This should be a relative path (and defaults to @'templates'@).
h1. Serving static content
Your website most likely has a few static files that need to be served up. If you have a large website you would run many of these from a separate domain (to reduce the amount of overhead from a heavy web server and complex processes), but often there are at least some files that need to be served up from the local disk.
µWeb has built-in facilities to serve static files, which prevent filesystem traversal by those lesser-behaved browsers. A browser requesting http://example.com/../secret_configuration.txt should *not* get the keys to your database server. The static handler has a base (configurable) directory from which all static content is served. For a client it is impossible to 'browse' up from that directory, preventing these leaks of information.
The MIME-Type for the content will be determined using the @mimetypes@ module (available by default in Python), based on the file's extension.
h2. Static handler demonstration
In your router configuration, add a route the directs to the static handler:
<pre><code class="python">
ROUTES = (
...
('/images/(.*)', 'Static'),
...
)
</code></pre>
This will cause the following behaviour:
* A browser requesting http://example.com/images/fish.jpg will be presented with the content from @'static/fish.jpg'@
* A browser requesting http://example.com/images/../../secret.txt will be presented with the content from @'static/secret.txt'@ (if it exists)
h2. Static directory configuration
As can be seen in the previous example, the content is served from the @'static'@ directory (this is not dependent on the selected route. This path is relative to the absolute path of the PageMaker itself. If the PageMaker module exists on @'/var/www/project/controller.py'@ then the default static directory is @'/var/www/project/static/'@.
This default path can be changed by setting the @PUBLIC_DIR@ class variable of PageMaker. The path can be made absolute simply by providing one:
<pre><code class="python">
import uweb
class DifferentPublic(uweb.PageMaker):
PUBLIC_DIR = '/var/www/project/public_http'
</code></pre>
h2. 404 on static content
Whenever a request for static content (through @Static@) cannot be fulfilled, the method @_StaticNotFound@ is called, with the requested relative path as the sole argument. The default response for which is a simple plain text:
<pre><code class="python">
def _StaticNotFound(self, _path):
message = 'This is not the path you\'re looking for. No such file %r' % (
self.req.env['PATH_INFO'])
return response.Response(message, content_type='text/plain', httpcode=404)
</code></pre>
Override this page if you want to provide your user with a more informative or styled response.
h1. Model admin
µWeb comes with a minimal admin interface based on the [[model]]
h2. Add the Admin mixin class to PageMaker
This is as simple as adding the Admin mixin class as one of the ancestors of your PageMaker. It is generally advisable to place mixin classes before the base class):
<pre><code class="python">
# Package imports
from . import model
# uWeb imports
import uweb
from uweb.pagemaker import admin
class Pages(admin.AdminMixin, uweb.PageMaker):
</code></pre>
After this, you will want to tell the admin code where to find your model, this happens inside your PageMaker class:
<pre><code class="python">
ADMIN_MODEL = model
</code></pre>
h2. Router details
To setup a route to the admin pages, you will need to add a specific route to your router.
<pre><code class="python">
ROUTES = (
...
# Admin routes
('/admin(/.*)?', '_Admin'),
...
)
</code></pre>
After this, you can navigate to http://localhost:8082/admin to see the database's contents, manipulate records and view the documentation stored inside your model's methods.
h1. Login and sessions
h2. OpenID
To enable users of your website to log in using OpenID, there are only a few steps that need to be taken:
h3. Add the OpenID mixin class to PageMaker
This is as simple as adding the OpenID mixin class as one of the ancestors of your PageMaker. It is generally advisable to place mixin classes before the base class):
<pre><code class="python">
import uweb
from uweb.pagemaker import login
class Pages(login.OpenIdMixin, uweb.PageMaker):
</code></pre>
h3. Set up routes to the OpenID validator
The following routes (or similar ones) should be added to the [[router]]:
<pre><code class="python">
ROUTES = (
...
('/OpenIDLogin/?(\w+)?', '_OpenIdInitiate')
('/OpenIDValidate', '_OpenIdValidate')
...
)
</code></pre>
The optional capture after @'OpenIDLogin'@ here is to provide the optional provider URL (instead of through the POST field @'openid_provider'@).
h3. Add handlers to PageMaker for success, sucess, failure, etc:
<pre><code class="python">
def OpenIdAuthCancel(self, message):
return 'OpenID Authentication canceled by user: %s' % message
def OpenIdAuthFailure(self, message):
return 'Authentication failed: %s' % message
def OpenIdAuthSuccess(self, auth_dict):
# Authentication succeeded, the auth_dict contains the information we received from the provider
#
# Next: Retrieve user information from database, or create a new user
# Store the user's session (in the database, cookie, or both)
session_id = base64.urlsafe_b64encode(os.urandom(30))
self.req.AddCookie('OpenIDSession', session_id, max_age=86400)
return 'OpenID Authentication successful!'
def OpenIdProviderBadLink(self, message):
return 'Bad OpenID Provider URL: %s' % message
def OpenIdProviderError(self, message):
return 'The OpenID provider did not respond as expected: %r' % message
</code></pre>
h2. Underdark Login Framework
Using the Underdark Login Framework requires steps comparable to using OpenID, but comes with a little more default setup. The system has two modes for logging in, one that is a straightforward plaintext form submit. This is slightly easier to implement, but when used without SSL it is highly vulnerable to man-in-the-middle (MITM) attacks. The second mode is a Javascript enabled mode where the password is hashed (using the SHA-1 algorithm), and to prevent replay attacks (from a MITM), a random 'challenge' is provided for each login attempt, which is also hashed with the result. This prevents a MITM from learning the password value, or using it to log in later (though the plaintext communication remains visible).
h3. Add the ULF mixin class to PageMaker
<pre><code class="python">
import uweb
from uweb.pagemaker import login
class Pages(login.LoginMixin, uweb.PageMaker):
</code></pre>
h3. Set up routes to the OpenID validator
The following routes (or similar ones) should be added to the [[router]]:
<pre><code class="python">
ROUTES = (
...
('/ULF-Challenge', '_ULF_Challenge'),
('/ULF-Login', '_ULF_Verify'),
...
)
</code></pre>
h3. Add handlers to PageMaker for success sucess and failure:
<pre><code class="python">
def _ULF_Failure(self, secure):
reutrn 'ULF authentication failed (secure mode: %d)' % secure
def _ULF_Success(self, secure):
return 'ULF authentication successful! (secure mode: %d)' % secure
</code></pre>
h1. Persistent storage between requests
µWeb allows you to store objects in a process-persistent storage. This means that the storage will be properly persistent and available when µWeb is in [[standalone]] mode. When running on top of [[apache]], this persistence is only as good as the apache process, which is typically a couple hundred to a few thousand requests.
h2. Default users of the persistent storage
By default, the [[TemplateParser]] and the various database connectors are stored in the persistent storage. This has the benefit that pre-parsed templates will not need to be read from disk on subsequent requests. For databases the benefit is that connections need not be made on-the-fly, but can mostly be retrieved from the storage.
h2. Storing persistent values
Storing persistent values is done with the @Set@ method, as follows:
<pre><code class="python">
def _PostInit(self):
if 'connection' not in self.persistent:
self.persistent.Set('connection', self._MakeConnection())
</code></pre>
In the example above, the database connection is only created, and added to the persistent storage, if it's not already present. This way expensive but reusable actions can be optimized by performing them only once (or once every few so many requests, if running on Apache).
h2. Retrieving persistent values
Retrieving stored values works much like this, but uses the @Get@ method:
<pre><code class="python">
def DatabaseAccess(self):
with self.persistent.Get('connection') as cursor:
cursor.Execute('INSERT INTO `message` SET `text` = "success!"')
</code></pre>
This uses the connection we created (or still had) during @_PostInit@, and uses it to update the database.
In case a key has is not present in the persistent storage (because it wasn't set in the process' lifetime or because it was exlicitly dropped), the @Get@ method has an optional second argument, that is returned when the key is not present:
<pre><code class="python">
def FirstVisit(self):
when = self.persistent.Get('first_visit_time', 'just now')
return 'Your first visit was %s.' % when
</code></pre>
This will return the stored date and time when there was a previously recorded visit, or the text _just now_ if there was no previous time logged.
Finally, the persistent storage has a @SetDefault@ method, that acts much like the similarly named dictionary method. It returns the value for the given key, but if it's not present, it will set the key to the provided value, and return it as well. With this, we can improve on our first-visit tracker, and in one call retrieve or store the first time someone visited:
<pre><code class="python">
def FirstVisit(self):
when = self.persistent.SetDefault('first_visit_time', datetime.datetime.now())
return 'Your first visit was %s.' % when
</code></pre>
h2. Deleting persistent values
If for any reason you need to delete a value from the persistent storage, this can be done using the @Del@ method. The given key name is removed from the storage. *N.B.:* If the key was already removed from the storage (this can happen if the delete code runs more than once, or the key was not defined in the process' lifetime), no error is raised. It is assumed that removing the key is the only desired action.
<pre><code class="python">
def DeletePersistentKey(self, key):
self.persistent.Del(key)
</code></pre>