Tools

Here, we discuss the idea of Tools and what Tools CherryPy supplies for you. To read about making your own Tools, see CustomTools.

Using tools

Tools are a great way to package up behavior that happens outside your page handlers. For example, you can add static directory serving with the builtin staticdir tool with just a few lines in your config file:

[/docroot]
tools.staticdir.on: True
tools.staticdir.root: "/path/to/app"
tools.staticdir.dir: 'static'

This turns on the staticdir tool for all URLs that start with "/docroot". You can also enable and configure tools per controller or per handler using _cp_config:

class docroot(object):

    _cp_config = {'tools.staticdir.on': True,
                  'tools.staticdir.root: "/path/to/app",
                  'tools.staticdir.dir': 'static'}

But we can do even better by using the builtin decorator support that all Tools have:

class docroot(object):

    @tools.staticdir(root="/path/to/app", dir='static')
    def page(self):
       ...

...and in this case, we can do even better because tools.staticdir is a 'HandlerTool', and therefore can be used directly as a page handler:

class docroot(object):

    static = tools.staticdir.handler(
                 section='static', root="/path/to/app", dir='static')

Finally, you can use (most) Tools directly, by calling the function they wrap. They expose this via the 'callable' attribute:

def page(self):
    tools.response_headers.callable([('Content-Language', 'fr')])
    return "Bonjour, le Monde!"
page.exposed = True

Because the underlying function is wrapped in a tool, you need to call help(tools.whatevertool.callable) if you want the docstring for it. Using help(tools.whatevertool) will give you help on how to use it as a Tool (for example, as a decorator).

Tools also are also inspectable automatically. They expose their own arguments as attributes:

>>> dir(cherrypy.tools.session_auth)
[..., 'anonymous', 'callable', 'check_username_and_password',
'do_check', 'do_login', 'do_logout', 'handler', 'login_screen',
'on_check', 'on_login', 'on_logout', 'run', 'session_key']

This makes IDE calltips especially useful, even when writing config files!

Builtin tools

tools.accept

A tool for verifying that the client is willing to accept the Content-Type of the response.

tools.accept.media, if provided, should be the Content-Type value (as a string) or values (as a list or tuple of strings) which the current request can emit. The client's acceptable media ranges (as declared in the Accept request header) will be matched in order to these Content-Type values; the first such string is returned. That is, the return value will always be one of the strings provided in the 'media' arg (or None if 'media' is None).

The return value doesn't mean anything when used as a Tool, but you can call tools.accept.callable(media) directly to dispatch based on the client's preferred Content-Type:

def select(self):
    mtype = tools.accept.callable(['text/html', 'text/plain'])
    if mtype == 'text/html':
        return "<h2>Page Title</h2>"
    else:
        return "PAGE TITLE"
select.exposed = True

Regardless of whether you call it directly or just turn on the Tool, if no match is found, then HTTPError 406 (Not Acceptable) is raised. Note that most web browsers send */* as a (low-quality) acceptable media range, which should match any Content-Type. In addition, "...if no Accept header field is present, then it is assumed that the client accepts all media types."

tools.auth_basic

New in 3.2

auth_basic is a tool which hooks at before_handler to perform the server-side of HTTP Basic Access Authentication, as specified in RFC 2617.

auth_basic supersedes and deprecates the basic_auth Tool. auth_basic has an API which is simpler and more flexible than basic_auth.

If the request has an 'authorization' header with a 'Basic' scheme, this tool attempts to authenticate the credentials supplied in that header. If the request has no 'authorization' header, or if it does but the scheme is not 'Basic', or if authentication fails, the tool sends a 401 response with a 'WWW-Authenticate' Basic header.

If authentication is successful, cherrypy.request.login will contain the user name.

Arguments:

• realm: a string containing the authentication realm.

• checkpassword: a callable which checks the authentication credentials. Its signature is checkpassword(realm, username, password), where username and password are the values obtained from the request's 'authorization' header. If authentication succeeds, checkpassword returns True, else it returns False.

Example usage, using the built-in checkpassword_dict function which uses a dict as the credentials store:

import cherrypy.lib.auth_basic

userpassdict = {'bird' : 'bebop', 'ornette' : 'wayout'}
checkpassword = cherrypy.lib.auth_basic.checkpassword_dict(userpassdict)
basic_auth = {'tools.auth_basic.on': True,
              'tools.auth_basic.realm': 'earth',
              'tools.auth_basic.checkpassword': checkpassword,
}
app_config = { '/' : basic_auth }

tools.auth_digest

New in 3.2

auth_digest is a tool which hooks at before_handler to perform HTTP Digest Access Authentication, as specified in RFC 2617.

If the request has an 'authorization' header with a 'Digest' scheme, this tool authenticates the credentials supplied in that header. If the request has no 'authorization' header, or if it does but the scheme is not "Digest", or if authentication fails, the tool sends a 401 response with a 'WWW-Authenticate' Digest header.

If authentication is successful, cherrypy.request.login will contain the user name.

auth_digest supersedes and deprecates the digest_auth Tool. The API of auth_digest allows greater choice for the kind of credentials store, and supports keeping only HA1 hashes in the store instead of plaintext passwords.

Arguments:

• realm: a string containing the authentication realm.

• get_ha1: a callable which looks up a username in a credentials store and returns the HA1 string, which is defined in the RFC to be MD5(username : realm : password). The callable's signature is: get_ha1(realm, username), where username is obtained from the request's 'authorization' header. If username is not found in the credentials store, get_ha1() returns None.

• key: a secret string known only to the server, used in the synthesis of nonces.

The auth_digest module provides three built-in helper functions that make it easy to use any of three different kinds of credential stores:

• get_ha1_dict_plain(): Returns a get_ha1 function which obtains a plaintext password from a dictionary of the form: {username : password}

If you want a simple dictionary-based authentication scheme, with plaintext passwords, use cherrypy.lib.auth_digest.get_ha1_dict_plain(my_userpass_dict) as the value for tools.digest_auth.get_ha1.

• get_ha1_dict(): Returns a get_ha1 function which obtains a HA1 password hash from a dictionary of the form: {username : HA1}.

If you want a dictionary-based authentication scheme, but with pre-computed HA1 hashes instead of plain-text passwords, use cherrypy.lib.auth_digest.get_ha1_dict(my_userha1_dict) as the value for tools.digest_auth.get_ha1.

• get_ha1_file_htdigest(): Returns a get_ha1 function which obtains a HA1 password hash from a flat file with lines of the same format as that produced by the Apache htdigest utility.

If you want to use an Apache htdigest file as the credentials store, use cherrypy.lib.auth_digest.get_ha1_file_htdigest(my_htdigest_file) as the value for tools.digest_auth.get_ha1.

It is recommended that the filename argument be an absolute path, to avoid problems.

If you want to use some other form of credential store, e.g. an SQLite database, you only need write a suitable get_ha1() function for it.

Example usage, using the built-in get_ha1_dict_plain function which uses a dict of plaintext passwords as the credentials store:

import cherrypy.lib.auth_digest

userpassdict = {'alice' : '4x5istwelve'}
get_ha1 = cherrypy.lib.auth_digest.get_ha1_dict_plain(userpassdict)
digest_auth = {'tools.auth_digest.on': True,
               'tools.auth_digest.realm': 'wonderland',
               'tools.auth_digest.get_ha1': get_ha1,
               'tools.auth_digest.key': 'a565c27146791cfb',
}
app_config = { '/' : digest_auth }

Example usage, using the built-in get_ha1_file_htdigest function which uses an Apache-style htdigest file as the credentials store:

import cherrypy.lib.auth_digest

get_ha1 = cherrypy.lib.auth_digest.get_ha1_file_htdigest('/path/to/my_htdigest_file')
digest_auth = {'tools.auth_digest.on': True,
               'tools.auth_digest.realm': 'wonderland',
               'tools.auth_digest.get_ha1': get_ha1,
               'tools.auth_digest.key': 'a565c27146791cfb',
}
app_config = { '/' : digest_auth }

Important note: auth_digest hooks at before_handler rather than on_start_resource because auth_digest may need the entity body in order to calculate the digest. At on_start_resource the request body has not been read yet. The entity body is needed if the client has specified qop="auth-int" in the Authorization request header.

tools.basic_auth

Deprecated: The basic_auth tool has been deprecated in CherryPy 3.2, superseded by the auth_basic tool which has a better API. basic_auth will be removed in CherryPy 3.3.

A tool for doing basic authentication. It takes a "realm" setting (a string) and a "users" dict of {username: password} pairs (or a callable which returns that dict). If authentication is successful, cherrypy.request.login will contain the user name. If authentication fails, 401 Unauthorized is raised, the 'WWW-Authenticate' response header is set, and cherrypy.request.login is set to False.

tools.caching

This tool tries to be an (in-process) HTTP/1.1-compliant cache. It's not quite there yet, but it's probably good enough for most sites.

In general, GET responses are cached (along with selecting headers) and, if another request arrives for the same resource, the caching Tool will return 304 Not Modified if possible, or serve the cached response otherwise. It also sets request.cached to True if serving a cached representation, and sets request.cacheable to False (so it doesn't get cached again).

If POST, PUT, or DELETE requests are made for a cached resource, they invalidate (delete) any cached response.

For more information see Caching.

tools.decode

Use this tool to decode cherrypy.request.params (GET and POST query arguments) from on-the-wire strings to Unicode. If you think you know exactly what encoding the client used, and want to be strict about it, set tools.decode.encoding; otherwise, set tools.decode.default_encoding as needed (it defaults to UTF-8). Note that, if the encodings you supply fail, the tool will fall back to decoding from ISO-8859-1 (as the HTTP spec requires).

tools.digest_auth

Deprecated: The digest_auth tool has been deprecated in CherryPy 3.2, superseded by the auth_digest tool which supports various kinds of credentials stores, and allows for keeping HA1 hashes instead of plaintext passwords in the store. digest_auth will be removed in CherryPy 3.3.

A tool for doing Digest authentication (RFC 2617). It takes a "realm" setting (a string) and a "users" dict of {username: password} pairs (or a callable which returns that dict). If authentication is successful, cherrypy.request.login will contain the user name. If authentication fails, 401 Unauthorized is raised, the 'WWW-Authenticate' response header is set, and cherrypy.request.login is set to False.

tools.encode

Encode the outgoing response body, from Unicode to an encoded string.

• tools.encode.encoding: If specified, the tool will error if the response cannot be encoded with it. Otherwise, the tool will use the 'Accept-Charset' request header to attempt to provide suitable encodings, usually attempting utf-8 if the client doesn't specify a charset, but following RFC 2616 and trying ISO-8859-1 if the client sent an empty 'Accept-Charset' header.
• tools.encode.text_only: New in 3.1. If True (the default, and what always happened in earlier versions), responses with a Content-Type starting with 'text/' would be encoded, and all other Content-Types skipped. If False, the Content-Type response header is not examined; all documents will be encoded.
• tools.encode.add_charset: New in 3.1. If True (the default, and what always happened in earlier versions), encoded responses would have their Content-Types altered by setting the "charset" parameter. If False, the Content-Type response header is not altered.

tools.err_redirect

Turn this tool on to redirect all unhandled errors to a different page. Supply the new URL via tools.err_redirect.url. By default, this raises InternalRedirect?. To use HTTPRedirect, set tools.err_redirect.internal to False.

tools.etags

This tool validates the current ETag response header against If-Match and If-None-Match headers, and raises "304 Not Modified" or "412 Precondition Failed" as needed. If tools.etags.autotags is True, an ETag response-header value will be provided from an MD5 hash of the response body (unless some other code has already provided an ETag header). If False (the default), the ETag will not be automatic--your code must provide it before this tool is called (in the before_finalize phase).

tools.expires

A tool for influencing cache mechanisms using the 'Expires' header.

tools.expires.secs must be either an int or a datetime.timedelta, and indicates the number of seconds between response.time and when the response should expire. The 'Expires' header will be set to (response.time + secs). If zero (the default), the following "cache prevention" headers are also set:

'Pragma': 'no-cache'
'Cache-Control': 'no-cache'

If tools.expires.force is False (the default), the following headers are checked: 'Etag', 'Last-Modified', 'Age', 'Expires'. If any are already present, none of the above response headers are set.

tools.flatten

Wraps response.body in a generator that recursively iterates over body. This allows cherrypy.response.body to consist of 'nested generators'; that is, a set of generators that yield generators.

tools.gzip

This tool gzips (compresses) the response body if possible, and sets the 'Content-Encoding' and 'Vary' headers appropriately. The client must send an 'Accept-Encoding' request header that prefers 'gzip' or 'x-gzip'. If a suitable encoding cannot be produced, then 406 Not Acceptable is raised.

tools.ignore_headers

Deletes request headers whose field names are included in tools.ignore_headers.headers. This is a useful tool for working behind certain HTTP servers and proxies; for example, Apache duplicates the work that CP does for 'Range' headers, and will doubly-truncate the response.

tools.log_headers

When enabled, the headers of every request will be sent to CherryPy's log when an unanticipated error occurs.

tools.log_tracebacks

When turned on, all unhandled errors will have their tracebacks sent to CherryPy's log.

tools.nsgmls

A tool for validating the response according to NSGMLS. If errors are encountered, the response body is replaced with the error output.

tools.proxy

Example:

tools.proxy(base=None, local='X-Forwarded-Host', 
            remote='X-Forwarded-For', scheme='X-Forwarded-Proto')

This changes the base URL (scheme://host[:port][/path]), and is most useful when running a CP server behind Apache or some other webserver. For example, Apache may receive a Request-URI of http://www.example.com/app/page, but rewrites it to http://localhost:8080/page before forwarding the request to CherryPy. This tool tells CherryPy how to get the original URI.

tools.proxy.local defines the request header which will be used to auto-fill the new request.base. If you want the new request.base to include path info (not just the host), you must explicitly set base to the full base path, and ALSO set tools.proxy.local to "" (empty string), so that the X-Forwarded-Host request header (which never includes path info) does not override it.

New in CP 3: cherrypy.request.remote.ip (the IP address of the client) will be rewritten if the header specified by tools.proxy.remote is valid. By default, 'remote' is set to 'X-Forwarded-For'. If you do not want to rewrite remote.ip, set the 'remote' arg to an empty string.

Example config for a mod_python site running at http://my.vhost/apps/app/:

[/]
tools.proxy.on: True
tools.proxy.base: "http://my.vhost/apps/app/"

tools.redirect

This one is so simple we might as well show the whole function instead of just the signature:

def redirect(url='', internal=True):
    """Raise InternalRedirect or HTTPRedirect to the given url."""
    if internal:
        raise cherrypy.InternalRedirect(url)
    else:
        raise cherrypy.HTTPRedirect(url)

Raises InternalRedirect? or HTTPRedirect.

• url: the new URL to which to redirect.
• internal: True (the default) to perform an internal redirect; CherryPy loops and starts a new request completely within the same thread/process. False to perform an external redirect; the client (browser) will be told to fetch the new URL instead. If the new URL is located on a different host or domain, an external redirect must be used. See the notes about redirecting POST if you're doing that.

tools.referer

def referer(pattern, accept=True, accept_missing=False, error=403, message=):

Raises HTTPError if the client's Referer header does not pass our test.

• pattern: a regular expression pattern to test against the Referer.
• accept: if True (the default), the Referer must match the pattern; if False, the Referer must NOT match the pattern.
• accept_missing: if True, permit requests with no Referer header. Defaults to False.
• error: the HTTP error code to return to the client on failure. Defaults to 403 Forbidden.
• message: a string to include in the response body on failure. Defaults to 'Forbidden Referer header.'

tools.response_headers

Use this tool to set static response headers. Here's the complete source code for the Tool, to show you again how easy it can be to make your own tools:

def response_headers(headers=None):
    """Set headers on the response."""
    for name, value in (headers or []):
        cherrypy.response.headers[name] = value
cherrypy.tools.response_headers = cherrypy.Tool('on_start_resource', response_headers)

tools.session_auth

This tool checks for the presence of a "username" key in the session. If not found, the request is redirected to a login screen.

Depending on the actual URI, the tool may show other pages:

• If the path ends with "login_screen", an HTML login form is shown.
• If the path ends with "do_login?username=foo&password=bar&from_page=/path/to/page", then the username/password pair is checked. If they pass, the user is redirected to the 'from_page' URI. Otherwise, the HTML login form is returned.
• If the path ends with "do_logout?from_page=/path/to/page", and the request.method is POST, then the user is logged out, and redirected to the 'from_age' URI.

Optional config entries:

• anonymous(): a function (default: pass) which returns an anonymous username. Called when the session key does not map to a known user. Called by do_check.
• check_username_and_password(username, password): a function which returns a str error message if the pair does not pass; otherwise, return False. Called by do_login. Defaults to "pass" (a no-op). You must write a check_username_and_password function for this tool to actually do anything.
• debug: default False. If True, spit out helpful messages in the log.
• login_screen(from_page='..', username='', error_msg=''): a function which returns the HTML login form. The arguments are expected to be interpolated into the HTML.
• on_check(username): a function to perform extra actions when a (username, password) pair is checked. Called at the end of do_check. Default: pass.
• on_login(username): a function to perform extra actions when a user is logged in. Called at the end of do_login. Default: pass.
• on_logout(username): a function to perform extra actions when a user is logged out. Called at the end of do_logout. Default: pass.
• session_key: The name of the key in the session to use for looking up users. Defaults to "username".

tools.staticdir

Serves static resources from the given (.root +) tools.staticdir.dir. See StaticContent.

tools.staticfile

Serves a static resource from the given (.root +) tools.staticfile.filename. See StaticContent.

tools.sessions

See CherryPySessions.

tools.tidy

A tool for validating the response according to HTML Tidy. If errors are encountered, the response body is replaced with the error output.

If either tools.tidy.indent or .wrap are specified, then response.body will be set to the output of tidy. If .warnings is True (the default), the response body will contain them. Otherwise, only errors will change the body. Note that we use the standalone Tidy tool rather than the python mxTidy module. This is because this module does not seem to be stable and it crashes on some HTML pages (which means that the server would also crash). You must set tools.tidy.tidy_path to the location of the Tidy executable on your system. You must also provide a value for temp_dir, the location in which to write the temporary files for shuttling output to the Tidy executable.

tools.trailing_slash

A tool that lets you control whether URL's with a missing or extra trailing slash should raise HTTPRedirect. It's on by default, with these settings:

tools.trailing_slash.on = True
tools.trailing_slash.missing = True
tools.trailing_slash.extra = False

That is, if a trailing slash is missing for an index handler, HTTPRedirect is raised. But if a non-index handler has an extra slash, it's not redirected by default.

tools.xmlrpc

Don't use this directly. Instead, it's used by the XMLRPCController. To use it, have your controllers subclass cherrypy._cptools.XMLRPCController. You can also supply the following optional config entries:

• tools.xmlrpc.encoding: 'utf-8'
• tools.xmlrpc.allow_none: 0

XML-RPC is a rather discontinuous layer over HTTP; dispatching to the appropriate handler must first be performed according to the URL, and then a second dispatch step must take place according to the RPC method specified in the request body. It also allows a superfluous "/RPC2" prefix in the URL, supplies its own handler args in the body, and requires a 200 OK "Fault" response instead of 404 when the desired method is not found. Therefore, XML-RPC cannot be implemented for CherryPy via a Tool alone. The Controller acts as the dispatch target for the first half (based on the URL); it then reads the RPC method from the request body and does its own second dispatch step based on that method. It also reads body params, and returns a Fault on error. The XMLRPCDispatcher strips any /RPC2 prefix; if you aren't using /RPC2 in your URL's, you can safely skip turning on the XMLRPCDispatcher. Otherwise, you need to use declare it in config:

[global]
request.dispatch: cherrypy.dispatch.XMLRPCDispatcher()