tornado.gen — Simplify asynchronous code

tornado.gen is a generator-based interface to make it easier to work in an asynchronous environment. Code using the gen module is technically asynchronous, but it is written as a single generator instead of a collection of separate functions.

For example, the following asynchronous handler:

class AsyncHandler(RequestHandler):
    @asynchronous
    def get(self):
        http_client = AsyncHTTPClient()
        http_client.fetch("http://example.com",
                          callback=self.on_fetch)

    def on_fetch(self, response):
        do_something_with_response(response)
        self.render("template.html")

could be written with gen as:

class GenAsyncHandler(RequestHandler):
    @gen.coroutine
    def get(self):
        http_client = AsyncHTTPClient()
        response = yield http_client.fetch("http://example.com")
        do_something_with_response(response)
        self.render("template.html")

Most asynchronous functions in Tornado return a Future; yielding this object returns its result.

For functions that do not return Futures, Task works with any function that takes a callback keyword argument (most Tornado functions can be used in either style, although the Future style is preferred since it is both shorter and provides better exception handling):

@gen.coroutine
def get(self):
    yield gen.Task(AsyncHTTPClient().fetch, "http://example.com")

You can also yield a list of Futures and/or Tasks, which will be started at the same time and run in parallel; a list of results will be returned when they are all finished:

@gen.coroutine
def get(self):
    http_client = AsyncHTTPClient()
    response1, response2 = yield [http_client.fetch(url1),
                                  http_client.fetch(url2)]

For more complicated interfaces, Task can be split into two parts: Callback and Wait:

class GenAsyncHandler2(RequestHandler):
    @asynchronous
    @gen.coroutine
    def get(self):
        http_client = AsyncHTTPClient()
        http_client.fetch("http://example.com",
                          callback=(yield gen.Callback("key"))
        response = yield gen.Wait("key")
        do_something_with_response(response)
        self.render("template.html")

The key argument to Callback and Wait allows for multiple asynchronous operations to be started at different times and proceed in parallel: yield several callbacks with different keys, then wait for them once all the async operations have started.

The result of a Wait or Task yield expression depends on how the callback was run. If it was called with no arguments, the result is None. If it was called with one argument, the result is that argument. If it was called with more than one argument or any keyword arguments, the result is an Arguments object, which is a named tuple (args, kwargs).

Decorators

tornado.gen.coroutine(func)

Decorator for asynchronous generators.

Any generator that yields objects from this module must be wrapped in either this decorator or engine.

Coroutines may “return” by raising the special exception Return(value). In Python 3.3+, it is also possible for the function to simply use the return value statement (prior to Python 3.3 generators were not allowed to also return values). In all versions of Python a coroutine that simply wishes to exit early may use the return statement without a value.

Functions with this decorator return a Future. Additionally, they may be called with a callback keyword argument, which will be invoked with the future’s result when it resolves. If the coroutine fails, the callback will not be run and an exception will be raised into the surrounding StackContext. The callback argument is not visible inside the decorated function; it is handled by the decorator itself.

From the caller’s perspective, @gen.coroutine is similar to the combination of @return_future and @gen.engine.

tornado.gen.engine(func)

Callback-oriented decorator for asynchronous generators.

This is an older interface; for new code that does not need to be compatible with versions of Tornado older than 3.0 the coroutine decorator is recommended instead.

This decorator is similar to coroutine, except it does not return a Future and the callback argument is not treated specially.

In most cases, functions decorated with engine should take a callback argument and invoke it with their result when they are finished. One notable exception is the RequestHandler HTTP verb methods, which use self.finish() in place of a callback argument.

Yield points

Instances of the following classes may be used in yield expressions in the generator. Futures may be yielded as well; their result method will be called automatically when they are ready. Additionally, lists of any combination of these objects may be yielded; the result is a list of the results of each yield point in the same order.

class tornado.gen.Task(func, *args, **kwargs)

Runs a single asynchronous operation.

Takes a function (and optional additional arguments) and runs it with those arguments plus a callback keyword argument. The argument passed to the callback is returned as the result of the yield expression.

A Task is equivalent to a Callback/Wait pair (with a unique key generated automatically):

result = yield gen.Task(func, args)

func(args, callback=(yield gen.Callback(key)))
result = yield gen.Wait(key)

class tornado.gen.Callback(key)

Returns a callable object that will allow a matching Wait to proceed.

The key may be any value suitable for use as a dictionary key, and is used to match Callbacks to their corresponding Waits. The key must be unique among outstanding callbacks within a single run of the generator function, but may be reused across different runs of the same function (so constants generally work fine).

The callback may be called with zero or one arguments; if an argument is given it will be returned by Wait.

class tornado.gen.Wait(key)

Returns the argument passed to the result of a previous Callback.

class tornado.gen.WaitAll(keys)

Returns the results of multiple previous Callbacks.

The argument is a sequence of Callback keys, and the result is a list of results in the same order.

WaitAll is equivalent to yielding a list of Wait objects.

class tornado.gen.YieldPoint

Base class for objects that may be yielded from the generator.

Applications do not normally need to use this class, but it may be subclassed to provide additional yielding behavior.

start(runner)

Called by the runner after the generator has yielded.

No other methods will be called on this object before start.

is_ready()

Called by the runner to determine whether to resume the generator.

Returns a boolean; may be called more than once.

get_result()

Returns the value to use as the result of the yield expression.

This method will only be called once, and only after is_ready has returned true.

Other classes

exception tornado.gen.Return(value=None)

Special exception to return a value from a coroutine.

If this exception is raised, its value argument is used as the result of the coroutine:

@gen.coroutine
def fetch_json(url):
    response = yield AsyncHTTPClient().fetch(url)
    raise gen.Return(json_decode(response.body))

In Python 3.3, this exception is no longer necessary: the return statement can be used directly to return a value (previously yield and return with a value could not be combined in the same function).

By analogy with the return statement, the value argument is optional, but it is never necessary to raise gen.Return(). The return statement can be used with no arguments instead.

class tornado.gen.Arguments

The result of a yield expression whose callback had more than one argument (or keyword arguments).

The Arguments object is a collections.namedtuple and can be used either as a tuple (args, kwargs) or an object with attributes args and kwargs.