tornado.iostream — Convenient wrappers for non-blocking sockets

Utility classes to write to and read from non-blocking files and sockets.

Contents:

• BaseIOStream: Generic interface for reading and writing.
• IOStream: Implementation of BaseIOStream using non-blocking sockets.
• SSLIOStream: SSL-aware version of IOStream.
• PipeIOStream: Pipe-based IOStream implementation.

Base class

class tornado.iostream.BaseIOStream(io_loop=None, max_buffer_size=None, read_chunk_size=4096)

A utility class to write to and read from a non-blocking file or socket.

We support a non-blocking write() and a family of read_*() methods. All of the methods take callbacks (since writing and reading are non-blocking and asynchronous).

When a stream is closed due to an error, the IOStream’s error attribute contains the exception object.

Subclasses must implement fileno, close_fd, write_to_fd, read_from_fd, and optionally get_fd_error.

Main interface

BaseIOStream.write(data, callback=None)

Write the given data to this stream.

If callback is given, we call it when all of the buffered write data has been successfully written to the stream. If there was previously buffered write data and an old write callback, that callback is simply overwritten with this new callback.

BaseIOStream.read_bytes(num_bytes, callback, streaming_callback=None)

Run callback when we read the given number of bytes.

If a streaming_callback is given, it will be called with chunks of data as they become available, and the argument to the final callback will be empty. Otherwise, the callback gets the data as an argument.

BaseIOStream.read_until(delimiter, callback)

Run callback when we read the given delimiter.

The callback will get the data read (including the delimiter) as an argument.

BaseIOStream.read_until_regex(regex, callback)

Run callback when we read the given regex pattern.

The callback will get the data read (including the data that matched the regex and anything that came before it) as an argument.

BaseIOStream.read_until_close(callback, streaming_callback=None)

Reads all data from the socket until it is closed.

If a streaming_callback is given, it will be called with chunks of data as they become available, and the argument to the final callback will be empty. Otherwise, the callback gets the data as an argument.

Subject to max_buffer_size limit from IOStream constructor if a streaming_callback is not used.

BaseIOStream.close(exc_info=False)

Close this stream.

If exc_info is true, set the error attribute to the current exception from sys.exc_info (or if exc_info is a tuple, use that instead of sys.exc_info).

BaseIOStream.set_close_callback(callback)

Call the given callback when the stream is closed.

BaseIOStream.closed()

Returns true if the stream has been closed.

BaseIOStream.reading()

Returns true if we are currently reading from the stream.

BaseIOStream.writing()

Returns true if we are currently writing to the stream.

BaseIOStream.set_nodelay(value)

Sets the no-delay flag for this stream.

By default, data written to TCP streams may be held for a time to make the most efficient use of bandwidth (according to Nagle’s algorithm). The no-delay flag requests that data be written as soon as possible, even if doing so would consume additional bandwidth.

This flag is currently defined only for TCP-based IOStreams.

New in version 3.1.

Methods for subclasses

BaseIOStream.fileno()

Returns the file descriptor for this stream.

BaseIOStream.close_fd()

Closes the file underlying this stream.

close_fd is called by BaseIOStream and should not be called elsewhere; other users should call close instead.

BaseIOStream.write_to_fd(data)

Attempts to write data to the underlying file.

Returns the number of bytes written.

BaseIOStream.read_from_fd()

Attempts to read from the underlying file.

Returns None if there was nothing to read (the socket returned EWOULDBLOCK or equivalent), otherwise returns the data. When possible, should return no more than self.read_chunk_size bytes at a time.

BaseIOStream.get_fd_error()

Returns information about any error on the underlying file.

This method is called after the IOLoop has signaled an error on the file descriptor, and should return an Exception (such as socket.error with additional information, or None if no such information is available.

Implementations

class tornado.iostream.IOStream(socket, *args, **kwargs)

Socket-based IOStream implementation.

This class supports the read and write methods from BaseIOStream plus a connect method.

The socket parameter may either be connected or unconnected. For server operations the socket is the result of calling socket.accept. For client operations the socket is created with socket.socket, and may either be connected before passing it to the IOStream or connected with IOStream.connect.

A very simple (and broken) HTTP client using this class:

import tornado.ioloop
import tornado.iostream
import socket

def send_request():
    stream.write(b"GET / HTTP/1.0\r\nHost: friendfeed.com\r\n\r\n")
    stream.read_until(b"\r\n\r\n", on_headers)

def on_headers(data):
    headers = {}
    for line in data.split(b"\r\n"):
       parts = line.split(b":")
       if len(parts) == 2:
           headers[parts[0].strip()] = parts[1].strip()
    stream.read_bytes(int(headers[b"Content-Length"]), on_body)

def on_body(data):
    print data
    stream.close()
    tornado.ioloop.IOLoop.instance().stop()

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
stream = tornado.iostream.IOStream(s)
stream.connect(("friendfeed.com", 80), send_request)
tornado.ioloop.IOLoop.instance().start()

connect(address, callback=None, server_hostname=None)

Connects the socket to a remote address without blocking.

May only be called if the socket passed to the constructor was not previously connected. The address parameter is in the same format as for socket.connect, i.e. a (host, port) tuple. If callback is specified, it will be called when the connection is completed.

If specified, the server_hostname parameter will be used in SSL connections for certificate validation (if requested in the ssl_options) and SNI (if supported; requires Python 3.2+).

Note that it is safe to call IOStream.write while the connection is pending, in which case the data will be written as soon as the connection is ready. Calling IOStream read methods before the socket is connected works on some platforms but is non-portable.

class tornado.iostream.SSLIOStream(*args, **kwargs)

A utility class to write to and read from a non-blocking SSL socket.

If the socket passed to the constructor is already connected, it should be wrapped with:

ssl.wrap_socket(sock, do_handshake_on_connect=False, **kwargs)

before constructing the SSLIOStream. Unconnected sockets will be wrapped when IOStream.connect is finished.

The ssl_options keyword argument may either be a dictionary of keywords arguments for ssl.wrap_socket, or an ssl.SSLContext object.

class tornado.iostream.PipeIOStream(fd, *args, **kwargs)

Pipe-based IOStream implementation.

The constructor takes an integer file descriptor (such as one returned by os.pipe) rather than an open file object. Pipes are generally one-way, so a PipeIOStream can be used for reading or writing but not both.

Exceptions

exception tornado.iostream.StreamClosedError

Exception raised by IOStream methods when the stream is closed.

Note that the close callback is scheduled to run after other callbacks on the stream (to allow for buffered data to be processed), so you may see this error before you see the close callback.