Transports and protocols (low-level API)

Gruvi is built around the transport/protocol abstraction layers that are documented in PEP 3156 and implemented by asyncio.

A transport is a standard interface to a communications channel. Different types of channels implement different interfaces. Since Gruvi uses libuv / pyuv, its transports are mostly wrappers around the various pyuv.Handle classes. Transport methods are always non-blocking.

A protocol is a callback based interface that is connected to a transport. The transport calls specific callbacks on the protocol when specific events occur. The callbacks are always non-blocking, but a protocol may expose certain protocol operations are part of their API that are switch points.

As a programmer you will use transports and protocols occasionally, but you will mostly use the higher-level interface provided by Gruvi. The exception is when adding support for a new protocol.

Transports

The following transport classes are available:

exception TransportError

A transport error.

class BaseTransport

Base class for pyuv based transports. There is no public constructor.

start(protocol)

Bind to protocol and start calling callbacks on it.

get_write_buffer_size()

Return the total number of bytes in the write buffer.

get_write_buffer_limits()

Return the write buffer limits as a (low, high) tuple.

set_write_buffer_limits(high=None, low=None)

Set the low and high watermark for the write buffer.

resume_reading()

Resume calling callbacks on the protocol.

As a relaxation from the requirements in Python asyncio, this method may be called even if reading was already paused, and also if a close is pending. This simplifies protocol design, especially in combination with SSL where reading may need to be enabled by the transport itself, without knowledge of the protocol, to complete handshakes.

pause_reading()

Pause calling callbacks on the protocol.

This method may be called even if reading was already paused or a close is pending. See the note in resume_reading().

close()

Close the transport after all oustanding data has been written.

abort()

Close the transport immediately.

get_extra_info(name, default=None)

Get transport specific data.

The following information is available for all transports:

Name	Description
'handle'	The pyuv handle that is being wrapped.

class Transport(handle, mode='rw')

A connection oriented transport.

The handle argument is the pyuv handle for which to create the transport. It must be a pyuv.Stream instance, so either a pyuv.TCP, pyuv.Pipe or a pyuv.TTY.

The mode argument specifies if this is transport is read-only ('r'), write-only ('w') or read-write ('rw').

get_write_buffer_size()

Return the total number of bytes in the write buffer.

resume_reading()

Resume calling callbacks on the protocol.

As a relaxation from the requirements in Python asyncio, this method may be called even if reading was already paused, and also if a close is pending. This simplifies protocol design, especially in combination with SSL where reading may need to be enabled by the transport itself, without knowledge of the protocol, to complete handshakes.

pause_reading()

Pause calling callbacks on the protocol.

This method may be called even if reading was already paused or a close is pending. See the note in resume_reading().

write(data)

Write data to the transport.

writelines(seq)

Write all elements from seq to the transport.

write_eof()

Shut down the write direction of the transport.

can_write_eof()

Whether this transport can close the write direction.

get_extra_info(name, default=None)

Get transport specific data.

In addition to the fields from BaseTransport.get_extra_info(), the following information is also available:

Name	Description
'sockname'	The socket name i.e. the result of the getsockname() system call.
'peername'	The peer name i.e. the result of the getpeername() system call.
'winsize'	The terminal window size as a (cols, rows) tuple. Only available for pyuv.TTY handles.
'unix_creds'	The Unix credentials of the peer as a (pid, uid, gid) tuple. Only available for pyuv.Pipe handles on Unix.

class DatagramTransport(handle, mode='rw')

A datagram transport.

The handle argument is the pyuv handle for which to create the transport. It must be a pyuv.UDP instance.

The mode argument specifies if this is transport is read-only ('r'), write-only ('w') or read-write ('rw').

get_write_buffer_size()

Return the total number of bytes in the write buffer.

resume_reading()

Resume calling callbacks on the protocol.

As a relaxation from the requirements in Python asyncio, this method may be called even if reading was already paused, and also if a close is pending. This simplifies protocol design, especially in combination with SSL where reading may need to be enabled by the transport itself, without knowledge of the protocol, to complete handshakes.

pause_reading()

Pause calling callbacks on the protocol.

This method may be called even if reading was already paused or a close is pending. See the note in resume_reading().

sendto(data, addr=None)

Send a datagram containing data to addr.

The addr argument may be omitted only if the handle was bound to a default remote address.

SSL/TLS support

SSL and TLS support is available by means of a special SslTransport transport:

class SslTransport(handle, context, server_side, server_hostname=None, do_handshake_on_connect=True, close_on_unwrap=True)

An SSL/TLS transport.

The handle argument is the pyuv handle on top of which to layer the SSL transport. It must be a pyuv.Stream instance, so either a pyuv.TCP, pyuv.Pipe or a pyuv.TTY.

SSL transports are always read-write, so the handle provided needs to support that.

The context argument specifies the ssl.SSLContext to use. You can use create_ssl_context() to create a new context which also works on Python 2.x where ssl.SSLContext does not exist.

The server_side argument specifies whether this is a server side or a client side transport.

The optional server_hostname argument can be used to specify the hostname you are connecting to. You may only specify this parameter if your Python version supports SNI.

The optional do_handshake_on_connect argument specifies whether to to start the SSL handshake immediately. If False, then the connection will be unencrypted until do_handshake() is called. The default is to start the handshake immediately.

The optional close_on_unwrap argument specifies whether you want the ability to continue using the connection after you call unwrap() of when an SSL “close_notify” is received from the remote peer. The default is to close the connection.

get_extra_info(name, default=None)

Return transport specific data.

The following fields are available, in addition to the information exposed by Transport.get_extra_info().

Name	Description
'ssl'	The internal ssl.SSLObject instance used by this transport.
'sslctx'	The ssl.SSLContext instance used to create the SSL object.

pause_reading()

Stop reading data.

Flow control for SSL is a little bit more complicated than for a regular Transport because SSL handshakes can occur at any time during a connection. These handshakes require reading to be enabled, even if the application called pause_reading() before.

The approach taken by Gruvi is that when a handshake occurs, reading is always enabled even if pause_reading() was called before. I believe this is the best way to prevent complex read side buffering that could also result in read buffers of arbitrary size.

The consequence is that if you are implementing your own protocol and you want to support SSL, then your protocol should be able to handle a callback even if it called pause_reading() before.

resume_reading()

Resume reading data.

See the note in pause_reading() for special considerations on flow control with SSL.

do_handshake()

Start the SSL handshake.

This method only needs to be called if this transport was created with do_handshake_on_connect set to False (the default is True).

The handshake needs to be synchronized between the both endpoints, so that SSL record level data is not incidentially interpreted as plaintext. Usually this is done by starting the handshake directly after a connection is established, but you can also use an application level protocol.

unwrap()

Remove the security layer.

Use this method only if you want to send plaintext data on the connection after the security layer has been removed. In all other cases, use close().

If the unwrap is initiated by us, then any data sent after it will be buffered until the corresponding close_notify response is received from our peer.

If the unwrap is initiated by the remote peer, then this method will acknowledge it. You need an application level protocol to determine when to do this because the receipt of a close_notify is not communicated to the application.

close()

Cleanly shut down the SSL protocol and close the transport.

HAVE_SSL_BACKPORTS

A boolean that indicates whether the certain Python 3.x SSL feature are available on Python 2.x as well. This requires the presence of a C extension module that is compiled when Gruvi is installed. This attribute should be set to True on all platforms expect Windows where the C extension is not normally compiled.

The features that require the SSL backports module are

• Setting or quering the compression of an SSL connection.
• Setting the anonymous Diffie Hellman parameters.
• Querying the SSL channel bindings.

On Python 3.x this is always set to False as the SSL backports are not needed there.

create_ssl_context(**sslargs)

Create a new SSL context in a way that is compatible with different Python versions.

Protocols

A Protocols is a collection of named callbacks. A protocol is are attached to a transport, and its callbacks are called by the transport when certain events happen.

The following protocol classes are available:

exception ProtocolError

A protocol error.

class BaseProtocol(timeout=None)

Base class for all protocols.

The timeout argument specifies a default timeout for various protocol operations.

connection_made(transport)

Called when a connection is made.

connection_lost(exc)

Called when a connection is lost.

pause_writing()

Called when the write buffer in the transport has exceeded the high water mark. The protocol should stop writing new data.

resume_writing()

Called when the write buffer in the transport has fallen below the low water mark. The protocol can start writing data again.

class Protocol(timeout=None)

Base class for connection oriented protocols.

The timeout argument specifies a default timeout for various protocol operations.

data_received(data)

Called when a new chunk of data is received.

eof_received()

Called when an EOF is received.

class DatagramProtocol(timeout=None)

Base classs for datagram oriented protocols.

The timeout argument specifies a default timeout for various protocol operations.

datagram_received(data, addr)

Called when a new datagram is received.

error_received(exc)

Called when an error has occurred.

The following class does not exist in PEP 3156 but is a useful base class for most protocols:

class MessageProtocol(message_handler=None, timeout=None)

Base class for message oriented protocols.

Creating transports and protocols

Transports and protocols operate in pairs; there is always exactly one protocol for each transport. A new transport/protocol pair can be created using the factory functions below:

switchpoint create_connection(protocol_factory, address, ssl=False, ssl_args={}, family=0, flags=0, local_address=None, timeout=None, mode='rw')

Create a new client connection.

This method creates a new pyuv.Handle, connects it to address, and then waits for the connection to be established. When the connection is established, the handle is wrapped in a transport, and a new protocol instance is created by calling protocol_factory. The protocol is then connected to the transport by calling the transport’s start() method which in turn calls connection_made() on the protocol. Finally the results are returned as a (transport, protocol) tuple.

The address may be either be a string, a (host, port) tuple, a pyuv.Stream handle or a file descriptor:

• If the address is a string, this method connects to a named pipe using a pyuv.Pipe handle.
• If the address is a tuple, this method connects to a TCP/IP service using a pyuv.TCP handle. The host element of the tuple the IP address or DNS name, and the port element is the port number or service name. The tuple is always passed to getaddrinfo() for resolution together with the family and flags arguments.
• If the address is a pyuv.Stream instance, it must be an already connected stream.
• If the address is a file descriptor, then it is attached to a pyuv.TTY stream if os.isatty() returns true, or to a pyuv.Pipe instance otherwise.

The ssl parameter indicates whether SSL should be used on top of the stream transport. If an SSL connection is desired, then ssl can be set to True or to an ssl.SSLContext instance. In the former case a default SSL context is created. In the case of Python 2.x the ssl module does not define an SSL context object and you can use create_ssl_context() instead which works across all supported Python versions. The ssl_args argument may be used to pass keyword arguments to SslTransport.

If an SSL connection was selected, the resulting transport will be a SslTransport instance, otherwise it will be a Transport instance.

The mode parameter specifies if the transport should be put in read-only ('r'), write-only ('w') or read-write ('rw') mode. For TTY transports, the mode must be either read-only or write-only. For all other transport the mode should usually be read-write.

The local_address keyword argument is relevant only for TCP transports. If provided, it specifies the local address to bind to.

switchpoint create_server(protocol_factory, address=None, ssl=False, ssl_args={}, family=0, flags=0, backlog=128)

Create a new network server.

This creates one or more pyuv.Handle instances bound to address, puts them in listen mode and starts accepting new connections. For each accepted connection, a new transport is created which is connected to a new protocol instance obtained by calling protocol_factory.

The address argument may be either be a string, a (host, port) tuple, or a pyuv.Stream handle:

• If the address is a string, this method creates a new pyuv.Pipe instance and binds it to address.
• If the address is a tuple, this method creates one or more pyuv.TCP handles. The host element of the tuple the IP address or DNS name, and the port element is the port number or service name. The tuple is passed to getaddrinfo() for resolution together with the family and flags arguments. A transport is created for each resolved address.
• If the address is a pyuv.Stream handle, it must already be bound to an address.

The ssl parameter indicates whether SSL should be used for accepted connections. See create_connection() for a description of the ssl and ssl_args parameters.

The backlog parameter specifies the listen backlog i.e the maximum number of not yet accepted connections to queue.

The return value is a Server instance that can be used to control the listening transports.

Endpoints

Endpoints wrap transports and protocols for client and server side connections, and provide a more object-oriented way of working with them.

class Endpoint(protocol_factory, timeout=None)

A communications endpoint.

The protocol_factory argument constructs a new protocol instance. Normally you would pass a Protocol subclass.

The timeout argument specifies a timeout for various network operations.

timeout

The network timeout.

close()

Close the endpoint.

class Client(protocol_factory, timeout=None)

A client endpoint.

transport

Return the transport, or None if not connected.

protocol

Return the protocol, or None if not connected.

switchpoint connect(address, **kwargs)

Connect to address and wait for the connection to be established.

See create_connection() for a description of address and the supported keyword arguments.

switchpoint close()

Close the connection.

class Server(protocol_factory, timeout=None)

A server endpoint.

addresses

The addresses this server is listening on.

connections

An iterator yielding the (transport, protocol) pairs for each connection.

connection_made(transport, protocol)

Called when a new connection is made.

connection_lost(transport, protocol, exc=None)

Called when a connection is lost.

switchpoint listen(address, ssl=False, ssl_args={}, family=0, flags=0, backlog=128)

Create a new transport, bind it to address, and start listening for new connections.

See create_server() for a description of address and the supported keyword arguments.

switchpoint close()

Close the listening sockets and all accepted connections.

switchpoint run()

Run the event loop and start serving requests.

This method is useful in scripts that run only one server instance. In more complicated applications you normally call Hub.switch() explicitly.