Transports and protocols (low-level API)¶
Gruvi is built around the transport/protocol abstraction layers that are documented in PEP 3156 and implemented by asyncio.
A transport is a standard interface to a communications channel. Different
types of channels implement different interfaces. Since Gruvi uses libuv /
pyuv, its transports are mostly wrappers around the various
pyuv.Handle
classes. Transport methods are always non-blocking.
A protocol is a callback based interface that is connected to a transport. The transport calls specific callbacks on the protocol when specific events occur. The callbacks are always non-blocking, but a protocol may expose certain protocol operations are part of their API that are switch points.
As a programmer you will use transports and protocols occasionally, but you will mostly use the higher-level interface provided by Gruvi. The exception is when adding support for a new protocol.
Transports¶
The following transport classes are available:
-
exception
TransportError
¶ A transport error.
-
class
BaseTransport
¶ Base class for
pyuv
based transports. There is no public constructor.-
start
(protocol)¶ Bind to protocol and start calling callbacks on it.
-
get_write_buffer_size
()¶ Return the total number of bytes in the write buffer.
-
get_write_buffer_limits
()¶ Return the write buffer limits as a
(low, high)
tuple.
-
set_write_buffer_limits
(high=None, low=None)¶ Set the low and high watermark for the write buffer.
-
resume_reading
()¶ Resume calling callbacks on the protocol.
As a relaxation from the requirements in Python asyncio, this method may be called even if reading was already paused, and also if a close is pending. This simplifies protocol design, especially in combination with SSL where reading may need to be enabled by the transport itself, without knowledge of the protocol, to complete handshakes.
-
pause_reading
()¶ Pause calling callbacks on the protocol.
This method may be called even if reading was already paused or a close is pending. See the note in
resume_reading()
.
-
close
()¶ Close the transport after all oustanding data has been written.
-
abort
()¶ Close the transport immediately.
-
get_extra_info
(name, default=None)¶ Get transport specific data.
The following information is available for all transports:
Name Description 'handle'
The pyuv handle that is being wrapped.
-
-
class
Transport
(handle, server_hostname=None, mode='rw')¶ A connection oriented transport.
The handle argument is the pyuv handle for which to create the transport. It must be a
pyuv.Stream
instance, so either apyuv.TCP
,pyuv.Pipe
or apyuv.TTY
.The server_hostname argument specifies the host name of the remote peer, if available and applicable for the handle.
The mode argument specifies if this is transport is read-only (
'r'
), write-only ('w'
) or read-write ('rw'
).-
get_write_buffer_size
()¶ Return the total number of bytes in the write buffer.
-
resume_reading
()¶ Resume calling callbacks on the protocol.
As a relaxation from the requirements in Python asyncio, this method may be called even if reading was already paused, and also if a close is pending. This simplifies protocol design, especially in combination with SSL where reading may need to be enabled by the transport itself, without knowledge of the protocol, to complete handshakes.
-
pause_reading
()¶ Pause calling callbacks on the protocol.
This method may be called even if reading was already paused or a close is pending. See the note in
resume_reading()
.
-
write
(data, handle=None)¶ Write data to the transport.
-
writelines
(seq)¶ Write all elements from seq to the transport.
-
write_eof
()¶ Shut down the write direction of the transport.
-
can_write_eof
()¶ Whether this transport can close the write direction.
-
get_extra_info
(name, default=None)¶ Get transport specific data.
In addition to the fields from
BaseTransport.get_extra_info()
, the following information is also available:Name Description 'sockname'
The socket name i.e. the result of the getsockname()
system call.'peername'
The peer name i.e. the result of the getpeername()
system call.'winsize'
The terminal window size as a (cols, rows)
tuple. Only available forpyuv.TTY
handles.'unix_creds'
The Unix credentials of the peer as a (pid, uid, gid)
tuple. Only available forpyuv.Pipe
handles on Unix.'server_hostname'
The host name of the remote peer prior to address resolution, if applicable.
-
-
class
DatagramTransport
(handle, mode='rw')¶ A datagram transport.
The handle argument is the pyuv handle for which to create the transport. It must be a
pyuv.UDP
instance.The mode argument specifies if this is transport is read-only (
'r'
), write-only ('w'
) or read-write ('rw'
).-
get_write_buffer_size
()¶ Return the total number of bytes in the write buffer.
-
resume_reading
()¶ Resume calling callbacks on the protocol.
As a relaxation from the requirements in Python asyncio, this method may be called even if reading was already paused, and also if a close is pending. This simplifies protocol design, especially in combination with SSL where reading may need to be enabled by the transport itself, without knowledge of the protocol, to complete handshakes.
-
pause_reading
()¶ Pause calling callbacks on the protocol.
This method may be called even if reading was already paused or a close is pending. See the note in
resume_reading()
.
-
sendto
(data, addr=None)¶ Send a datagram containing data to addr.
The addr argument may be omitted only if the handle was bound to a default remote address.
-
SSL/TLS support¶
SSL and TLS support is available by means of a special SslTransport
transport:
-
class
SslTransport
(handle, context, server_side, server_hostname=None, do_handshake_on_connect=True, close_on_unwrap=True)¶ An SSL/TLS transport.
The handle argument is the pyuv handle on top of which to layer the SSL transport. It must be a
pyuv.Stream
instance, so either apyuv.TCP
,pyuv.Pipe
or apyuv.TTY
.SSL transports are always read-write, so the handle provided needs to support that.
The context argument specifies the
ssl.SSLContext
to use. You can usecreate_default_context()
to create a new context which also works on Python 2.x wheressl.SSLContext
does not exist.The server_side argument specifies whether this is a server side or a client side transport.
The optional server_hostname argument can be used to specify the hostname you are connecting to. You may only specify this parameter if your Python version supports SNI.
The optional do_handshake_on_connect argument specifies whether to to start the SSL handshake immediately. If False, then the connection will be unencrypted until
do_handshake()
is called. The default is to start the handshake immediately.The optional close_on_unwrap argument specifies whether you want the ability to continue using the connection after you call
unwrap()
of when an SSL “close_notify” is received from the remote peer. The default is to close the connection.-
get_extra_info
(name, default=None)¶ Return transport specific data.
The following fields are available, in addition to the information exposed by
Transport.get_extra_info()
.Name Description 'ssl'
The internal ssl.SSLObject
instance used by this transport.'sslctx'
The ssl.SSLContext
instance used to create the SSL object.
-
pause_reading
()¶ Stop reading data.
Flow control for SSL is a little bit more complicated than for a regular
Transport
because SSL handshakes can occur at any time during a connection. These handshakes require reading to be enabled, even if the application calledpause_reading()
before.The approach taken by Gruvi is that when a handshake occurs, reading is always enabled even if
pause_reading()
was called before. I believe this is the best way to prevent complex read side buffering that could also result in read buffers of arbitrary size.The consequence is that if you are implementing your own protocol and you want to support SSL, then your protocol should be able to handle a callback even if it called
pause_reading()
before.
-
resume_reading
()¶ Resume reading data.
See the note in
pause_reading()
for special considerations on flow control with SSL.
-
do_handshake
()¶ Start the SSL handshake.
This method only needs to be called if this transport was created with do_handshake_on_connect set to False (the default is True).
The handshake needs to be synchronized between the both endpoints, so that SSL record level data is not incidentially interpreted as plaintext. Usually this is done by starting the handshake directly after a connection is established, but you can also use an application level protocol.
-
unwrap
()¶ Remove the security layer.
Use this method only if you want to send plaintext data on the connection after the security layer has been removed. In all other cases, use
close()
.If the unwrap is initiated by us, then any data sent after it will be buffered until the corresponding close_notify response is received from our peer.
If the unwrap is initiated by the remote peer, then this method will acknowledge it. You need an application level protocol to determine when to do this because the receipt of a close_notify is not communicated to the application.
-
close
()¶ Cleanly shut down the SSL protocol and close the transport.
-
-
create_default_context
(purpose=None, **kwargs)¶ Create a new SSL context in the most secure way available on the current Python version. See
ssl.create_default_context()
.
Protocols¶
A Protocols is a collection of named callbacks. A protocol is are attached to a transport, and its callbacks are called by the transport when certain events happen.
The following protocol classes are available:
-
exception
ProtocolError
¶ A protocol error.
-
class
BaseProtocol
(timeout=None)¶ Base class for all protocols.
The timeout argument specifies a default timeout for various protocol operations.
-
connection_made
(transport)¶ Called when a connection is made.
-
connection_lost
(exc)¶ Called when a connection is lost.
-
pause_writing
()¶ Called when the write buffer in the transport has exceeded the high water mark. The protocol should stop writing new data.
-
resume_writing
()¶ Called when the write buffer in the transport has fallen below the low water mark. The protocol can start writing data again.
-
-
class
Protocol
(timeout=None)¶ Base class for connection oriented protocols.
The timeout argument specifies a default timeout for various protocol operations.
-
data_received
(data)¶ Called when a new chunk of data is received.
-
eof_received
()¶ Called when an EOF is received.
-
-
class
DatagramProtocol
(timeout=None)¶ Base classs for datagram oriented protocols.
The timeout argument specifies a default timeout for various protocol operations.
-
datagram_received
(data, addr)¶ Called when a new datagram is received.
-
error_received
(exc)¶ Called when an error has occurred.
-
The following class does not exist in PEP 3156 but is a useful base class for most protocols:
-
class
MessageProtocol
(message_handler=None, timeout=None)¶ Base class for message oriented protocols.
Creating transports and protocols¶
Transports and protocols operate in pairs; there is always exactly one protocol for each transport. A new transport/protocol pair can be created using the factory functions below:
-
switchpoint
create_connection
(protocol_factory, address, ssl=False, server_hostname=None, local_address=None, family=0, flags=0, ipc=False, timeout=None, mode='rw')¶ Create a new client connection.
This method creates a new
pyuv.Handle
, connects it to address, and then waits for the connection to be established. When the connection is established, the handle is wrapped in a transport, and a new protocol instance is created by calling protocol_factory. The protocol is then connected to the transport by calling the transport’sstart()
method which in turn callsconnection_made()
on the protocol. Finally the results are returned as a(transport, protocol)
tuple.The address may be either be a string, a (host, port) tuple, a
pyuv.Stream
handle or a file descriptor:- If the address is a string, this method connects to a named pipe using a
pyuv.Pipe
handle. The address specifies the pipe name. - If the address is a tuple, this method connects to a TCP/IP service using
a
pyuv.TCP
handle. The first element of the tuple specifies the IP address or DNS name, and the second element specifies the port number or service name. - If the address is a
pyuv.Stream
instance, it must be an already connected stream. - If the address is a file descriptor, then it is attached to a
pyuv.TTY
stream ifos.isatty()
returns true, or to apyuv.Pipe
instance otherwise.
The ssl parameter indicates whether an SSL/TLS connection is desired. If so then an
ssl.SSLContext
instance is used to wrap the connection using thessl
module’s asynchronous SSL support. The context is created as follows. If ssl is anSSLContext
instance, it is used directly. If it is a function, it is called with the connection handle as an argument and it must return a context . If it isTrue
then a default context is created usinggruvi.create_default_context()
. To disable SSL (the default), passFalse
. If SSL is active, the return transport will be anSslTransport
instance, otherwise it will be aTransport
instance.The server_hostname parameter is only relevant for SSL connections, and specifies the server hostname to use with SNI (Server Name Indication). If no server hostname is provided, the hostname specified in address is used, if available.
The local_address keyword argument is relevant only for TCP transports. If provided, it specifies the local address to bind to.
The family and flags keyword arguments are used to customize address resolution for TCP handles as described in
socket.getaddrinfo()
.The mode parameter specifies if the transport should be put in read-only (
'r'
), write-only ('w'
) or read-write ('rw'
) mode. For TTY transports, the mode must be either read-only or write-only. For all other transport the mode should usually be read-write.- If the address is a string, this method connects to a named pipe using a
-
switchpoint
create_server
(protocol_factory, address=None, ssl=False, family=0, flags=0, ipc=False, backlog=128)¶ Create a new network server.
This creates one or more
pyuv.Handle
instances bound to address, puts them in listen mode and starts accepting new connections. For each accepted connection, a new transport is created which is connected to a new protocol instance obtained by calling protocol_factory.The address argument may be either be a string, a
(host, port)
tuple, or apyuv.Stream
handle:- If the address is a string, this method creates a new
pyuv.Pipe
instance and binds it to address. - If the address is a tuple, this method creates one or more
pyuv.TCP
handles. The first element of the tuple specifies the IP address or DNS name, and the second element specifies the port number or service name. A transport is created for each resolved address. - If the address is a
pyuv.Stream
handle, it must already be bound to an address.
The ssl parameter indicates whether SSL should be used for accepted connections. See
create_connection()
for a description.The family and flags keyword arguments are used to customize address resolution for TCP handles as described in
socket.getaddrinfo()
.The ipc parameter indicates whether this server will accept new connections via file descriptor passing. This works for pyuv.Pipe handles only, and the user is required to call
Server.accept_connection()
whenever a new connection is pending.The backlog parameter specifies the listen backlog i.e the maximum number of not yet accepted active opens to queue. To disable listening for new connections (useful when ipc was set), set the backlog to
None
.The return value is a
Server
instance.- If the address is a string, this method creates a new
Endpoints¶
Endpoints wrap transports and protocols for client and server side connections, and provide a more object-oriented way of working with them.
-
class
Endpoint
(protocol_factory, timeout=None)¶ A communications endpoint.
The protocol_factory argument constructs a new protocol instance. Normally you would pass a
Protocol
subclass.The timeout argument specifies a timeout for various network operations.
-
timeout
¶ The network timeout.
-
close
()¶ Close the endpoint.
-
-
class
Client
(protocol_factory, timeout=None)¶ A client endpoint.
-
transport
¶ Return the transport, or
None
if not connected.
-
protocol
¶ Return the protocol, or
None
if not connected.
-
switchpoint
connect
(address, **kwargs)¶ Connect to address and wait for the connection to be established.
See
create_connection()
for a description of address and the supported keyword arguments.
-
switchpoint
close
()¶ Close the connection.
-
-
class
Server
(protocol_factory, timeout=None)¶ A server endpoint.
-
addresses
¶ The addresses this server is listening on.
-
connections
¶ An iterator yielding the (transport, protocol) pairs for each connection.
-
accept_connection
(handle, ssl=False)¶ Accept a new connection on handle. This method needs to be called when a connection was passed via file descriptor passing.
-
handle_connection
(client, ssl)¶ Handle a new connection with handle client.
This method exists so that it can be overridden in subclass. It is not intended to be called directly.
-
connection_made
(transport, protocol)¶ Called when a new connection is made.
-
connection_lost
(transport, protocol, exc=None)¶ Called when a connection is lost.
-
switchpoint
listen
(address, ssl=False, family=0, flags=0, ipc=False, backlog=128)¶ Create a new transport, bind it to address, and start listening for new connections.
See
create_server()
for a description of address and the supported keyword arguments.
-
switchpoint
close
()¶ Close the listening sockets and all accepted connections.
-
switchpoint
run
()¶ Run the event loop and start serving requests.
This method stops serving when a CTRL-C/SIGINT is received. It is useful for top-level scripts that run only one server instance. In more complicated application you would call
Hub.switch()
directly, or wait on some kind of “request to shutdown” event.
-