osBrain uses by default the
pickle module for serialization when
passing messages between agents. Serialization is, however, configurable.
osBrain can also use
raw means, actually, no serialization (i.e.: the user is
expected to send only raw bytes over those sockets).
Using some serializers such as
cloudpickle can be security risk. Those serializers allow
arbitrary code execution while deserializing data and therefore may wreck
or compromise your system. Only use those when communications between
agents are secured (i.e.: encrypted or in a local area network).
Note that different serializers might have different limitations.
'json' does not support serializing an object of
bytes type, while
'pickle' does support it.
Defining the serializer¶
Specifying the serializer only makes sense in server sockets, since clients will automatically detect and set the type they need in order to communicate accordingly with the server.
There are three ways in which the serializer can be specified:
- Global configuration.
- Specifying it at per-agent configuration.
- Specifying it at per-socket configuration.
By setting the
osbrain.config['SERIALIZER'] configuration variable, we
can change the default serializer. For example:
osbrain.config['SERIALIZER'] = 'json'
It is also possible to change the global default serializer by
OSBRAIN_DEFAULT_SERIALIZER environment variable.
Specifying the serializer at per-agent level will override the global configuration. This can be done as follows:
a1 = run_agent('a1', serializer='json')
Finally, we can specify the serializer at per-socket level. This will override any other configuration (global/per-agent). For example:
a1 = run_agent('a1', serializer='json') # Raw serialization will override json for this socket addr1 = a1.bind('PUB', 'alias1', serializer='raw')
PUBSUB messaging pattern¶
For the PUBSUB pattern, there is a special character (
b'\x80' as of now,
even though it could change at any time) that we use so as to let the agents
know what is the topic and what is the message itself. Note that the special
separator character is only required if there is a topic and the serialization
option is NOT set to
raw (read below for more information).
Considerations when using
raw serialization and PUBSUB pattern¶
Special care must be taken when working with
raw serialization and the
PUBSUB messaging pattern. Under those conditions, we decided to replicate the
raw ZeroMQ PUBSUB communication, in which the topic is sent along with the
message and is the handler the one that must take care of separating the topic
from the message it self.
Note that if we are using other type of serialization, it is safe to assume that what we are receiving only the original message, without any traces of the topic.