PyZMP

docs tests Python ROS Indigo Jade Kinetic ——– ——————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————–

PyZMP is a multiprocess library based on ZeroMQ.

The aim is to make experimenting with multiprocess and distributed architecture more solid and overall easier. If at all possible, the goal is to arrive at a minimal set of concepts, that makes solid and efficient distributed system easy to build.

Distributed systems models, as per wikipedia https://en.wikipedia.org/wiki/Distributed_computing#Models, can be classified as:

• Parallel algorithms in shared-memory model: This seems applicable in distributed software using a consensus algorithm as the shared memory.
• Parallel algorithms in message-passing model: This seems applicable in distributed software relying mostly on dataflow architecture, where the implementor can decide on the network structure
• Distributed algorithms in message-passing model: This seems to be the most widely used currently, (web backend model, relying on services available from multiple places for example)

We will focus on the latter first, while keeping in mind it is likely just a special case of the second (network cannot be controlled, algorithm on each node has to be the same). A good exercise here is how to keep a representation of the distribution coherent on each node, despite potential network partition that cna occur.

Doing an Analysis on existing distributed software architecture is likely a very broad task, but we can focus on just a few here, at least as a first step. These should be enough to implement any of the distributed systems models cited above:

• service / RPC oriented architectures ( usually implemented via https://en.wikipedia.org/wiki/Request%E2%80%93response )
• dataflow architecture ( usually implemented via https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern )

with different views for the user, more or less transparently : - make a request / send a task and (asynchronously or not) wait the response/result - receive dataflow from somewhere and send dataflow to somewhere else

Additionally, an interesting endeavour could be to see how https://en.wikipedia.org/wiki/Control_theory applies to such distributed systems (message passing <=> charge transfer).

Note : This is currently a personal perspective that likely require more thorough analysis, so feel free to send a Pull Request.

Repository structure

This repository has a few main branches:

• master : main branch, python dev workflow, releasing version tags into a pip package.

Apart from these we follow a feature branching workflow

How to use

Install ` pip install pyzmp`

Run self tests ` pyzmp`

How to develop

Clone this repository ` git clone http://github.com/asmodehn/pyzmp`

Create you virtualenv to workon using virtualenvwrapper ` mkvirtualenv pyzmpenv`

Install all dependencies via dev-requirements ` pip install -r dev-requirements.txt`

Run self tests ` pyzmp`

Run all tests (with all possible configurations) with tox ` tox`

Note : Tox envs are recreated every time to ensure consistency. So it s better to develop while in a non-tox-managed venv.

Tutorials and examples

A good example showing use of pyzmp, simple RPC client/server example

tutorials implementing multi node communication (under development)

File truncated at 100 lines see the full file