Performing asynchronous tasks with Plone using message queues.
The speakers had to connect to difference services, some using XML-RPC, others HTTP and yet others via SQL. They needed a reliable, effective and scalable solution. The best solution was using RabbitMQ as an external message broker. RabbitMQ implements the Advanced Message Queue Protocol (AMQP) open standard and is written in Erlang. It is industry tested and reliable.
You can compare AMQP with a mailbox: stuff gets put in and something else picks up the messages some unknown time later.
- A Message consists of label and payload.
- A Producer is a program which sends messages to exchange.
- A Consumer is a program which mostly waits to receive messages.
- An Exchange receives the messages from producers and pushes them to queues.
- A Bindings is how the messages get routed from the exchange to a queue.
- A Queue is a buffer that stores messages.
To make this easier in Plone, you can use collective.zamqp. The package is currently available via a personal GitHub repository, but it will be moved to the Collective in the future. It does not require a lot of code to make use of a message queue.