[rabbitmq-discuss] RPC style messaging with a clustered web application

Thu Dec 2 10:13:22 GMT 2010

Hi - Can anybody suggest how the following scenerio could be
accomplished using RabbitMQ?

There are 2 processes that need to communicate with one another. One
is a web application, the other a long running batch-processing
service (windows service). In the web layer a 'session' is created and
as part of that process, several 'request' messages are sent to the
windows service. On subsequent polling requests to the web
application, the user's session is re-constructed and an attempt is
made to update it from response messages returned by the batch-
processing service (usng a correlation ID to pick the relevant
messages from the queue). Currently this is achieved using a pair of
private MSMQ queues, which ties pairs of web apps and processing
services together on a single node (i.e. using sticky users sessions).

What we'd like to achieve is decoupling of these processes into a
cluster of web app nodes and a separate cluster of processing nodes.
The req part seems simple - the messages can be broadcast to a fan-out
(i think?) exchange and any one of the (identical) processing nodes
can pick up the messages.

What we are struggling with is getting the response routed back to the
session correctly. At this end there is no long-running process (i.e.
the session is re-hydrated with each polling request and could be
requested on any web app node).

I think that we probably need a response queue per session created at
session start and some routing rules for the response messages based
on the session ID. However, I'm not clear on how these queues could be
expired once their purpose is complete? or if this is an issue?
(though I assume it is since we have many 100s sessions a minute).
There is no definitive 'end' to the session - currently it's
serialized state just expires after x amount of idle time.