[rabbitmq-discuss] Dynamically binding to a topic exchange without losing messages?

Fri Oct 2 15:06:30 BST 2009

Hi,

I have a usage scenario that I am having a bit of trouble translating into
exchanges, queues and bindings. This is using the Erlang client.

I have a stream of messages that have the identifying characteristics <type>
and <id>. I would like to process these such that I can recognise a
particular <type>=START to indicate that all messages related to <id> that
follow are to be processed in some fashion by a new Erlang process.

So, I declare a topic exchange and bind a queue with the key 'START.*'. I
now get my "starter" messages, nice. My naive idea was to have this starter
create a new queue and bind it with key '*.ID' and then have a new Erlang
process work on the messages arriving there.

What I am trying to get my head around is, how can I quarantee that any
messages arriving for the new ID don't get lost while my starter process is
setting stuff up? For example, a message with key START.27 arrives, the
starter now sets up the new queue and binding against '*.27', but before
that is complete, a message with key X.27 got published (and potentially
lost).

One way I could see getting this to work, would be to have my starter
process read _all_ messages (not just the START ones), synchronously
starting the sub processes, and passing any non-starter messages to another
exchange where the sub-processes have bound their queues.

Somehow this "pass it along" approach seems a bit clunky to me. Is there a
more elegant way? Or, alternatively, does RabbitMQ Erlang Client have some
nice and easy functions to "consume message from here, look at it, and if
not for me, publish it to there"? It looks to me as if I need to transmit
the RoutingKey as part of the message, as it doesn't seem to be part of the
#content{} structure. But maybe I've missed something.

Thanks for any thoughts,
Robby
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20091002/2ed4ae49/attachment.htm 