[rabbitmq-discuss] Dynamically binding to a topic exchange without losing messages?

[Paul Jones]

Hi Robert,

In terms of your application, is only one Erlang process every going to
process a given stream? Would it be possible to change the model slightly,
and in a start message send the name of a queue instead? That way your
producer could allocate a queue, connect it to that topic, and send out a
message telling consumers where to listen.

(Some might argue this violates some AMQP design principles though, since it
starts coupling the sender to the receiver more than you'd otherwise hope)

Paul.

On Fri, Oct 2, 2009 at 3:06 PM, Robert Raschke <rtrlists at googlemail.com>wrote:

> Hi,
>
> I have a usage scenario that I am having a bit of trouble translating into
> exchanges, queues and bindings. This is using the Erlang client.
>
> I have a stream of messages that have the identifying characteristics
> <type> and <id>. I would like to process these such that I can recognise a
> particular <type>=START to indicate that all messages related to <id> that
> follow are to be processed in some fashion by a new Erlang process.
>
> So, I declare a topic exchange and bind a queue with the key 'START.*'. I
> now get my "starter" messages, nice. My naive idea was to have this starter
> create a new queue and bind it with key '*.ID' and then have a new Erlang
> process work on the messages arriving there.
>
> What I am trying to get my head around is, how can I quarantee that any
> messages arriving for the new ID don't get lost while my starter process is
> setting stuff up? For example, a message with key START.27 arrives, the
> starter now sets up the new queue and binding against '*.27', but before
> that is complete, a message with key X.27 got published (and potentially
> lost).
>
> One way I could see getting this to work, would be to have my starter
> process read _all_ messages (not just the START ones), synchronously
> starting the sub processes, and passing any non-starter messages to another
> exchange where the sub-processes have bound their queues.
>
> Somehow this "pass it along" approach seems a bit clunky to me. Is there a
> more elegant way? Or, alternatively, does RabbitMQ Erlang Client have some
> nice and easy functions to "consume message from here, look at it, and if
> not for me, publish it to there"? It looks to me as if I need to transmit
> the RoutingKey as part of the message, as it doesn't seem to be part of the
> #content{} structure. But maybe I've missed something.
>
> Thanks for any thoughts,
> Robby
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20091002/20160296/attachment.htm