[rabbitmq-discuss] Scenario ideas

Wed Aug 17 11:13:34 BST 2011

On 17 August 2011 10:39, Michael Bridgen <mikeb at rabbitmq.com> wrote:

> Hi Pete,
>
>
>  I have been playing with rabbitmq and am thinking through some common
>> scenarios I think I will need to deal with, I was wondering if anyone
>> would be able to take the time to read my thoughts and validate them or
>> throw a few more options in/give me some article links?
>>
>> I have a topic exchange and need to attach some consumers to process
>> messages.
>>
>> If I attach a consumer with no queue name, a unique one time queue will
>> be created and I can bind the exchange to this queue using a routing key
>> like 'animal.mammals.#'.
>>
>
> Not quite; you have to create the queue with an empty name, then consume
> from it using the name you get back in queue.declare_ok (and then bind it to
> the exchange).
>
>
>  This is great until
>>
>> - the consumer crashes, meaning the exchange will drop all messages (or
>> they get rejected depending on the publish settings I use)
>> or
>>
>
> Yes, that can happen. Though if the queue isn't exclusive or auto_delete
> (and thereby destroyed when the connection drops or consumer goes away), it
> will happily collect messages until the consumer returns. Of course, you'd
> have to have recorded its name somewhere, or generated the name yourself in
> a way you can repeat.
>
>
>  - I need another consumer to process animals.mammals.#, which I can't do
>> as I am using a one time queue and adding another will mean each message
>> is delivered to 2 queues.
>>
>
> What is the problem with delivering to two queues? If you are using a topic
> exchange, creating an anonymous, exclusive queue for each consumer is
> usually appropriate.
>
>
>  Are the above assumptions correct?
>>
>>
>> So my second option is to use a named queue. I attach a consumer with a
>> queue name and bind the exchange to this queue using a routing key like
>> 'animal.mammals.#'.
>>
>> Now if I need another consumer I simply bring another one up and attach
>> it to the same queue name, and they get the messages in round robin
>> which is great.
>>
>
> It is certainly an established pattern to use a durable, named queue for
> sharing out messages.  Queues do double duty in AMQP -- firstly as "work
> queues" of this kind, secondly as buffers for transmission.
>

That's good to know, after experimenting this is the thinking I have come
round to as it allows the consumers to die if they need to and the queues
will still be written to, using a named queue also enables more than one
consumer to attach to a queue and receive the messages in round robin, which
is a nice quick way to add process capacity if it's ever required.

>
> Take care with the order in which you do things, or at least be aware that
> unless you set a prefetch value, the first consumer will get all the
> messages in the queue at that time.

I noticed that with my experiments, prefetch is now set.

>
>
>  What's not so great is if I then decide to attach a consumer to this
>> queue name (maybe by accident) and bind the exchange to a routing key
>> 'animal.mammals.rabbit'
>>
>> Then the queue gets 2 copies of each message. Is there any way to avoid
>> this other than careful management of bindings? Maybe a way of telling
>> the exchange to only deliver the message once to a given queue?
>>
>
> No, it won't get two copies.  The routing will only ever deliver any given
> message once to a queue.
>
> The combination of wildcard routing and round-robin is unusual. Would you
> go into more detail about what you are trying to achieve?
>

I think I could have confused the issue here somewhat - because of how the
tutorials are laid out I mistakenly thought that the consumer could consume
on a given routing key, rather than the routing key just being a binding
between an exchange and a queue. So I was thinking I was consuming on a
routing key, whereas in reality that's not the case.

The tutorials (and further experimentation) It's been a useful exercise in
finding out just how flexible rabbitmq is, and with that flexibility comes
the potential to really break the queues and bindings (especially if the
application is allowed to blindly create queues and bindings).

I am going to go with an approach that means bindings and queues must be
manually created, and only give the application code permission to write to
the exchanges, and read from the queues.

>
>
> --Michael
> ______________________________**_________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.**rabbitmq.com<rabbitmq-discuss at lists.rabbitmq.com>
> https://lists.rabbitmq.com/**cgi-bin/mailman/listinfo/**rabbitmq-discuss<https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110817/82a0b78c/attachment.htm>