[rabbitmq-discuss] Broadcasting and STOMP (was Re: python stomp examples)

Wed Sep 10 10:22:12 BST 2008

Hi,

  First I wanted to thank you for this post, it was tremendously
informative for me and pointed me at some other very useful resources.
 I think much of my initial confusion came from a lack of clear
boundaries (in my head) between AMQP and STOMP within rabbitmq.  Also,
lthough this response cleared up a lot, it also prompted some further
questions.
  Again, I've now got a total of roughly 2 days experience with STOMP
and AMQP so I apologize if these questions are a bit on the weak side.

> This is a good question. Here's where STOMP's definition gets weaker,
> and where we start to see AMQP's semantics leaking through.
>
> The way AMQP works is that messages are
>
>  - sent by /producers/ to /exchanges/, and from there are
>  - forwarded to /queues/ along /bindings/, from which they are
>  - distributed round-robin(ish) to /consumers/
>
> As messages pass through an exchange, they are duplicated, and a /copy/
> is sent down each activated binding.
>
> In contrast, when messages leave a queue for a consumer, they are not
> duplicated. One message, sitting on a queue, is delivered to only one of
> the available consumers. Only if that consumer rejects the message
> somehow (e.g. by crashing or otherwise disconnecting before it
> acknowledges the message) will the message be sent on by the queue to
> some other consumer.
Is this round-robin functionality predictable in terms of the order in
which a set of consumers subscribed to a single queue will be
notified?  That is, if I have 5 consumers subscribed to one queue,
(A,B,C,D,E) and my producer sends a message which makes its way to the
queue, how does rabbitmq determine which of those 5 consumers should
receive the message?  If message 234 is sent and A rejects, then B
accepts it, will the next message 235 first be sent to consumer C?  Or
is this just kind of random?

I'm primarily concerned with broadcasting so this isn't a big deal to
me at the moment but I'm curious how things work.

> With the STOMP adapter, a plain old
>
>  SUBSCRIBE
>  destination: myqueue
>
> will create a new consumer. If there are multiple clients, all
> SUBSCRIBEing to the same queue, then there will be multiple consumers
> all on the same queue, leading to round-robin delivery to those clients.
>
> In order to get broadcast behaviour, you need to use a feature
> (originally contributed by Artur, now present in the trunk) which
> exposes a little more of AMQP's semantics through STOMP:
>
>  SUBSCRIBE
>  id: da9d4779-92b9-4cef-86c0-800bdc977f15
>  destination:
>  exchange: amq.topic
>  routing_key: #
>
> The empty-string used for the "destination" header tells the adapter to
> create a private queue for this one subscription, which will last for
> the duration of your connection, or until you UNSUBSCRIBE.
Awesome, this is what I was looking for.

> The "id" header should be some string unique to the current connection
> (!) which names the subscription. I've used a fresh GUID here, but
> anything connection-unique is acceptable.
Would there be any argument against using some version of a GUID-style
browser session cookie here?  This would provide me with a nice
mapping to my web application as well, to help keep things organized.

> The "exchange" header tells the adapter to bind the new private queue to
> the named exchange. All AMQP brokers come with preconfigured
> "amq.direct", "amq.topic" and "amq.fanout" exchanges, with semantics
> described by the AMQP specification (section 3.1.3 of
> http://jira.amqp.org/confluence/download/attachments/720900/amqp0-8.pdf?version=1 ).
Thanks for this link, although maybe I should have looked harder for
the spec myself.  Instead I fruitlessly searched about the rabbitmq
site for 'fanout' and 'topic'.

> The "routing_key" header is used as described in §3.1.3 to select which
> messages are copied into the queue by the exchange.
Based on the AMQP spec it looks like a topic exchange is best suited
to my needs.  This provides the same basic functionality as the
fanout, but also provides added granularity in terms of determining
who gets what in a hierarchical fashion ( I guess it doesn't
necessarily have to be hierarchical though ).

> The STOMP gateway doesn't provide a way of creating a new exchange, yet;
> you have to use a real AMQP client (one that can issue
> "exchange.declare" commands) for that, currently. Suggestions welcome!
Hmm.  From the AMQP spec it looks like I can create a 'durable'
exchange, which means that there shouldn't be any obstacle to creating
a hybrid producer client, right?  I am working primarily with python,
so I'm thinking I can use stomper+py-amqplib or python-qpid
(http://barryp.org/software/py-amqplib/) then just use amqp module to
add the exchange and do all my message passing with STOMP.

> To unsubscribe from a private queue,
>
>  UNSUBSCRIBE
>  id: da9d4779-92b9-4cef-86c0-800bdc977f15
>
> as usual, supplying the same "id" you supplied to the SUBSCRIBE.
>
> The private queue will be deleted when the UNSUBSCRIBE has completed.
>
> I've just committed
> http://hg.rabbitmq.com/rabbitmq-stomp/rev/8972d204473a, which contains
> some (ruby!) examples of topic broadcast, private-queue creation, and
> unsubscription.
Awesome, thanks for all that!

Cheers,
  Joe

>
> Regards,
>  Tony
> --
>  [][][] Tony Garnock-Jones     | Mob: +44 (0)7905 974 211
>   [][] LShift Ltd             | Tel: +44 (0)20 7729 7060
>  []  [] http://www.lshift.net/ | Email: tonyg at lshift.net
>