[rabbitmq-discuss] RabbitMQ as communication server for mobile devices

[Tim Watson]

Hi Michal,

On 11/14/2012 04:49 PM, Michal Levý wrote:
> Hi everyone
>
> I have following scenario:
> 1) Many Android clients (from 1000 to 10 000 clients) - not connected 
> all the time (messaging seems like good fit for this)
> 2) Each sending max 100 messages\day
> 3) Also needs to send some messages back from server but not very 
> frequently (10\day)
>
> I was looking on rabbit tutorials, AMQP spec etc.
> It looks like collecting data from androids is easy task. All will 
> send data into single exchange\queue.
> But what about sending messages to Androids from server?
> 1st i was thinking about having single queue and to subcribe from each 
> device with some kind of "filter" (based on some header field). I know 
> there are message brokers which support this scenario.
> But it seems Rabbit doesn't support this.

It isn't hard to set this up in Rabbit! You will need to think about 
solving the problem in AMQP terms though, which is admittedly a bit 
different from what you'd do with say JMS or whatever.

> Do i really have to create one "sending" queue for each client (using 
> direct exchange with proper "per device" routing key) ?

As a matter of fact, clients never send messages directly to queues, but 
rather send them to exchanges (as you've already noted), after which 
they're routed to zero or more queues based on the bindings you've set 
up (between the exchanges and queues). So for example, you can send a 
message to an exchange from one client and receive it on another a la 
pub/sub by using a topic exchange. You can apply content based routing 
using the headers exchange. If you've got data that needs to go out to 
the devices then you first need to figure out what your filters look 
like. Once these are in place, you can use one of the built in exchange 
types to place the messages into the broker's care. Each device that 
wishes to consume data can either create a temporary queue and bind it 
to the exchange(s) in the right way, or connect to an existing queue 
which is bound to a topic exchange for some set of keys that interest you.

> How much overhead queues in rabbit have ?

You can have lots (i.e., thousands) of queues without suffering too 
much, though there is obviously *some* overhead (memory, etc) and its 
worth planning your topology carefully so as to get the best out of 
Rabbit. We can help with that (here) by pointing you to the right 
resources and documentation, or if you want elaborate on some of the 
finer details of the design we often have lively and interesting 
discussions about how to solve various problems on this mailing list!

Going back to something you mentioned at the beginning of your post:

> It looks like collecting data from androids is easy task. All will 
> send data into single exchange\queue.

This really depends on what the whole messaging topology needs to look 
like. Sometimes you need to propagate messages into various places, and 
building blocks like fan-out and exchange-2-exchange bindings can help 
with this. Other times, it actually makes more sense to send some 
messages to a specific exchange and others to, well, another. As you're 
making design decisions such as these, you'll want to consider various 
factors which include performance and resource consumption on the 
broker, scaling up/out (as you add more clients, potentially cluster the 
broker and so on) and of course the management of all this 
infrastructure, from the perspective of both the client application(s) 
and the administration of the broker itself.

> It can be also management issue and really feels like overkill when 
> number of messages send from server to device is really low....
>

It's actually quite easy to set up a temporary, auto-delete (once the 
client disconnects) queue that is used simply to bind to an exchange 
that you're interested in. The semantics probably need some thought 
however: how are you going to consume messages (e.g., round robin, 
fan-out, etc), how will you know to delete them from the broker (using 
acks, etc) and so on. This is where you need to think carefully about 
your design. Either way, there shouldn't be much management to do if 
you're using temporary queues to make consumption and guarantee that the 
queues are deleted once you're finished with them (i.e., the client goes 
away). If you're using long lived (shared) queues then the design 
complexity pushes back into the exchanges and bindings you configure, 
but that's done just once and then clients need to know where/how to 
connect.

> Any advice will be appriciated.

If you've been through the tutorials, then hopefully it's clear that 
consuming from a queue is relatively simple and that you have a number 
of choices when it comes to 'filtering' what you receive. If you want to 
go into a bit more detail about these 'filters' then we can look at 
which topologies might best support your use cases.

Cheers,

Tim