[rabbitmq-discuss] RabbitMQ as communication server for mobile devices

[Michal Levy]

Hi Tim

On Thu, Nov 15, 2012 at 9:58 AM, Tim Watson <tim at rabbitmq.com> wrote:

> Hi Michal,
>
> On 11/14/2012 04:49 PM, Michal Levı wrote:
>
>> Hi everyone
>>
>> I have following scenario:
>> 1) Many Android clients (from 1000 to 10 000 clients) - not connected all
>> the time (messaging seems like good fit for this)
>> 2) Each sending max 100 messages\day
>> 3) Also needs to send some messages back from server but not very
>> frequently (10\day)
>>
>> I was looking on rabbit tutorials, AMQP spec etc.
>> It looks like collecting data from androids is easy task. All will send
>> data into single exchange\queue.
>> But what about sending messages to Androids from server?
>> 1st i was thinking about having single queue and to subcribe from each
>> device with some kind of "filter" (based on some header field). I know
>> there are message brokers which support this scenario.
>> But it seems Rabbit doesn't support this.
>>
>
> It isn't hard to set this up in Rabbit! You will need to think about
> solving the problem in AMQP terms though, which is admittedly a bit
> different from what you'd do with say JMS or whatever.


Im trying ;) ...but its still new concept for me...


>

 Do i really have to create one "sending" queue for each client (using
>> direct exchange with proper "per device" routing key) ?
>>
>
> As a matter of fact, clients never send messages directly to queues, but
> rather send them to exchanges (as you've already noted), after which
> they're routed to zero or more queues based on the bindings you've set up
> (between the exchanges and queues). So for example, you can send a message
> to an exchange from one client and receive it on another a la pub/sub by
> using a topic exchange. You can apply content based routing using the
> headers exchange. If you've got data that needs to go out to the devices
> then you first need to figure out what your filters look like. Once these
> are in place, you can use one of the built in exchange types to place the
> messages into the broker's care. Each device that wishes to consume data
> can either create a temporary queue and bind it to the exchange(s) in the
> right way, or connect to an existing queue which is bound to a topic
> exchange for some set of keys that interest you.
>

Yep, i know you always send message to exchange and not to queue directly
(and i think its very nice concept).
My question was more about "consuming part" - when i have messages for one
and only one device\client, only way to get this message into device is to
create queue for each of them (and using proper exchange types\binding) -
because there is not something like "subscribe to messages from queue xxxx
with header property DeviceId = 564165434" in Rabbit...

And you answered my question ;) ...i have to have one "incoming" queue for
each device.

Temporal queues created by devices are not option for us. We work with same
device for a long time (say few months) and we need to send messages to
devices even when they are not connected atm..
As i understand how exchanges works, if there is no queue bound to exchange
with correct routing key, messages are just dropped (or can be returned
back to publisher)...



>  How much overhead queues in rabbit have ?
>>
>
> You can have lots (i.e., thousands) of queues without suffering too much,
> though there is obviously *some* overhead (memory, etc) and its worth
> planning your topology carefully so as to get the best out of Rabbit. We
> can help with that (here) by pointing you to the right resources and
> documentation, or if you want elaborate on some of the finer details of the
> design we often have lively and interesting discussions about how to solve
> various problems on this mailing list!
>

I have made some tests and 1 durable queue created on rabbit running on
64bit Debian takes cca 10kb of memory. Thats pretty acceptable imho

"Design" is probably too strong word for what i have now :) ....just
thinking...



> Going back to something you mentioned at the beginning of your post:
>
>  It looks like collecting data from androids is easy task. All will send
>> data into single exchange\queue.
>>
>
> This really depends on what the whole messaging topology needs to look
> like. Sometimes you need to propagate messages into various places, and
> building blocks like fan-out and exchange-2-exchange bindings can help with
> this. Other times, it actually makes more sense to send some messages to a
> specific exchange and others to, well, another. As you're making design
> decisions such as these, you'll want to consider various factors which
> include performance and resource consumption on the broker, scaling up/out
> (as you add more clients, potentially cluster the broker and so on) and of
> course the management of all this infrastructure, from the perspective of
> both the client application(s) and the administration of the broker itself.
>
>
Yep, i'v oversiplified here.
Im planning to have different exchanges\queues for data messages send from
devices independently, separate for responses for server requests and
different for logging etc...
Not set in stone yet but im starting to see the light ;)


>  It can be also management issue and really feels like overkill when
>> number of messages send from server to device is really low....
>>
>
> It's actually quite easy to set up a temporary, auto-delete (once the
> client disconnects) queue that is used simply to bind to an exchange that
> you're interested in. The semantics probably need some thought however: how
> are you going to consume messages (e.g., round robin, fan-out, etc), how
> will you know to delete them from the broker (using acks, etc) and so on.
> This is where you need to think carefully about your design. Either way,
> there shouldn't be much management to do if you're using temporary queues
> to make consumption and guarantee that the queues are deleted once you're
> finished with them (i.e., the client goes away). If you're using long lived
> (shared) queues then the design complexity pushes back into the exchanges
> and bindings you configure, but that's done just once and then clients need
> to know where/how to connect.
>
>  Any advice will be appriciated.
>>
>
> If you've been through the tutorials, then hopefully it's clear that
> consuming from a queue is relatively simple and that you have a number of
> choices when it comes to 'filtering' what you receive. If you want to go
> into a bit more detail about these 'filters' then we can look at which
> topologies might best support your use cases.
>

When i was talking about 'filters' it was really meant as "how to subscribe
to *sigle* *queue *and receive only some messages based on some condition",
not how to distribute messages into different queues based on exchange
type\routing key really :)

Thanks very much for your time Tim!

Cheers

Michal


>
> Cheers,
>
> Tim
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20121115/bbafdd3f/attachment.htm>