[rabbitmq-discuss] Is rabbitmq appropriate for messaging to thousands of transient clients?

[Josh Gruenberg]

Hi there!

My team is responsible for updating a large-scale game-server platform
for mobile clients.  We currently see peaks with ~1000 simultaneous
connected clients, with each session often relaying an average 1
message/sec (so, perhaps 2000 msg/sec bidirectionally).  The current
implementation uses a proprietary socket protocol for client/server
communications, but we anticipate growing pains as we move toward our
targeted 10x growth for this platform.  To better distribute the load
and virtualize the cluster, we are considering using rabbitmq for
messaging between the clients and servers.

Is this a good idea?


Here are some of our specific concerns (apologies if any of these have
been addressed in previous posts; I couldn't find much relevant
discussion):

1. The intuitive implementation would involve creating a private queue
for each client to receive messages.  This would imply 10k
client-specific queues and 10k connections.  Any concerns there?

2. Mobile gaming sessions are typically short-lived; current
projections suggest we may sometimes see 10 sessions starting/ending
per second (implying 10 connections and queues being created and
destroyed per sec, with several associated bindings each).  Will this
thrashing be sustainable?

3. Mobile clients are notoriously bad at maintaining persistent
connections, so we see frequent disconnect/reconnect behavior.  (This
is one of the primary reasons we're attracted to MQ in the first
place, BTW: the server can fire and forget!)  I expect this could
result in retransmission of messages that were sent from client to
rabbit just prior to connection-loss.  Will AMQP/rabbit help with
this, or would we need to implement our own deduplication logic?

4. Security is obviously a concern. We've brainstormed the following strategy:
    - provision temporary credentials for each client session to
authorize access to the shared vhost (revoked upon session expiry)
    - create each client queue from the server, sending its name to the client
    - manage all queue bindings from the server in response to client requests
This way, all routing configuration would be handled by the server,
and clients would only know the name of their one queue (and a few
exchanges for messaging specific server-side actors.)  Is this wise?


Do these characteristics spell trouble?  Are there existing real-world
installations that successfully connect large client-bases directly to
rabbitmq?

Thanks in advance for any advice,
-joshng