[rabbitmq-discuss] 100,000+ connections

Mon Feb 24 16:50:59 GMT 2014

We're supporting up to a million clients in production using a 2 tier
architecture of rabbitmq plus websockets/sockjs. We think we can continue
to scale out globally a couple more orders of magnitude, even as we
increase message rates substantially.

ml

On Mon, Feb 24, 2014 at 11:41 AM, Jason McIntosh <mcintoshj at gmail.com>wrote:

> Right now, we've scaled up to about 2000 concurrent connections from over
> 100 different machines.  We have on average 5-6 queues for each "app" and
> probably 30 or so connections for each application.  Intermittent
> connectivity is more of an issue for your client rather than the server.
>  Unless you need clustering, in which case your cluster nodes all have to
> be up and connected to each other on a reliable network.  If that's not the
> case, federation/shovel is your friend.
>
> But back to topology - we actually have a local rabbit process on a series
> of "cloud" computers.  Each rabbit server has an applications'
> configuration and then they shovel to an "enterprise" system in a different
> data center.  Rabbit in this case acts as a local cache allowing WAN
> failure.  This is probably overkill in some ways but it makes the system
> really really reliable.  The servers are beefier machines that are
> clustered and if we have to scale connections, we can add more server nodes
> (each enterprise server right now is handling about 1000 clients, and
> that's expected to grow significantly).  We have an F5 LTM in front of the
> enterprise machines to allow load balancing between the nodes.
>
> From what we've seen - this topology looks like it'll scale almost
> indefinitely.  You should be able to scale up to 100k concurrent clients as
> long as you've got the ram/disk space.  And you can always break your stuff
> into smaller pieces as needed.
>
> Jason
>
>
> On Thu, Feb 20, 2014 at 9:51 PM, Chris Eineke <ceineke at gmail.com> wrote:
>
>> Hello everyone,
>>
>> I'm trying to find out what broker setup could support 100,000+
>> concurrent clients with 20-30 queues per client, with numbers steadily
>> increasing over time. We would be sending mostly small messages (max. 4KiB,
>> non-binary). One pitfall is that clients aren't guaranteed to be online
>> 100% of the time, so we have to be able to deal with issues related to
>> intermittent connectivity.
>>
>> So far my idea is to install a RabbitMQ broker on each client and use the
>> federation plugin to connect the client-side brokers to the server-side
>> broker/cluster. Has RabbitMQ been exercised with a large number of
>> federated queues/exchanges? Has anybody reported their configuration
>> settings and performance numbers with related set-ups?
>>
>> Best regards,
>>   Chris
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>
>>
>
>
> --
> Jason McIntosh
> https://github.com/jasonmcintosh/
> 573-424-7612
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140224/029d2642/attachment.html>