[rabbitmq-discuss] Max messages allowed in a queue in RabbitMQ?

Thu Apr 18 09:40:29 BST 2013

Hi,

On 18 Apr 2013, at 04:30, mag8 wrote:

> Can anyone provide me with answers for my below questions:
> 

Each of these will be limited by some combination of the available memory, disk space and the level of producer/consumer throughput you're able to achieve as they're combined in various ways. There are no system enforced limits. You might like to look at http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2009-February/003042.html and 

> 1) Max queues allowed in RabbitMQ?

Thousands (or even tens of thousands) of queues should be no problem at all, though each object (e.g., queues, exchanges, bindings, etc) will take up some memory and/or disk space. By default, Erlang will enforce a maximum number of concurrent processes (i.e., lightweight threads) at around 32768 IIRC. Each queue is managed by its own process and each connection can result in several more, so if you're planning on having a very large number of active queues in a single node (?) and using them all at the same time, then you may need to tweak the emulator arguments rabbit passes the VM by setting +P <new limit> to a higher limit.

You're also likely to use up many Gb just with the overhead for each queue / connection pretty fast, so you're going to need a pretty meaty server to handle millions of both. Tens of thousands should be no problem at all, providing they fit into RAM.

> 2) Max messages allowed in a queue in RabbitMQ?

There is no limit to the number of messages in a queue (and we frequently see hundreds of thousands and even millions of enqueued messages discussed on this mailing list), although each message will take up some amount of memory (which varies depending on whether the message is persistent or transient, whether or not it has been paged to disk anyway to prevent memory overload, etc) and queue response times may vary depending on size.

> 3) Max size of a message in a queue?
> 

Although there is no max message size enforced by rabbit, messages *must* be able to fit into the available memory. Using 32-bit Erlang, 536870911 bytes is maximum binary size, which with 64-bit it's 2305843009213693951 bytes. This seems unlikely to bite you (no pun intended), and you're likely to bump into other system limits first I would've thought.

Cheers,
Tim