[rabbitmq-discuss] queue limits

Mon Nov 22 07:38:53 GMT 2010

On 22 November 2010 10:33, Matthew Sackman <matthew at rabbitmq.com> wrote:

> On Mon, Nov 22, 2010 at 10:11:11AM +1100, Steven Taylor wrote:
> > Are there any limits to how long or large a queue can get apart from disk
> > space?
>
> Yes. Unless you're using the rabbitmq-toke plugin, there is still a
> (very small) per message RAM cost in the form of an index from message
> guid to file+offset+size. Ultimately, you can end up with all your RAM
> in this index, and thus even though all messages are on disk, you won't
> be able to add more messages to a queue.
>

Fine

> Is there a limit to the message size?

Nope, but it must be able to fit in RAM, and it's probably risky to
> attempt any message greater than one quarter of the amount of RAM
> installed owing to copying of the message etc which is likely to occur.
>

Sounds sensible.

>
> > Is there a limit to the number of queue?
>
> Yes. Each queue in Rabbit is an Erlang process. By default, we start up
> the Erlang VM configured for a million processes, so you should be able
> to get close to that, especially off a single connection and channel. If
> you need to get to these sorts of numbers then you may well need to
> raise the Erlang VM process limit further. This will cost more RAM, but
> will allow you to create more AMQP resources.
>

So far we're talking 15 queus. I think we're okay. ;)

>
> > Rather than one bus/queue with
> > many message types, I'm creating a few queue for each simple units of
> work +
> > having only a few message types per queue.  Does anyone see a problem
> with
> > this approach when using RabbitMQ?
>
> No, but if you need high performance and scalability, you may run into
> issues. You may want to look at the queue expiry stuff as well and
> obviously judicious use of exclusive and auto-delete flags is warranted.
>
>
we're not talking super high performance (yet). I'll need to watch this and
get a feel for it.

> > I'm at that nervous stage of it looks good, but I'm not quite sure yet.
>  I
> > guess I'm asking for experiences without having fully tested.  I'm hoping
> to
> > avoid potential pitfalls and save some time.
>
> If in doubt, test ;) It _will_ fail at some point - and you need to
> discover where that point is and how close it is to your requirements.
>

I have been testing, but sometimes I will need you guys to help me know
what's going on underneath.

thanks,
-Steven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101122/460e3081/attachment.htm>