[rabbitmq-discuss] Is there a channel publish throughput limit to a queue?

Wed Oct 5 14:38:57 BST 2011

Thanks for your reply. that explains a lot.

Allen Kim

On 11-10-05 5:13 AM, "Tim Fox" <tim at rabbitmq.com> wrote:

>On 04/10/11 19:00, Allen Kim wrote:
>> Thanks for your reply. I have been waiting for someone to reply.
>>
>> 1. They are persistent messages (disk writing speed could slow down that
>> much?)
>
>A good disk can sync at about 200-300 syncs per second. If each
>published message is being synced to disk you should expect that level
>of performance. That's a hardware limit, not RabbitMQ related.
>
>If you want to go above that and still have your data actually persisted
>when you think it is (i.e. when the confirm is returned), then you
>either need to get a solid state disk, SAN, or disks with non volatile
>cache.
>
>Most desktop setups have disk write cache enabled by default, so you can
>get falsely highly rates of several thousand "syncs" per second when
>running on your cheap desktop/laptop only to be surprised when the rate
>goes down when you deploy to your production servers (since prod servers
>generally don't have cache enabled by default).
>
>> 2. NOT large message (it's around two lines 200-400 bytes)
>> 3. routing to only one queue
>> 4. NO transaction
>> 5. 3-4 consumers with basic.qos prefetch to 100.
>>
>> So, #1 and #5 matters?
>>
>> Allen
>>
>>
>>
>> On 11-10-04 4:37 AM, "Matthias Radestock"<matthias at rabbitmq.com>  wrote:
>>
>>> On 29/09/11 21:47, Allen Kim wrote:
>>>> I checked the channel, and found out that the channel was still
>>>> publishing to a queue.
>>> Looks like the queues couldn't keep up with the inbound publishing
>>>rate,
>>> so rabbit ended up buffering messages in the channel.
>>>
>>>> The speed of publishing was around 700/s to 800/s, which is definitely
>>>> not 2000/s.
>>> ...which is another indication of the above.
>>>
>>>> I was wondering if there is limit that affects publishing speed.
>>> Well, obviously rabbit cannot process messages at an infinite rate, so
>>> clearly there has to be a limit ;)
>>>
>>> However, 700Hz sounds quite low. On my desktop I can get more than 20
>>> times that. But there are many factors which can reduce that rate:
>>> - persistent messages
>>> - large messages
>>> - routing messages to lots of queues
>>> - using transactions
>>> - having lots of consumers with basic.qos prefetch
>>> etc.
>>>
>>>
>>> Matthias.
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>_______________________________________________
>rabbitmq-discuss mailing list
>rabbitmq-discuss at lists.rabbitmq.com
>https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss