[rabbitmq-discuss] Is there a channel publish throughput limit to a queue?
allen.kim at epicadvertising.com
Wed Oct 5 14:38:57 BST 2011
Thanks for your reply. that explains a lot.
On 11-10-05 5:13 AM, "Tim Fox" <tim at rabbitmq.com> wrote:
>On 04/10/11 19:00, Allen Kim wrote:
>> Thanks for your reply. I have been waiting for someone to reply.
>> 1. They are persistent messages (disk writing speed could slow down that
>A good disk can sync at about 200-300 syncs per second. If each
>published message is being synced to disk you should expect that level
>of performance. That's a hardware limit, not RabbitMQ related.
>If you want to go above that and still have your data actually persisted
>when you think it is (i.e. when the confirm is returned), then you
>either need to get a solid state disk, SAN, or disks with non volatile
>Most desktop setups have disk write cache enabled by default, so you can
>get falsely highly rates of several thousand "syncs" per second when
>running on your cheap desktop/laptop only to be surprised when the rate
>goes down when you deploy to your production servers (since prod servers
>generally don't have cache enabled by default).
>> 2. NOT large message (it's around two lines 200-400 bytes)
>> 3. routing to only one queue
>> 4. NO transaction
>> 5. 3-4 consumers with basic.qos prefetch to 100.
>> So, #1 and #5 matters?
>> On 11-10-04 4:37 AM, "Matthias Radestock"<matthias at rabbitmq.com> wrote:
>>> On 29/09/11 21:47, Allen Kim wrote:
>>>> I checked the channel, and found out that the channel was still
>>>> publishing to a queue.
>>> Looks like the queues couldn't keep up with the inbound publishing
>>> so rabbit ended up buffering messages in the channel.
>>>> The speed of publishing was around 700/s to 800/s, which is definitely
>>>> not 2000/s.
>>> ...which is another indication of the above.
>>>> I was wondering if there is limit that affects publishing speed.
>>> Well, obviously rabbit cannot process messages at an infinite rate, so
>>> clearly there has to be a limit ;)
>>> However, 700Hz sounds quite low. On my desktop I can get more than 20
>>> times that. But there are many factors which can reduce that rate:
>>> - persistent messages
>>> - large messages
>>> - routing messages to lots of queues
>>> - using transactions
>>> - having lots of consumers with basic.qos prefetch
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>rabbitmq-discuss mailing list
>rabbitmq-discuss at lists.rabbitmq.com
More information about the rabbitmq-discuss