[rabbitmq-discuss] problem with new persiter with 1000 topics/queues
chen650 at yahoo.com
Sat Aug 14 03:00:21 BST 2010
we are testing the new persiter (built from last week's default branch). when
there are 1000 queues with backed up messages, starting the consumers would
cause some queues to disappear. here are the steps to reproduce the problem:
1) create 1000 queues for 1000 topics.
2) publish messages to these topics. each message size is 20 KB.
3) when total number of queued messages bytes reaches 10GB, start 1000 consumers
with 1000 processes.
The broker would start having problem when several consumers start consuming
messages very fast, which causes other consumers error in login or
basic_consume. then some queues would disappear and the publishers would get
there was no problem if the num of topics/queues is less than 100. our machine
has 8GB mem and 4 disks RAID. we could get around 50MB/sec publish rate and 30
MB/sec consume rate.
Could someone help to look into this problem?
More information about the rabbitmq-discuss