[rabbitmq-discuss] When rabbitmq is clustered with one other node we see a very slow dequeue of messages

[James]

James <james.eddy at ...> writes:

> We have two setups with slightly different network setups (two pairs of 
Node A and B). One we see this issue on, the other we do not, so this is not 
an issue that can be always reproduced.

I see now why this is not always reproducible. On one of our setups there 
was no queueBind to the exchange on the failed server (Node B).

So this issue becomes always reproducible (in our environment) when it meets 
the following conditions.

(1) NODE_A and NODE_B in a cluster
(2) CLIENT_A0 connects to NODE_A creates topic exchange EXCHANGE_X, and 
publishes 1MSG/sec
(3) CLIENT_A1 connects to NODE_A creates and binds queue to EXCHANGE_X, 
CLIENT_A1 now is dequeueing messages at the rate of ~ 1MSG/sec
(4) CLIENT_B connects to NODE_B binds queue to EXCHANGE_X, CLIENT_B1 also is 
now dequeueing messages at the rate of ~ 1MSG/sec
(5) Pull the cable from NODE_A to NODE_B (anything that does not allow the 
tcp connection drop to be detected, making net_ticktime come into play. 
Making NODE_B not routable from NODE_A will work, or iptables DROP will 
work).

Result: CLIENT_A0 continues to publish at 1MSG/sec, after net_ticktime 
CLIENT_A1 is dequeueing much less then 1MSG/sec and falls behind.


* Note: "connects" is a localhost connection for this test

Thanks,
James