[rabbitmq-discuss] Blocked/blocking connections & Memory fluctuations

[Simon MacMullen]

On 18/12/13 17:11, shridharan muthu wrote:
> Hi there,
>
>     I am noticing couple of issues in our rabbit cluster (of 2 nodes).
>
>  1. I notice a lot of blocked and blocking connections in the management
>     admin UI but "rabbitmqctl list_connections" is not showing any
>     blocked or blocking connections. These connections are living over a
>     day and since we use php in our application layer, none of these
>     connections should not live more than 5 mins. Whats strange is that
>     this symptom is noticed in only one of the nodes of the cluster.

Which version are you running? This sounds like a bug that was fixed in 
3.0.3 where connections and channels might not be cleaned up from the 
management database when their node went down.

>  2. Memory doubles once in a few seconds on same node of the cluster.
>     May be GC is kicking in? Not sure whether it could relate to the
>     blocked/blocking connections. Since this node didn't hit memory
>     watermark level, logs doesn't have any warning/errors.

That sounds like it might be GC, but it's hard to tell.

>      On a related note, I have a question about redistributing queues
> after maintanence. For a cluster with 2 nodes (node 1 & node 2) with
> mirroring, lets say out of 12 queues, 6 lives in node 1 (as master) and
> other 6 lives in node 2.  For maintenance reasons, if I take node 2 down
> and put it back in the cluster, all queues will live node 1 at this
> point. Is there any way to redistribute the queues among these 2 nodes
> in this case to move 6 queues back to node 2?

You could use a higher priority "nodes" policy to move the master to 
node 2, then delete it again (albeit the queue would be unmirrored while 
this was taking place). Oh, but you need to be on at least RabbitMQ 
3.1.x for that to work, 3.0.x does not let a mirroring policy move the 
master.

OTOH if the queues are mirrored to all nodes anyway, why worry? A slave 
takes about as many resources as a master anyway, so all your masters 
being on one node and all the slaves on the other should be no big deal.

Cheers, Simon

-- 
Simon MacMullen
RabbitMQ, Pivotal