[rabbitmq-discuss] Clustering setup - Inputs
simon at rabbitmq.com
Thu Feb 13 18:07:53 GMT 2014
On 13/02/2014 17:07, Arun Rao wrote:
> Thank you so much, Simon, as always!
> I learnt from the documentation that queues are single threaded
> resource. What does this mean to delivering and acknowledging messages?
> If prefetch_count=10.
Queues are single processes, but they don't get involved with any of the
work of communicating AMQP, they just accept and dole out messages to
channel processes (possibly not on the same node) which do that.
> Cross cluster traffic are they asynchronous?
The general pattern is: if something is asynchronous in AMQP, then any
cross-cluster traffic associated with it is also asynchronous. So
publishing a messsage across a cluster is asynchronous, but starting a
consumer up with basic.consume is not.
However, there are a couple of exceptions to that: publishing with
mandatory=true (which is synchronous for [publishing channel] -> [queue]
inside the cluster) and setting basic.qos (which causes message delivery
[queue] -> [consuming channel] to be synchronous inside the cluster).
So both of these can have nasty performance in clusters.
With regard to mandatory publishing, it's just that way for historical
reasons, and it will become async in 3.3.0 anyway.
With regard to delivery when basic.qos is set, this is a painful one. As
I said in the previous mail, turning on basic.qos means that the
prefetch count is associated with the channel and hence could be shared
between multiple queues. That means that each queue needs to
synchronously check with the channel each time it wants to deliver a
message, to see if some other queue has used up the prefetch count. It
can't just keep a counter.
We're exploring ways to improve this situation in 3.3.x; one option
which we already have working is to provide a per-consumer prefetch
option separate from basic.qos; another is to optimise the common case
when basic.qos is used with a single consumer (although the transitions
are a bit painful there).
More information about the rabbitmq-discuss