[rabbitmq-discuss] Questions about how best to use RabbitMQ

[Alexis Richardson]

Steve

On Mon, Nov 3, 2008 at 10:12 PM, Ben Hood <0x6e6562 at gmail.com> wrote:
> Steve,
>
> On Mon, Nov 3, 2008 at 7:56 PM, Steve Jenson <stevej at gmail.com> wrote:
>> I see, so we wouldn't be able to take advantage of fanout exchanges?
>> Fanout exchanges seem efficiently implemented such that each queue
>> doesn't keep an individual copy of the message. If we can't use fanout
>> exchanges, that would seem to indiciate increased memory pressure on
>> the broker?
>
> I don't know if follow this. Are you saying that messages routed to
> queues using a fanout exchange are not copied, rather each queue holds
> a reference to each unique message?

If I may intrude...

Steve - I think you are asking if using direct and topic exchanges in
a pubsub scenatio is more expensive from a memory consumption point of
view than using fanout.  IF SO... then the answer is to your question
is 'no'.. at least in the first approximation.

Suppose you have N queues on ONE node of RabbitMQ.  They are all bound
to an exchange X such that a given message M will be routed to ALL the
N queues.  This would happen if:

a) X is a fanout exchange
b) X is a direct exchange and all bindings use the same binding key
c) X is a topic exchange and all bindings use the same binding pattern

Then what happens is that for message bodies ('payloads') over a
certain size (64 bytes) there is only one copy of M's body per node.
In the above case this means that because all N queues are on one node
they (in effect) share access to the body of M.  You do NOT have N
copies of M just because there are N queues.

And - this is independent of exchange type.

Notes:

1. This is the first approximation - in fact there is a small amount
of identifier metadata (headers) copied to each queue.  But if your
scenario were (say) M>1Kb and N<1000 then your memory consumption is
'small'.  (I can make this more precise if you like)
2. The above is true per node.  So if you have K nodes and your N
queues are spread across all K nodes, then you will make K copies of
M.
3. Also, finally, note that for M sized below 64 bytes the message
bodies are copied.

This is all taken care of by the evm.

(BTW - I'd like to thank the engineering team for helping me
understand how all this work in detail :-)

Can you let us know if this has answered your question please?  Sorry
to be verbose - I wanted to get a definitive answer on list for the
record.

Best wishes

alexis