[rabbitmq-discuss] Federation Issues: Disconnects and Failures on Queue Bind - 2012-01-20

Wed Jan 25 16:56:48 GMT 2012

Hi Simon,

Thanks for the details, I am no expert (greenish would be the word) in 
the internals of RabbitMQ.
I am running if HIPE:
   {rabbit,                    [{vm_memory_high_watermark, 0.6},
                                {collect_statistics_interval, 5000},
                                {hipe_compile, true}
                               ]
   },
   {mnesia,                    [{dc_dump_limit, 40},
                                {dump_log_write_threshold, 50000},
                                {send_compressed, 9},
                                {snmp, true}
                               ]
   },

SERVER_START_ARGS="+K true -smp enable"

I was also surprised on the performance numbers compared to other tests.
I downgraded both clusters to single nodes and still have the same 
percentages.
I'm tempted to start testing the shovel plugin (I need all messages to 
travel between sites),
but I really like the setup of Federation and the fail-over of the 
exchange (haven't tested
that yet, but assume it works. LOL).

Thanks

On 1/25/12 11:11 AM, Simon MacMullen wrote:
> On 24/01/12 18:37, DawgTool wrote:
>> What I was hoping to see that Cluster(002) would be about half the cpu
>> consumption as Cluster(001).
>> Or something like the scenario of publishing to a cluster on host 1 with
>> the queue on host 2.
>> Host 1 would take on the message handling from the publisher and hand it
>> off to host 2 which writes to the queue (spreads the load across the
>> cluster, but adds a little overhead).
>
> Remember that when you publish across federation there is more than 
> one publish involved! The federation plugin:
>
> * Binds a queue to the upstream exchange
> * Consumes messages from this queue
> * Republishes them downstream
>
> and then presumably something consumes them downstream too.
>
> So there is more work going on. I'm surprised it's 4x though, but the 
> federation plugin has not received any performance testing yet.
>
> I'm also surprised both nodes in both clusters were equally busy - or 
> does your test load balance across them?
>
>> Please let me know if I have missed something, or if you can think of
>> any tricks to reduce the cpu usage.
>
> Have you turned on hipe_compile? That should help at least.
>
> Cheers, Simon
>