[rabbitmq-discuss] Federated exchange slowdown

Simone Busoli simone.busoli at gmail.com
Sun Mar 18 11:42:58 GMT 2012


Following up on this issue, the problem has worsened to a great extent.
Here's a bit more information about the setup.
Two brokers B1 and B2, running version 2.7.1 and Erlang R14B04 on Windows,
about 15000 km away from each other. Both machines running the brokers are
physical machines with several cores, lots of RAM, fast disks, and reported
no problems in disk access, CPU or memory consumption.

We have two federated durable topic exchanges X1 and X2, the upstream
connection to the other broker is configured with an hearbeat of a few
seconds and the prefetch count is 100. X1 is bidirectionally federated with
max_hops = 1 to prevent messages from going in loops (which nonetheless
suffer the additional roundtrip before being discarded, a problem I
reported a while ago), X2 unidirectionally federated from B1 to B2.
Messages are all published on B1, all those published on X1 are persistent,
while those published on X2 aren't. Messages are published on X1 with an
average rate of 3 msg/sec, and on X2 with an average rate of 10 msg/sec.

What we experienced is that messages published on X1 queue up on the
outgoing queue on B1, and even worse on the outgoing queue on B2 (although
we don't care about them as the would be discarded once delivered thanks to
the max_hops setting). Looking at the Web UI it appears that the
acknowledges for those messages come in more slowly than messages are
published, but I really can't understand why this might be the case, given
the very low publish rate. The visible effect is a delay of even several
minutes between when a message is published on B1 and received on B2, but
again, only for the low-rate messages published on X1 (perhaps even worse
for the other direction if the messages were not thrown away by the
federation because of the max_hops setting, as the outgoing queue on B2 has
several thousands of messages versus the few hundreds of that on B1). In
other words, the only difference between the two federated exchanges
appears to be the bidirectionality of the first versus the
unidirectionality of the second and the fact that those published on the
first are persistent.
The logs of the two brokers contain apparently no relevant information, and
the network link between the two has always appeared to be in a good state.

Any help is appreciated

On Sat, Mar 17, 2012 at 05:09, Busoli, Simone <Simone.Busoli at ferrari.com>wrote:

>  Hello, I’ve experienced an unexpected behavior with a federated exchange
> whereby messages would build-up on the upstream queue for no apparent
> reasons.****
>
> The network link seemed ok, as well as the two machines on which the
> brokers are installed.****
>
> I would like to know under which circumstances the federation plugin on
> the downstream would slowdown pulling messages from the upstream.****
>
> ** **
>
> Thanks****
>
>
> _________________________________________________________________________________________________________________
> Questo messaggio e da intendersi esclusivamente ad uso del destinatario e
> puo contenere informazioni che sono di natura privilegiata, confidenziale
> o non divulgabile secondo le leggi vigenti. Se il lettore del presente
> messaggio non e il destinatario designato, o il dipendente/agente
> responsabile
> per la consegna del messaggio al destinatario designato, si informa che
> ogni disseminazione, distribuzione o copiatura di questa comunicazione e
> strettamente proibita anche ai sensi del decreto legislativo 196/03 . Se
> avete ricevuto questo messaggio per errore, vi preghiamo di notificarcelo
> immediatamente a mezzo e-mail di risposta e successivamente di procedere
> alla cancellazione di questa e-mail e relativi allegati dal vostro sistema.
>
> _________________________________________________________________________________________________________________
> This message is intended only for the use of the addressee and may contain
> information that is privileged, confidential and exempt from
> disclosure under applicable law. If the reader of this message is not the
> intended recipient, or the employee or agent responsible for delivering the
> message to the intended recipient, you are hereby notified that any
> dissemination, distribution or copying of this communication is strictly
> prohibited. If you have received this e-mail in error, please notify us
> immediately by return e-mail and delete this e-mail and all attachments
> from
> your system.
>
> _________________________________________________________________________________________________________________
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120318/86227ae9/attachment.htm>


More information about the rabbitmq-discuss mailing list