[rabbitmq-discuss] Testing Shovel on a WAN

MELIQUE Denis (MORPHO) denis.melique at morpho.com
Wed Mar 28 14:47:05 BST 2012

I'am testing shovel with Rabbit 2.7.1.
I set two clusters of 2 hosts RedHat 6, and a shovel configuration to forward incoming messages from one
cluster to the other.
Messages shoveled from one cluster to the other, are travelling through a bridge (Linux machine configured as bridge).
I can change the bridge configuration to simulate a WAN (slow speed, add latency, lose some packets),
I also can cut the WAN by resetting the bridge, wait a while and then reconnect it, to simulate a network outage.

The input throughput is about 4Mbit/s.

In most of cases everything is fine, I just notice two things :

When I slow the WAN to 1 Mbit/s, I see messages accumulating in the shovel queue, which is what as expected.
Then once my throughput go to zero, messages are slowly transferred to the distant cluster, and that's OK as well.

The problem is that messages are not processed in the same order on the distant cluster than on the local.
My feeling is that with 1 Mbit/s some messages fail to be transferred correctly, and are re-sent later.
( I have the same problem when I simulate a very high level of packet loss (30%), with a 10Mbit/s speed)

Is there such a logic in Shovel ?
Is there a way to configured Shovel to guaranty the sequence of messages ?

When a simulate a WAN outage lasting 2 hours, messages staying in the shovel queue have a status,
first set to "unacknowledged".
Then, after about 900 sec,  they all have their status changed to "ready".
When the network recovers, all the pending messages change back to "unacknowledged", before being sent
to the distant cluster.
It does not seems to be a problem for us, just I'd like to understand this behavior, and if it could impact rabbit performances.

Thank you.

" Ce courriel et les documents qui lui sont joints peuvent contenir des informations confidentielles ou ayant un caractère privé. S'ils ne vous sont pas destinés, nous vous signalons qu'il est strictement interdit de les divulguer, de les reproduire ou d'en utiliser de quelque manière que ce soit le contenu. Si ce message vous a été transmis par erreur, merci d'en informer l'expéditeur et de supprimer immédiatement de votre système informatique ce courriel ainsi que tous les documents qui y sont attachés."
" This e-mail and any attached documents may contain confidential or proprietary information. If you are not the intended recipient, you are notified that any dissemination, copying of this e-mail and any attachments thereto or use of their contents by any means whatsoever is strictly prohibited. If you have received this e-mail in error, please advise the sender immediately and delete this e-mail and all attached documents from your computer system."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120328/30769a2b/attachment.htm>

More information about the rabbitmq-discuss mailing list