[rabbitmq-discuss] Testing Shovel on a WAN

Wed Mar 28 16:57:11 BST 2012

On 28/03/12 14:47, MELIQUE Denis (MORPHO) wrote:
> When I slow the WAN to 1 Mbit/s, I see messages accumulating in the
> shovel queue, which is what as expected.
>
> Then once my throughput go to zero, messages are slowly transferred to
> the distant cluster, and that’s OK as well.
>
> The problem is that messages are not processed in the same order on the
> distant cluster than on the local.
>
> My feeling is that with 1 Mbit/s some messages fail to be transferred
> correctly, and are re-sent later.
>
> ( I have the same problem when I simulate a very high level of packet
> loss (30%), with a 10Mbit/s speed)
>
> Is there such a logic in Shovel ?

That seems unlikely. Packet loss should lead to redelivery at the TCP 
level, which the shovel should be unaware of.

Messages should only get redelivered if the consumer rejects them or the 
TCP connection is interrupted completely. Shovel won't reject messages, 
but maybe the connection is being interrupted and re-established? You 
should be able to see this in the logs. You would also see a 
"redelivered" message rate in mgmt.

> Is there a way to configured Shovel to guaranty the sequence of messages ?

This should already be happening.

> Second:
>
> ---------
>
> When a simulate a WAN outage lasting 2 hours, messages staying in the
> shovel queue have a status,
>
> first set to “unacknowledged”.
>
> Then, after about 900 sec, they all have their status changed to “ready”.
>
> When the network recovers, all the pending messages change back to
> “unacknowledged”, before being sent
>
> to the distant cluster.
>
> It does not seems to be a problem for us, just I’d like to understand
> this behavior, and if it could impact rabbit performances.

unacknowledged -> the message has been consumed by the remote end, but 
not acknowledged.

After 900 seconds presumably the connection is determined to be dead by 
RabbitMQ and then the messages go back into the queue ("ready").

Then when the network recovers, they are sent again and become 
"unacknowledged" again, until the acks come in.

Cheers, Simon

-- 
Simon MacMullen
RabbitMQ, VMware