[rabbitmq-discuss] Very slow get (dequeue) on node B if shovel on node A is configured with {ack_mode, on_confirm}

[Simon MacMullen]

That sounds very wrong.

Which version of RabbitMQ are you using? Can you post (a cut down 
version of) your PHP script somewhere? (I have no familiarity with the 
PHP client but I'd like to see what it's doing...)

Cheers, Simon

On 13/02/13 14:33, bratner bratner wrote:
> Hi!
>
> My setup includes a rabbitmq-c application that publishes messages on
> node A.
> The publishing rate is about 200mps, each message can be up to 10Kb.
>
> Shovel (node A) is configured to move them to node B.
> On node B i'm dequeuing the messages with a PHP test script with
> $queue->get(AMQP_AUTOACK).
> If i keep the publishing rate at 200mps then i can see that the queue on
> node A is empty and the Q on node B is
> filling up. The read-rate of my test script is really low.
>
> If i stop the publishing, 3-5 seconds later, my test script starts
> reading like crazy until the queue on node B is empty.
>
> Even If I slow down the publishing rate to 5mps , same is happening ,
> messages are piling up on node B until i dial down the pressure.
>
> This problem disappears if I set ack_mode to on_publish or no_ack. In
> this case the reader script reads with the publishing speed.
>
> My configuration :
>
>
>
> [
> {rabbit, [
> {log_levels, [{connection, error}]}
> ]},
> {rabbitmq_shovel,
> [ {shovels, [ {messagemover, [
> {sources, [
> {broker, "amqp://usera:pass@localhost/vhosta
> <amqp://cdrposter:4VkI6MKH@localhost/sipout>"},
> {declarations, [
> {'exchange.declare',[{exchange, <<"my-fanout">>},{type,
> <<"fanout">>},durable]},
> {'queue.declare',[{queue,<<"messages">>},durable]},
> {'queue.bind',[{exchange, <<"my-fanout">>},{queue, <<"messages">>}]}
> ]}
> ]},
> {destinations, [
> {brokers, [ "amqp://userb:pass@nodeB/vhostb
> <amqp://cdr_manager:Ci2XOb3b@10.200.10.218/cdrs>" ]},
> {declarations, [
> {'exchange.declare',[{exchange, <<"my-fanout">>},{type,
> <<"fanout">>},durable]},
> {'queue.declare',[{queue,<<"messages">>},durable]},
> {'queue.bind',[{exchange, <<"my-fanout">>},{queue, <<"messages">>}]}
> ]}
> ]},
> {queue, <<"messages">>},
> {prefetch_count, 200},
> {ack_mode, on_confirm},
> {publish_properties, [{delivery_mode, 2}]},
> {reconnect_delay, 5}
> ]}
> ]}
> ]}
> ].
>
>
> Thank you,
> Boris.
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>


-- 
Simon MacMullen
RabbitMQ, VMware