[rabbitmq-discuss] Rabbitmq-Shovel Timer-Based Publishing Possible?

[Allen Kim]

Message volume fluctuates from extremely high to very low depending on the
time of day.

We use a relatively large transaction size on the outbound, publishing,
connection to get high enough throughput on a high latency link for the
peak times of day.  However, at the off peak times of day, it can take up
to a few minutes for the shovel to accumulate enough messages to hit the
configured transaction commit size.

Allowing a configurable transaction size plus commit timer adds some
flexibility.  At peak times, the transaction size counter is always hit
first.  At low times, the timer hits first and ensures that  message
delivery does not stall.


Allen


On 11-03-16 11:46 AM, "Matthias Radestock" <matthias at rabbitmq.com> wrote:

>Allen,
>
>On 15/03/11 15:33, Allen Kim wrote:
>> Timer-based publish(shovel) means publishing on every seconds or minute
>> regardless of number of queue stacked.
>> In our case, if we publish only on based on number of queues, it will
>> publish very busy during day time and slowly at night.
>>
>> The worst case that I see is it does not shovel when there is no queue
>> increasing.
>> I want queues shovelled every few seconds, guaranteed, so that other
>> cluser receive messages not too late.
>
>The shovel maintains permanent connections and reconnects them on
>failure. Messages get shovelled straight away.
>
>Does that meet your requirements? Perhaps I am misunderstanding your use
>case, but I don't see the benefit of shovelling at regular intervals
>only instead of all the time.
>
>Regards,
>
>Matthias.