[rabbitmq-discuss] Rabbitmq-Shovel Timer-Based Publishing Possible?
allen.kim at epicadvertising.com
Thu Mar 17 14:01:01 GMT 2011
Replying in the context
On 11-03-16 6:00 PM, "Matthias Radestock" <matthias at rabbitmq.com> wrote:
>Allen Kim wrote:
>> Message volume fluctuates from extremely high to very low depending on
>> time of day.
>> We use a relatively large transaction size on the outbound, publishing,
>> connection to get high enough throughput on a high latency link for the
>> peak times of day. However, at the off peak times of day, it can take
>> to a few minutes for the shovel to accumulate enough messages to hit the
>> configured transaction commit size.
>> Allowing a configurable transaction size plus commit timer adds some
>> flexibility. At peak times, the transaction size counter is always hit
>> first. At low times, the timer hits first and ensures that message
>> delivery does not stall.
>So you are employing transactions in order to get guaranteed delivery,
Sorry, I am not talking about guaranteed delivery, but I am talking about
delivery on time.
Think about this situation,
Data center A has a transaction, which must be passed to data center B IN
I haven't thought about guaranteed delivery that you mentioned, which
sounds really good.
>I can see at least two alternative approaches here...
>1) instead of committing after a certain size/time, grow the tx as long
>as there are messages and only commit when there is a gap.
I don't get "when there is a gap". Gap between what?
>2) use confirms (http://www.rabbitmq.com/extensions.html#confirms) and
>only ack a message to the source once it has been confirmed by the
>Both approaches require zero config - no obscure time/count knobs to
>tweak - which makes them rather appealing.
>Unfortunately the shovel currently doesn't support either of these. I've
>filed a bug to look into the latter.
More information about the rabbitmq-discuss