[rabbitmq-discuss] Performance of "blocking publisher-confirms" vs. transactions

[Matt Pietrek]

Simon,

What you're saying make sense from a purely technical standpoint. We could
just not consider the message sent until seeing the publisher confirm.

However, implementing this in our code would push a fair amount of
complexity back into our application layer. It's already a highly complex
state machine and adding another state, e.g. "the message *may* have been
sent", would make things far worse than they already are.

Also, I noticed that you have an exactly-once QOS requirement, whereas we
don't. We need to be tolerant of duplicate messages, but we absolutely must
get every message at least once to drive the state machine forward. Losing
a message in the "confirm window" could cause entire operations to stall.
And adding async callback into our state machine logic to change
operational state from "maybe sent" to "guaranteed sent" is a non-starter
with the other system architects.


On Thu, Feb 16, 2012 at 3:06 PM, Simone Busoli <simone.busoli at gmail.com>wrote:

> Hello Matt, thanks for the explanation and sorry for not reading the
> previous thread carefully, I did now.
>
> I can't tell you what differences you should expect between using a
> blocking publisher confirms model and transactions as I've never used
> transactions before, but allow me to insist a bit more on why the first may
> fit your scenario by describing mine. I'm using RabbitMQ in an environment
> where high reliability, together with strict ordering of messages and
> exactly-once quality of service are needed. We're using asynchronous
> publisher confirms to achieve that.
>
> You seem to be worried about a scenario in which you believe some messages
> to be in the hands of the server while instead they aren't due to the
> asynchronous nature of publisher confirms, and especially that the
> publishers may not be there to republish them in case something goes wrong.
> Without knowing much about your system I would suggest that you don't
> consider the messages as "delivered" until you receive the asynchronous
> confirm from the broker, and just pretend they were never published until
> this happens. The problem you describe seems to arise because you consider
> published messages as "delivered", which is not true until you receive a
> confirm.
> So what's worst case scenario difference between publisher confirms used
> in this way compared to transactions? If a publisher dies, in the first
> case you would consider as published only the messages that have been
> confirmed, thus pretending the unconfirmed ones to have never existed, in
> the second case an hypothetical ongoing transaction is rolled back. I think
> the difference here lies only in the number of messages you have tried to
> publish which may not have reached the broker. In the case of transactions
> it is at most 1, with async publisher confirms it may be more than 1 (it
> will be 1 with synchronous publisher confirms), but I don't think this
> number changes the nature of the problem.
>
> On Thu, Feb 16, 2012 at 22:44, Matt Pietrek <mpietrek at skytap.com> wrote:
>
>> Hi Simone,
>>
>> | Can you describe why you think publisher confirms are not appropriate
>> in your scenario?
>>
>> I think if you read the paragraph that starts with "At the heart of
>> things" it describes the scenario in enough detail. In a follow up, Simon
>> MacMullen appears to agree that publisher confirms aren't ideal for us.
>>
>> In short, when we publish a message, we must know that the broker safely
>> has it. With transactions, OR a blocking publisher-confirm model, the
>> client is assured it has it.
>>
>> HOWEVER, all the publisher-confirm descriptions/examples I've seen assume
>> an async model: Publish a message and some time later you'll be told that
>> the message is safely in the broker's hands. However, that window between
>> those two events is the crux of the problem in our environment.
>>
>> Assume we went with the async publisher-confirm model as suggested. What
>> if our client dies and is not around to re-send the message? Or what if the
>> client dies, and milliseconds later the broker dies? No publisher confirm
>> would be forthcoming, and hence, lost messages.
>>
>> Top level problem for us: Our state machine, which persists every
>> critical change of state to a database, will be incorrect. We will have
>> recorded "Yup, that message was sent", when in reality it might not have
>> made it to broker, or the broker died before the confirm could be sent.
>>
>> By not recording that we've sent the message until AFTER a
>> transaction/publisher-confirm completes, we can guarantee the integrity of
>> our state-machine. However, as we've all seen, transactions impose a limit
>> on message throughput. I'm simply trying to understand if there's any
>> non-trivial message rate difference expected between using a transaction
>> vs. waiting in my publish routine for the confirm.
>>
>>
>> | If you happen to be using the .NET API this is already implemented, the
>> WaitForConfirms and WaitForConfirmsOrDie on the IModel interface provide
>> this behavior.
>>
>> Unfortunately, I'm not on a .NET client. I'm using Python/Pika. I'd
>> consider going to the c-library (wrapped from Python) if there's some
>> non-trivial benefit to using a synchronous publisher-confirm.
>>
>> Thanks,
>>
>> Matt
>>
>> On Thu, Feb 16, 2012 at 1:08 PM, Simone Busoli <simone.busoli at gmail.com>wrote:
>>
>>> Hi Matt, I've gone through the old thread very quickly so forgive me if
>>> I'm missing something. Your first sentence leaves me with some doubts:
>>>
>>>  As I've detailed in other posts to this DL (http://bit.ly/zf6sKC), we
>>>> have situation where straightforward publisher-confirms aren't appropriate
>>>> in our scenario. We need to know the broker has received the message, and
>>>> we have to assume our client can die at any moment, so retransmitting
>>>> messages is not an option for us.
>>>>
>>>
>>> Publisher confirms guarantee that the broker has taken care of the
>>> message and it has been either delivered to a queue or to a consumer. This
>>> is a strong delivery guarantee, even if the client dies. Coupled with
>>> persistent messages and durable exchange/queues, it guarantees
>>> at-least-once delivery. Can you describe why you think publisher confirms
>>> are not appropriate in your scenario?
>>>
>>>
>>>> What I'd like to know is: Is there any benefit to creating our own
>>>> "blocking publisher-confirm publishing"? That is, we'd do the following:
>>>>
>>>> Publish a single message
>>>> Block on an event 'X'
>>>> When the publisher confirm comes in, signal 'X'
>>>> Return from the publish call
>>>>
>>>
>>> If you happen to be using the .NET API this is already implemented, the
>>> WaitForConfirms and WaitForConfirmsOrDie on the IModel interface provide
>>> this behavior.
>>>
>>>
>>>> In a mirrored queue scenario, is there any advantage to doing this vs.
>>>> using straight-up transactions?
>>>>
>>>> Matt
>>>>
>>>> p.s. I realize in the above pseudocode that multiple threads would be
>>>> necessary - One to block and the other to listen for the publisher-confirm.
>>>>
>>>>
>>>> _______________________________________________
>>>> rabbitmq-discuss mailing list
>>>> rabbitmq-discuss at lists.rabbitmq.com
>>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120217/8977dc21/attachment.htm>