[rabbitmq-discuss] tx_commit() semantics with async connections (e.g Pika SelectConnection)

Fri Aug 17 22:12:27 BST 2012

Thanks for the background Gavin. It's glad to get confirmation that I was
on the right track with my understanding.

A few thoughts:

It might be worth adding some sort of warning output if there's a
synchronous command in the output buffer and then another "method" (e.g.,
Basic.Publish) get's added.

It would be nice to have some mechanism of saying "Go send whatever output
data you have, right now", then wait for some data to arrive back from the
server. The incoming data wouldn't have to be processed before returning,
but you'd at least know that you could safely send more methods without
breaking AMQP rules.

My thinking is that after calling tx_commit() in my message handler, I
could call this method.

Lastly, it would be really helpful to update the Pika doc to reflect what
you said in your reply. In particular, this except from the
Channel.tx_commit() description appears incorrect, or at least confusing:

*This is a synchronous method that will not allow other commands to be send
to the AMQP broker until it has completed.*

Thanks again,

Matt
On Fri, Aug 17, 2012 at 12:38 PM, Gavin M. Roy <gmr at meetme.com> wrote:

>
>
> On Fri, Aug 17, 2012 at 3:13 PM, Matt Pietrek <mpietrek at skytap.com> wrote:
>
>> I've been drilling into an issue for a few days where my Pika channel is
>> abruptly closed. I'm pretty sure I've found either:
>>
>>    - A large gap in my understanding
>>    - A misuse of the Rabbit/Pika APIs
>>
>> Depending on your perspective, this seems the most likely, in that there
> is not any logic in Pika for managing Tx. It lets you send the RPC calls
> and leaves it to you to manage state outside of the connection/channel.
>
>>
>>    - A serious bug in Pika.
>>
>> My original understanding about transactions is that calling tx_select()
>> is a synchronous operation, and that when it returns, whatever actions
>> you've sent to the broker are committed.
>>
>> However, in stepping through the Pika tx_commit() code, it seems like it
>> all it does is add the Tx.Commit message to an outgoing buffer, and that
>> the tx_commit() call returns without anything actually sent to the broker.
>> (I've set breakpoints in the debugger at strategic points to verify this.)
>>
> This is correct, because pika is event-loop driven, it is putting the
> frame in the output buffer and returning to you. The event loop will write
> the data out as the operating system tells it it can You are correct that
> it is not blocking on the Tx.Commit until the data is written over the
> socket.
>
>> In my message handler, I'm writing a reply to the incoming message, using
>> the same channel. What I'm seeing (verified via WireShark) is that the
>> Tx.Commit message is followed by some number of Basic.Publish,
>> Content-Header, and Content-Body records corresponding to my reply message
>> writes.
>>
> Your frames are sent in the order the methods are called.
>
>> This violates the AMQP standard, which says that nothing should be sent
>> to the broker after sending a Tx.Commit and before receiving a Tx.Commit-Ok
>> reply. (If it helps understanding, my incoming message queue has a few
>> messages in it already, so I see a series of incoming messages immediately
>> after starting.)
>>
> You should add a callback for the Tx.Commit-Ok reply and then send your
> messages. You are correct that Pika is not enforcing the behavior of
> blocking until Tx.Commit-Ok.
>
> You could open a 2nd channel and send your replies over that.
>
>> I see that the tx_commit() method takes a callback method, but if I were
>> to simply block in my message handler, waiting for the callback to be
>> invoked, I'd deadlock because Pika's buffer processing code doesn't get a
>> chance to run.
>>
> How are you blocking? If you're using any truly blocking method, it will
> stop the event loop from running. Since you're on the async loop, I'd use a
> timer and a state variable to toggle between the various states you're
> trying to achieve.
>
> Gavin
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120817/a415ade2/attachment.htm>