[rabbitmq-discuss] is anybody using tx?
aar at cpan.org
Tue Jun 28 08:54:03 BST 2011
On 27-06-2011 at 19:51, Matthias Radestock wrote:
>>I have daemons processing external events and writing to a RDMS,
>>filesystem and RabbitMQ. If something fails -sort of a distributed
>>transaction- I can just rollback everything on RabbitMQ.
>Interesting. What guarantees are you getting out of this? Presumably
>you cannot (easily) roll back the file system write. And
>RDBMS transaction and RabbitMQ transaction are separate, how
>undo the first one when the second one fails?
If a filesystem write fails, I rollback both the RDBMS and
RabbitMQ transactions. Then I send a tx.commit to RabbitMQ; if
this fails I rollback the RDBMS. Finally, I commit the RDBMS.
My message consumers are idempotent and always check object
status in RDBMS before processing messages, so should I fail to
commit to RDBMS after having committed to RabbitMQ, those
messages will be ignored.
I don't care about filesystem, as I have periodic daemons
removing old or unused files anyway.
>More broadly, what is the recovery strategy, i.e. what do you after
>you've done the rolling back? Retry again after a while?
If the external event is synchronous, I just return a temporary
error and have the external client re-submit its request. If
it's asynchronous (like a message consumed from an internal
queue) I just close the AMQP channel without acking the message
and restart consuming later.
>>Plus, what Tim Fox said about atomicity.
>Are you relying on tx atomicity at the moment? If so, can you
>elaborate on the specific use case? Also is the "only atomic
>single queue" restriction of AMQP's tx ok for your app?
Hmm. Er... that limitation sounds new to me. Didn't pay
I'm not totally relying on disk-level atomicity; however the
ability to commit or rollback a bunch of messages with a single
command sounds still more 'atomic' than submitting, say, 5
messages one by one (I might lose network connection to RabbitMQ
between those submits).
More information about the rabbitmq-discuss