[rabbitmq-discuss] is anybody using tx?

Tue Jun 28 08:54:03 BST 2011

On 27-06-2011 at 19:51, Matthias Radestock wrote:

  >>I have daemons processing external events and writing to a RDMS,
  >>filesystem and RabbitMQ. If something fails -sort of a distributed
  >>transaction- I can just rollback everything on RabbitMQ.
  >
  >Interesting. What guarantees are you getting out of this? Presumably
  >you cannot (easily) roll back the file system write. And 
since the
  >RDBMS transaction and RabbitMQ transaction are separate, how 
do you
  >undo the first one when the second one fails?

If a filesystem write fails, I rollback both the RDBMS and 
RabbitMQ transactions. Then I send a tx.commit to RabbitMQ; if 
this fails I rollback the RDBMS. Finally, I commit the RDBMS.

My message consumers are idempotent and always check object 
status in RDBMS before processing messages, so should I fail to 
commit to RDBMS after having committed to RabbitMQ, those 
messages will be ignored.
I don't care about filesystem, as I have periodic daemons 
removing old or unused files anyway.

  >More broadly, what is the recovery strategy, i.e. what do you after
  >you've done the rolling back? Retry again after a while?

If the external event is synchronous, I just return a temporary 
error and have the external client re-submit its request. If 
it's asynchronous (like a message consumed from an internal 
queue) I just close the AMQP channel without acking the message 
and restart consuming later.

  >>Plus, what Tim Fox said about atomicity.
  >
  >Are you relying on tx atomicity at the moment? If so, can you
  >elaborate on the specific use case? Also is the "only atomic 
for a
  >single queue" restriction of AMQP's tx ok for your app?

Hmm. Er... that limitation sounds new to me. Didn't pay 
attention :-)

I'm not totally relying on disk-level atomicity; however the 
ability to commit or rollback a bunch of messages with a single 
command sounds still more 'atomic' than submitting, say, 5 
messages one by one (I might lose network connection to RabbitMQ 
between those submits).

Alessandro