[rabbitmq-discuss] Responsibility Transfer using RabbitMQ .NET API: How it works?

Fri Jan 28 00:13:36 GMT 2011

Alfonso,

> In this case in my code I should handle unexpected exceptions (txSelect,
> basic.publish, txCommit) and also basic.return in cause immediate and/or
> mandatory message can't be delivered, right?

Ah, you're right; I forgot about the exceptions.  So, yes, you need to
take into account both exceptions (failed commits) and basic.returns
(unroutable messages).

Cheers,
Alex

On Fri, Jan 28, 2011 at 12:40:35AM +0100, Alfonso Pantoja wrote:
> Hi Alex,
> 
> 
> Firstly, thank you very much for your answer. You've really helped me.
> As you suggested we'll update RabbitMQ to version 2.2.0 as soon as possible.
> It is great you are introducing publisher confirms in next release because
> it will offer more fine control to message publishing.
> 
> 
> Regarding transfer responsibility I have one more doubt.
> As you said, "TxCommit will succeed if messages are lost during routing (due
> to non-existing queues)",
> so does it mean that TxCommit launches an exception when it fails?
> 
> In this case in my code I should handle unexpected exceptions (txSelect,
> basic.publish, txCommit) and also basic.return in cause immediate and/or
> mandatory message can't be delivered, right?
> 
> 
> 
> Regards,
> 
> Alfonso
> 
> 
> 
> 
> 2011/1/27 Alexandru Scvorţov <alexandru at rabbitmq.com>
> 
> > Hi Alfonso,
> >
> > > I've been googling a lot and some people suggested using this schema:
> > >
> > > ch.TxSelect()
> > > ch.BasicPublish(.....)
> > > ch.TxCommit()
> > ...
> > > Reading the API user guide I've found that this is called "transfer
> > > responsibility".
> >
> > That's correct.  The only way right now to transfer responsibility for
> > a message to the broker is using transactions.
> >
> > That said, I don't think it's exactly what you want.  The TxCommit will
> > succeed if messages are lost during routing (due to non-existing
> > queues).
> >
> >
> > > My question is:
> > >
> > > Being TxCommit a void function I tested this block of code using a
> > > non-existing routing key and I didn't get any exception.
> > > Since safe publishing would be highly desiderable non routed messages
> > should
> > > be detected because the application ACKs the messages of "events queue"
> > > when after evaluating its content has successfully sent another message
> > to
> > > the "results queue" (currently if a a routing key is not being routed to
> > a
> > > queue the messages are silently dropped and I have no way to detect the
> > > failure so the application ACKs the messages from "events queue")
> > >
> > > Please, can someone tell me how I can't get that "CommitOK" responses?
> >
> > As far as I know, there's no way cause the broker to throw an exception
> > for unroutable messages.
> >
> > What you could do instead is this:
> >  - as you've suggested, publish with mandatory, immediate, and
> >    persistent set;
> >  - do publishes to persistent queues;
> >  - wrap publishes in that tx code;
> >  - have the publisher watch for basic.returns and handle somehow lost
> >    messages (maybe rollback the transaction, republish to a different
> >    and commit).
> >
> > The message flags and the transactions should ensure that the broker
> > either save the messages to disk or send it to a consumer.  The
> > ReturnHandler should allow you to republish or somehow handle lost
> > messages.
> >
> > Basic.publish/basic.return aren't synchronous out of the box and aren't
> > meant to be used like that.  For unroutable messages, the broker sends
> > the basic.return immediately after processing the basic.publish, so if
> > you publish to a non-existent queue, you should expect the basic.return
> > soonish; you could use this to make publishes/returns somewhat
> > synchronous, but I wouldn't recommend it.
> >
> > BTW, the AMQP spec says you shouldn't rely on the broker's behaviour
> > when publishing mandatory/immediate messages inside transactions, but I
> > think there are any pitfalls doing this with rabbit.
> >
> >
> > As you may have guessed by now, the scenario you described isn't handled
> > very nicely by AMQP.  We're introducing publisher confirms to remedy
> > this.  The code's on default now, and should be in the next release.
> > The basic idea is that the broker sends a basic.ack back to the
> > publisher when it's dealt with a message.  In addition, the basic.return
> > would always be sent before the basic.ack, so the publisher would know
> > when the message is no longer its responsibility.  Of course, this
> > isn't synchronous, but it is fast, and the logic is a lot simpler than
> > trying to do the same thing with transactions.
> >
> >
> > > PS: I'm using the API version 1.7.2 and RabbitMQ server is the same
> > version.
> >
> > You really should upgrade.  Since 1.7.2, we've fixed lots of bugs
> > (including a few serious ones), introduced a new persister (which is
> > quite relevant if you care about transfer of responsibility) and added some
> > very nice management and monitoring features (including a web UI).
> > Publisher confirms are on default now, and should be in the next
> > release.
> >
> >
> > Hope this helps,
> >
> > Cheers,
> > Alex
> >
> > On Mon, Jan 24, 2011 at 05:47:17PM +0100, Alfonso Pantoja wrote:
> > > Hi,
> > >
> > > I've developed an application that consumes messages from a queue
> > ("events
> > > queue") and depending on their data
> > > it publishes (per received message) a message to another queue ("results
> > > queue")
> > >
> > > The logic that checks the content of the received messages is publishing
> > > messages using BasicPublish (with mandatory=2, immediate=false and
> > setting
> > > deliveryMode=2 to the messages to be sent)
> > > My concern is that BasicPublish is asynchronous  and the only exception I
> > > can get is when there is no connection to RabbitMQ or the destination
> > > exchange does not exist.
> > >
> > > Since the application logic at this point is synchronous I can't use
> > > BasicReturn in order to use a handler when messages can't be delivered.
> > >
> > > I've been googling a lot and some people suggested using this schema:
> > >
> > > ch.TxSelect()
> > > ch.BasicPublish(.....)
> > > ch.TxCommit()
> > >
> > > and also is suggested that "commit-ok" will be returned if the message
> > has
> > > been published safely to the broker.
> > >
> > > Reading the API user guide I've found that this is called "transfer
> > > responsibility".
> > >
> > > Copy & paste follows:
> > >
> > > --
> > > To transfer responsibility for delivery of a message to a broker
> > > • ensure (ahead of time) that the target queue exists and is durable,
> > > • select Tx mode using IModel.TxSelect,
> > > • publish the message with the "mandatory" flag set and DeliveryMode set
> > > equal to 2, and
> > > • commit the Tx transaction using IModel.TxCommit.
> > >
> > > Once a broker replies with CommitOk (i.e. the TxCommit() call returns to
> > the
> > > caller), it has taken
> > > responsibility for keeping the message on disk and on the target queue
> > until
> > > some other application
> > > retrieves and acknowledges the message.
> > > A commit is not required after every message: batching of publications
> > may
> > > be done, depending on the
> > > precise delivery guarantees the publishing application requires.
> > > Responsibility can also be placed with an external database, even further
> > > along the chain - see the section
> > > on interaction with external resources below
> > > ---
> > >
> > > My question is:
> > >
> > > Being TxCommit a void function I tested this block of code using a
> > > non-existing routing key and I didn't get any exception.
> > > Since safe publishing would be highly desiderable non routed messages
> > should
> > > be detected because the application ACKs the messages of "events queue"
> > > when after evaluating its content has successfully sent another message
> > to
> > > the "results queue" (currently if a a routing key is not being routed to
> > a
> > > queue the messages are silently dropped and I have no way to detect the
> > > failure so the application ACKs the messages from "events queue")
> > >
> > > Please, can someone tell me how I can't get that "CommitOK" responses?
> > >
> > >
> > > Thank you in advance.
> > >
> > >
> > > Alfonso
> > >
> > > PS: I'm using the API version 1.7.2 and RabbitMQ server is the same
> > version.
> >
> > > _______________________________________________
> > > rabbitmq-discuss mailing list
> > > rabbitmq-discuss at lists.rabbitmq.com
> > > https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> >
> >