[rabbitmq-discuss] Responsibility Transfer using RabbitMQ .NET API: How it works?

[Alfonso Pantoja]

Hi Alex,


Firstly, thank you very much for your answer. You've really helped me.
As you suggested we'll update RabbitMQ to version 2.2.0 as soon as possible.
It is great you are introducing publisher confirms in next release because
it will offer more fine control to message publishing.


Regarding transfer responsibility I have one more doubt.
As you said, "TxCommit will succeed if messages are lost during routing (due
to non-existing queues)",
so does it mean that TxCommit launches an exception when it fails?

In this case in my code I should handle unexpected exceptions (txSelect,
basic.publish, txCommit) and also basic.return in cause immediate and/or
mandatory message can't be delivered, right?



Regards,

Alfonso




2011/1/27 Alexandru Scvorţov <alexandru at rabbitmq.com>

> Hi Alfonso,
>
> > I've been googling a lot and some people suggested using this schema:
> >
> > ch.TxSelect()
> > ch.BasicPublish(.....)
> > ch.TxCommit()
> ...
> > Reading the API user guide I've found that this is called "transfer
> > responsibility".
>
> That's correct.  The only way right now to transfer responsibility for
> a message to the broker is using transactions.
>
> That said, I don't think it's exactly what you want.  The TxCommit will
> succeed if messages are lost during routing (due to non-existing
> queues).
>
>
> > My question is:
> >
> > Being TxCommit a void function I tested this block of code using a
> > non-existing routing key and I didn't get any exception.
> > Since safe publishing would be highly desiderable non routed messages
> should
> > be detected because the application ACKs the messages of "events queue"
> > when after evaluating its content has successfully sent another message
> to
> > the "results queue" (currently if a a routing key is not being routed to
> a
> > queue the messages are silently dropped and I have no way to detect the
> > failure so the application ACKs the messages from "events queue")
> >
> > Please, can someone tell me how I can't get that "CommitOK" responses?
>
> As far as I know, there's no way cause the broker to throw an exception
> for unroutable messages.
>
> What you could do instead is this:
>  - as you've suggested, publish with mandatory, immediate, and
>    persistent set;
>  - do publishes to persistent queues;
>  - wrap publishes in that tx code;
>  - have the publisher watch for basic.returns and handle somehow lost
>    messages (maybe rollback the transaction, republish to a different
>    and commit).
>
> The message flags and the transactions should ensure that the broker
> either save the messages to disk or send it to a consumer.  The
> ReturnHandler should allow you to republish or somehow handle lost
> messages.
>
> Basic.publish/basic.return aren't synchronous out of the box and aren't
> meant to be used like that.  For unroutable messages, the broker sends
> the basic.return immediately after processing the basic.publish, so if
> you publish to a non-existent queue, you should expect the basic.return
> soonish; you could use this to make publishes/returns somewhat
> synchronous, but I wouldn't recommend it.
>
> BTW, the AMQP spec says you shouldn't rely on the broker's behaviour
> when publishing mandatory/immediate messages inside transactions, but I
> think there are any pitfalls doing this with rabbit.
>
>
> As you may have guessed by now, the scenario you described isn't handled
> very nicely by AMQP.  We're introducing publisher confirms to remedy
> this.  The code's on default now, and should be in the next release.
> The basic idea is that the broker sends a basic.ack back to the
> publisher when it's dealt with a message.  In addition, the basic.return
> would always be sent before the basic.ack, so the publisher would know
> when the message is no longer its responsibility.  Of course, this
> isn't synchronous, but it is fast, and the logic is a lot simpler than
> trying to do the same thing with transactions.
>
>
> > PS: I'm using the API version 1.7.2 and RabbitMQ server is the same
> version.
>
> You really should upgrade.  Since 1.7.2, we've fixed lots of bugs
> (including a few serious ones), introduced a new persister (which is
> quite relevant if you care about transfer of responsibility) and added some
> very nice management and monitoring features (including a web UI).
> Publisher confirms are on default now, and should be in the next
> release.
>
>
> Hope this helps,
>
> Cheers,
> Alex
>
> On Mon, Jan 24, 2011 at 05:47:17PM +0100, Alfonso Pantoja wrote:
> > Hi,
> >
> > I've developed an application that consumes messages from a queue
> ("events
> > queue") and depending on their data
> > it publishes (per received message) a message to another queue ("results
> > queue")
> >
> > The logic that checks the content of the received messages is publishing
> > messages using BasicPublish (with mandatory=2, immediate=false and
> setting
> > deliveryMode=2 to the messages to be sent)
> > My concern is that BasicPublish is asynchronous  and the only exception I
> > can get is when there is no connection to RabbitMQ or the destination
> > exchange does not exist.
> >
> > Since the application logic at this point is synchronous I can't use
> > BasicReturn in order to use a handler when messages can't be delivered.
> >
> > I've been googling a lot and some people suggested using this schema:
> >
> > ch.TxSelect()
> > ch.BasicPublish(.....)
> > ch.TxCommit()
> >
> > and also is suggested that "commit-ok" will be returned if the message
> has
> > been published safely to the broker.
> >
> > Reading the API user guide I've found that this is called "transfer
> > responsibility".
> >
> > Copy & paste follows:
> >
> > --
> > To transfer responsibility for delivery of a message to a broker
> > • ensure (ahead of time) that the target queue exists and is durable,
> > • select Tx mode using IModel.TxSelect,
> > • publish the message with the "mandatory" flag set and DeliveryMode set
> > equal to 2, and
> > • commit the Tx transaction using IModel.TxCommit.
> >
> > Once a broker replies with CommitOk (i.e. the TxCommit() call returns to
> the
> > caller), it has taken
> > responsibility for keeping the message on disk and on the target queue
> until
> > some other application
> > retrieves and acknowledges the message.
> > A commit is not required after every message: batching of publications
> may
> > be done, depending on the
> > precise delivery guarantees the publishing application requires.
> > Responsibility can also be placed with an external database, even further
> > along the chain - see the section
> > on interaction with external resources below
> > ---
> >
> > My question is:
> >
> > Being TxCommit a void function I tested this block of code using a
> > non-existing routing key and I didn't get any exception.
> > Since safe publishing would be highly desiderable non routed messages
> should
> > be detected because the application ACKs the messages of "events queue"
> > when after evaluating its content has successfully sent another message
> to
> > the "results queue" (currently if a a routing key is not being routed to
> a
> > queue the messages are silently dropped and I have no way to detect the
> > failure so the application ACKs the messages from "events queue")
> >
> > Please, can someone tell me how I can't get that "CommitOK" responses?
> >
> >
> > Thank you in advance.
> >
> >
> > Alfonso
> >
> > PS: I'm using the API version 1.7.2 and RabbitMQ server is the same
> version.
>
> > _______________________________________________
> > rabbitmq-discuss mailing list
> > rabbitmq-discuss at lists.rabbitmq.com
> > https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110128/64fe947c/attachment-0001.htm>