[rabbitmq-discuss] Responsibility Transfer using RabbitMQ .NET API: How it works?

Thu Jan 27 04:34:55 GMT 2011

Hi Alfonso,

> I've been googling a lot and some people suggested using this schema:
> 
> ch.TxSelect()
> ch.BasicPublish(.....)
> ch.TxCommit()
...
> Reading the API user guide I've found that this is called "transfer
> responsibility".

That's correct.  The only way right now to transfer responsibility for
a message to the broker is using transactions.

That said, I don't think it's exactly what you want.  The TxCommit will
succeed if messages are lost during routing (due to non-existing
queues).

> My question is:
> 
> Being TxCommit a void function I tested this block of code using a
> non-existing routing key and I didn't get any exception.
> Since safe publishing would be highly desiderable non routed messages should
> be detected because the application ACKs the messages of "events queue"
> when after evaluating its content has successfully sent another message to
> the "results queue" (currently if a a routing key is not being routed to a
> queue the messages are silently dropped and I have no way to detect the
> failure so the application ACKs the messages from "events queue")
> 
> Please, can someone tell me how I can't get that "CommitOK" responses?

As far as I know, there's no way cause the broker to throw an exception
for unroutable messages.

What you could do instead is this:
  - as you've suggested, publish with mandatory, immediate, and
    persistent set;
  - do publishes to persistent queues;
  - wrap publishes in that tx code;
  - have the publisher watch for basic.returns and handle somehow lost
    messages (maybe rollback the transaction, republish to a different
    and commit).

The message flags and the transactions should ensure that the broker
either save the messages to disk or send it to a consumer.  The
ReturnHandler should allow you to republish or somehow handle lost
messages.

Basic.publish/basic.return aren't synchronous out of the box and aren't
meant to be used like that.  For unroutable messages, the broker sends
the basic.return immediately after processing the basic.publish, so if
you publish to a non-existent queue, you should expect the basic.return
soonish; you could use this to make publishes/returns somewhat
synchronous, but I wouldn't recommend it.

BTW, the AMQP spec says you shouldn't rely on the broker's behaviour
when publishing mandatory/immediate messages inside transactions, but I
think there are any pitfalls doing this with rabbit.

As you may have guessed by now, the scenario you described isn't handled
very nicely by AMQP.  We're introducing publisher confirms to remedy
this.  The code's on default now, and should be in the next release.
The basic idea is that the broker sends a basic.ack back to the
publisher when it's dealt with a message.  In addition, the basic.return
would always be sent before the basic.ack, so the publisher would know
when the message is no longer its responsibility.  Of course, this
isn't synchronous, but it is fast, and the logic is a lot simpler than
trying to do the same thing with transactions.

> PS: I'm using the API version 1.7.2 and RabbitMQ server is the same version.

You really should upgrade.  Since 1.7.2, we've fixed lots of bugs
(including a few serious ones), introduced a new persister (which is
quite relevant if you care about transfer of responsibility) and added some
very nice management and monitoring features (including a web UI).
Publisher confirms are on default now, and should be in the next
release.

Hope this helps,

Cheers,
Alex

On Mon, Jan 24, 2011 at 05:47:17PM +0100, Alfonso Pantoja wrote:
> Hi,
> 
> I've developed an application that consumes messages from a queue ("events
> queue") and depending on their data
> it publishes (per received message) a message to another queue ("results
> queue")
> 
> The logic that checks the content of the received messages is publishing
> messages using BasicPublish (with mandatory=2, immediate=false and setting
> deliveryMode=2 to the messages to be sent)
> My concern is that BasicPublish is asynchronous  and the only exception I
> can get is when there is no connection to RabbitMQ or the destination
> exchange does not exist.
> 
> Since the application logic at this point is synchronous I can't use
> BasicReturn in order to use a handler when messages can't be delivered.
> 
> I've been googling a lot and some people suggested using this schema:
> 
> ch.TxSelect()
> ch.BasicPublish(.....)
> ch.TxCommit()
> 
> and also is suggested that "commit-ok" will be returned if the message has
> been published safely to the broker.
> 
> Reading the API user guide I've found that this is called "transfer
> responsibility".
> 
> Copy & paste follows:
> 
> --
> To transfer responsibility for delivery of a message to a broker
> • ensure (ahead of time) that the target queue exists and is durable,
> • select Tx mode using IModel.TxSelect,
> • publish the message with the "mandatory" flag set and DeliveryMode set
> equal to 2, and
> • commit the Tx transaction using IModel.TxCommit.
> 
> Once a broker replies with CommitOk (i.e. the TxCommit() call returns to the
> caller), it has taken
> responsibility for keeping the message on disk and on the target queue until
> some other application
> retrieves and acknowledges the message.
> A commit is not required after every message: batching of publications may
> be done, depending on the
> precise delivery guarantees the publishing application requires.
> Responsibility can also be placed with an external database, even further
> along the chain - see the section
> on interaction with external resources below
> ---
> 
> My question is:
> 
> Being TxCommit a void function I tested this block of code using a
> non-existing routing key and I didn't get any exception.
> Since safe publishing would be highly desiderable non routed messages should
> be detected because the application ACKs the messages of "events queue"
> when after evaluating its content has successfully sent another message to
> the "results queue" (currently if a a routing key is not being routed to a
> queue the messages are silently dropped and I have no way to detect the
> failure so the application ACKs the messages from "events queue")
> 
> Please, can someone tell me how I can't get that "CommitOK" responses?
> 
> 
> Thank you in advance.
> 
> 
> Alfonso
> 
> PS: I'm using the API version 1.7.2 and RabbitMQ server is the same version.

> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss