[rabbitmq-discuss] Questions on RabbitMQ

Sat Sep 25 13:59:37 BST 2010

On Sep 25, 2010, at 6:07 AM, Ralf Kistner wrote:

> 2 a. Are there any recommended practices for running a (Ruby) process
> to consume messages in an infinite loop?

Check out the AMQP stuff in Ruby. It uses EventMachine. EM will run until you tell it to stop, which is what you're looking for.

> 2 b. The Ruby script would occasionally be unable to process a
> message. This could be because of a temporary lock on the database, an
> invalid message from the mobile client, or a bug on the server. The
> ideal would be for these messages to be retried a few times, then
> moved to a different queue if it still fails. The messages in the
> queue could then be inspected manually, and moved back to the original
> queue when the bug was fixed.
> How would this be accomplished? Does RabbitMQ have something like a
> dead letter queue? Or would I have to move the message to a different
> queue myself (in the Ruby script), in case of an error?
> Note that I can't return any messages to the sender, I want to keep it
> on the server in a different queue.

Redelivery is something I'm working on in my own application. I use it to update databases at remote locations that might or might not be online. If I get a timeout error, I write my message into a Riak bucket. There is a Quartz-based job that runs every five minutes that checks this retry bucket and simply deletes the messages and dumps them back into the broker. I haven't put it in yet, but I'm planning on implementing a "maxRetries" so that when a message exceeds the retry number, it will be either sent to a notification queue where it gets emailed, or moved to another Riak bucket where I can write an RSS feed over the top of it.

> 3. What is the best way to create a backup of all the message sent to
> the server? Does RabbitMQ automatically do this, or should I write an
> additional consumer to create a backup?

A NoSQL store where you keep the entire message in JSON format seems to me the safest and most performant way to persist messages with enough information to be able to resubmit the message at a later time. I have security tokens in the headers (I don't use logins but tokens) and of course I have to capture the exchange and routing key information. I use JSON data for my messages, so this fits nicely together but you could also base64 encode your data.

You could use an additional consumer (e.g. on routing key "#") but I let the consumer decide whether to persist a message to Riak or not.

Thanks!

J. Brisbin
http://jbrisbin.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100925/315aeb74/attachment.htm>