[rabbitmq-discuss] Questions on RabbitMQ

Sun Oct 3 18:25:49 BST 2010

Thanks for the response.

EventMachine looks perfect for my Ruby script.

I'm already using MongoDB for other parts of my application, so I'll
use it to implement a similar approach to what you suggested.
My messages will mostly be binary data, but that shouldn't make a big
difference.

I'll use an additional consumer (on routing key "#") , to ensure that
the messages are all persisted even when my consumer is broken and
silently discards messages (this happened to me in the past).

Ralf

On Sat, Sep 25, 2010 at 2:59 PM, Jon Brisbin <jon at jbrisbin.com> wrote:
> On Sep 25, 2010, at 6:07 AM, Ralf Kistner wrote:
>
> 2 a. Are there any recommended practices for running a (Ruby) process
> to consume messages in an infinite loop?
>
> Check out the AMQP stuff in Ruby. It uses EventMachine. EM will run until
> you tell it to stop, which is what you're looking for.
>
> 2 b. The Ruby script would occasionally be unable to process a
> message. This could be because of a temporary lock on the database, an
> invalid message from the mobile client, or a bug on the server. The
> ideal would be for these messages to be retried a few times, then
> moved to a different queue if it still fails. The messages in the
> queue could then be inspected manually, and moved back to the original
> queue when the bug was fixed.
> How would this be accomplished? Does RabbitMQ have something like a
> dead letter queue? Or would I have to move the message to a different
> queue myself (in the Ruby script), in case of an error?
> Note that I can't return any messages to the sender, I want to keep it
> on the server in a different queue.
>
> Redelivery is something I'm working on in my own application. I use it to
> update databases at remote locations that might or might not be online. If I
> get a timeout error, I write my message into a Riak bucket. There is a
> Quartz-based job that runs every five minutes that checks this retry bucket
> and simply deletes the messages and dumps them back into the broker. I
> haven't put it in yet, but I'm planning on implementing a "maxRetries" so
> that when a message exceeds the retry number, it will be either sent to a
> notification queue where it gets emailed, or moved to another Riak bucket
> where I can write an RSS feed over the top of it.
>
> 3. What is the best way to create a backup of all the message sent to
> the server? Does RabbitMQ automatically do this, or should I write an
> additional consumer to create a backup?
>
> A NoSQL store where you keep the entire message in JSON format seems to me
> the safest and most performant way to persist messages with enough
> information to be able to resubmit the message at a later time. I have
> security tokens in the headers (I don't use logins but tokens) and of course
> I have to capture the exchange and routing key information. I use JSON data
> for my messages, so this fits nicely together but you could also base64
> encode your data.
> You could use an additional consumer (e.g. on routing key "#") but I let the
> consumer decide whether to persist a message to Riak or not.
>
> Thanks!
> J. Brisbin
> http://jbrisbin.com/
>
>
>
>
>
>