[rabbitmq-discuss] Republishing messages when a broker is unreachable

Jason J. W. Williams jasonjwwilliams at gmail.com
Tue Jun 28 04:07:17 BST 2011

Seems like you're just moving the point of failure around. Granted it's closer to the app, but what if your local broker fails in such a way that your app is still running (e.g. oom, out of disk, etc)? I would either make the action that is generating the publish not return until it gets a publisher confirm from RMQ or implement a separate thread/process to handle the publishing to RMQ and implementing a simple disk-based persistency (don't accept the message to publish until you've gotten it written to a file on the disk, then publish out of memory and only hit the disk again to reap the message lazily after it's been marked "dirty"). We've used the latter approach very successfully. When the app restarts, the publisher thread/process reads it's "backup" directory and publishes the existing (and non-dirty) messages to RMQ. 

Running multiple RMQ brokers in a loose federation with Shovel adds a lot of moving parts that can be difficult to debug. Just my 2 cents. 


Sent via iPhone

Is your email Premiere?

On Jun 27, 2011, at 19:18, James Carr <james.r.carr at gmail.com> wrote:

> After some googling I see that one recommended strategy is to install
> rabbitmq on the clients and use the shovel plugin to push messages
> from local queues to other exchanges. I like the approach but at the
> same time if an organization requires support contracts on anything
> used in a production quality revenue generating way that can shoot the
> support costs to the stratosphere. ;)
> Another problem that comes to mind is the fact that quite few clients
> in our webfarm are already approaching their limits on memory and CPU
> usage... installing rabbitmq on each of them feels a bit heavy.
> Any other recommended solutions?
> Thanks,
> James
> On Mon, Jun 27, 2011 at 6:57 PM, James Carr <james.r.carr at gmail.com> wrote:
>> Hi All,
>> Although I can think of several homebrew strategies, what are some
>> existing strategies/libraries/frameworks for retrying messages that
>> were unpublishable due to the broker being unreachable (network
>> outage, maintenance, etc)?
>> Thanks,
>> James
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

More information about the rabbitmq-discuss mailing list