[rabbitmq-discuss] Republishing messages when a broker is unreachable

[James Carr]

Yeah, that was along a similar idea I was considering. In node.js I
already do this by having a webworker (which runs in a separate
process) be responsible for publishing and retrying. In java we could
probably do something similar and use a simple ehcache based solution
(since it already overflows to disk when a certain memory threshold is
reached), caching and retrying messages that are published while the
broker is down. We could then use ehcache's write to disk on crash if
that process crashes.

Sigh... it does feel like a bit of shuffling the point of failure around. :)

Thanks,
James

On Mon, Jun 27, 2011 at 10:07 PM, Jason J. W. Williams
<jasonjwwilliams at gmail.com> wrote:
> Seems like you're just moving the point of failure around. Granted it's closer to the app, but what if your local broker fails in such a way that your app is still running (e.g. oom, out of disk, etc)? I would either make the action that is generating the publish not return until it gets a publisher confirm from RMQ or implement a separate thread/process to handle the publishing to RMQ and implementing a simple disk-based persistency (don't accept the message to publish until you've gotten it written to a file on the disk, then publish out of memory and only hit the disk again to reap the message lazily after it's been marked "dirty"). We've used the latter approach very successfully. When the app restarts, the publisher thread/process reads it's "backup" directory and publishes the existing (and non-dirty) messages to RMQ.
>
> Running multiple RMQ brokers in a loose federation with Shovel adds a lot of moving parts that can be difficult to debug. Just my 2 cents.
>
> -J
>
> Sent via iPhone
>
> Is your email Premiere?
>
> On Jun 27, 2011, at 19:18, James Carr <james.r.carr at gmail.com> wrote:
>
>> After some googling I see that one recommended strategy is to install
>> rabbitmq on the clients and use the shovel plugin to push messages
>> from local queues to other exchanges. I like the approach but at the
>> same time if an organization requires support contracts on anything
>> used in a production quality revenue generating way that can shoot the
>> support costs to the stratosphere. ;)
>>
>> Another problem that comes to mind is the fact that quite few clients
>> in our webfarm are already approaching their limits on memory and CPU
>> usage... installing rabbitmq on each of them feels a bit heavy.
>>
>> Any other recommended solutions?
>>
>> Thanks,
>> James
>>
>> On Mon, Jun 27, 2011 at 6:57 PM, James Carr <james.r.carr at gmail.com> wrote:
>>> Hi All,
>>>
>>> Although I can think of several homebrew strategies, what are some
>>> existing strategies/libraries/frameworks for retrying messages that
>>> were unpublishable due to the broker being unreachable (network
>>> outage, maintenance, etc)?
>>>
>>> Thanks,
>>> James
>>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>