[rabbitmq-discuss] Cluster Pathology

Thu Feb 12 03:25:09 GMT 2009

Hey Drew,

Our experience was that messages weren't lost (if marked persistent on
a durable queue), they just weren't consumed until the consumers were
restarted.

-J

On Wed, Feb 11, 2009 at 8:23 PM, Drew Smathers <drew.smathers at gmail.com> wrote:
> On Wed, Feb 11, 2009 at 6:34 PM, Jason J. W. Williams
> <jasonjwwilliams at gmail.com> wrote:
>> Drew,
>>
>>> Now, if node A is taken down, producer A of course can't produce
>>> messages due to socket errors, but producer B continues producing
>>> messages not knowing they actually are just getting dropped with no
>>> queue to route to (since Q is on node A).  What's even more surprising
>>> to me on this matter is that if producer B restarts and publishes
>>> messages, the messages are still blackholed.
>>
>> Are the messages from the reconnected Producer B being blackholed, or
>> just not being consumed by consumer B? What we've seen is that
>> Producer B (in this case) recreates the queue, but consumer B (still
>> connected) isn't considered subscribed to it, so the messages just
>> build up.
>
> Ah, sorry. My test wasn't quite correct because the node was not fully
> halted (I was stopping with stop_app instead of stop).  When stopping
> with "rabbitmqctl stop" the behavior is as you've describe above: when
> a new publisher connects the queue is recreated on the connected
> broker, but publishers that are already connected are in a bad state:
> they lose their messages.  The same is true for consumers - they stop
> receiving messages until reconnected.
>
> -Drew
>