[rabbitmq-discuss] Cluster Pathology

[Drew Smathers]

On Wed, Feb 11, 2009 at 6:34 PM, Jason J. W. Williams
<jasonjwwilliams at gmail.com> wrote:
> Drew,
>
>> Now, if node A is taken down, producer A of course can't produce
>> messages due to socket errors, but producer B continues producing
>> messages not knowing they actually are just getting dropped with no
>> queue to route to (since Q is on node A).  What's even more surprising
>> to me on this matter is that if producer B restarts and publishes
>> messages, the messages are still blackholed.
>
> Are the messages from the reconnected Producer B being blackholed, or
> just not being consumed by consumer B? What we've seen is that
> Producer B (in this case) recreates the queue, but consumer B (still
> connected) isn't considered subscribed to it, so the messages just
> build up.

Ah, sorry. My test wasn't quite correct because the node was not fully
halted (I was stopping with stop_app instead of stop).  When stopping
with "rabbitmqctl stop" the behavior is as you've describe above: when
a new publisher connects the queue is recreated on the connected
broker, but publishers that are already connected are in a bad state:
they lose their messages.  The same is true for consumers - they stop
receiving messages until reconnected.

-Drew