[rabbitmq-discuss] Mgmt plug-in on 2.3.0

[Matthew Sackman]

Hi Paul,

On Wed, Feb 02, 2011 at 07:11:11PM -0500, Paul Pearcy wrote:
>   Many thanks for the quick response. You can take a look at the log files here:
> http://dl.dropbox.com/u/12095883/rabbit%40dm-essearchd101.log.1
> http://dl.dropbox.com/u/12095883/rabbit%40dm-essearchd101-sasl.log.1

** Reason for termination == 
** {{badmatch,false},
    [{rabbit_amqqueue_process,assert_invariant,1},
     {rabbit_amqqueue_process,noreply,1},
     {gen_server2,handle_msg,2},
     {proc_lib,wake_up,3}]}

Wow. Something's _very_ broken.

It crashed, with a transaction in flight, the queue state does think
it's empty, which is odd because the failure of assert_invariant is
requires the queue to not be empty,... oh, I see what it is. The
sync_timeout can cause messages to become eligable for delivery, but we
don't then immediately call run_message_queue. Hence the queue can reach
a situation where it has messages eligable for delivery, with consumers
available to receive said messages, and yet we don't push those msgs out
asap.

It's a bug introduced by the addition of confirms, but it can cause this
crash even if you're just using transactions. It's a race condition,
which is why we've not seen it in testing. The solution is to not use
2.3 - go back to 2.2 for the time being. The only case where 2.3 is safe
is when you're using neither transactions nor confirms and you might
need infinite memory too. This might be interpreted as general advice to
avoid 2.3 for the time being, I'm afraid.

> When I initially brought things up, I had a 2.2 Java client connecting which may have caused some of the errors. I updated my app to the 2.3 client and received what appeared to be the same behavior. 

No - no errors should be caused by using a 2.2 client with the 2.3
broker - and that's not what's caused this problem.

The year of the Rabbit bodes well.

Matthew