[rabbitmq-discuss] Unexpected server crash and .NET API: disconnect is not detected?

Thu Mar 31 10:38:31 BST 2011

Hi Emile,

I'm using a class based on Subscription (the core functionality is the same
as Subscription in v2.2.0 and the only differences are that I have new
methods and made public a pair of private properties) called from a
RpcServer class (which I also made my own modifications but calls my custom
Subscription class the same way as the v2.2.0 provided classes)

I can reproduce the issue (but I'm not sure if it is a expected behavior) by
doing the following steps:

1. Send  messages to a queue
2. Start a consumer application
3. Disable the network connection in order to simulate an unexpected crash.

What I see is that the my custom RpcServer class which recover  the messages
keep looping with the foreach although there connection is closed.

public void MainLoop()
{

            foreach (BasicDeliverEventArgs evt in m_subscription)
            {
                    ProcessRequest(evt);
            }

 }

In my custom Subscription class I see that the method Next is returning all
events even if the consumer is not running and no exception is launched when
is called by foreach loop.
(note: Next() is identical to Subscription class in v2.2.0 because I didn't
modiy it)

        public BasicDeliverEventArgs Next()
        {
            try
            {
                QueueingBasicConsumer consumer = m_consumer;
                if (consumer == null)
                {
                    // Closed!
                    m_latestEvent = null;
                }
                else
                {

                    m_latestEvent =
(BasicDeliverEventArgs)consumer.Queue.Dequeue();
                }
            }
            catch (EndOfStreamException)
            {
                m_latestEvent = null;
            }
            return m_latestEvent;
        }

My question is: should consumer.Queue.Dequeue() raise an exception when
"consumer" property IsRunning is false?
Currently this is not happening.

In my code a workaround could be to check consumer.IsRunning and assign
m_lastestEvent to null when if it is "false" and in MainLoop handle that or
to launch an excepcion which would cause MainLoop to end.

Any advice would be appreaciated.

Thank you in advance.

Regards,

Alfonso

2011/3/30 Emile Joubert <emile at rabbitmq.com>

> Hi Alfonso,
>
>
> On 30/03/11 15:43, Alfonso wrote:
>
>> Hi,
>>
>> This is a difficult one: I've observed a couple of times that my
>> consumer (.NET API 2.2.0) doesn't notice when the server that is
>> running RabbitMQ (2.2.0) crashes.
>> Please note that I mean server crash -unexpected shutdown, OS crash,
>> etc.- not RabbitMQ crash.
>>
>> My application is consuming from a queue using "foreach" to get the
>> messages and I has an event handler attached to the onShutdown event.
>>
>
> I'm not able to reproduce this using version 2.2.0 of the broker and the
> .NET client, implementing the the Subscription pattern. A connectivity
> problem immediately causes an OperationInterruptedException to be thrown
> from ConnectionBase.Dispose().
>
>
>  Time ago disconnection tests consisting of disconnecting the consumer
>> from the web admin console were made and the event handler was fired
>> always. So disconnections are caught and logged properly.
>>
>> The problem is that the consumer got stuck when the server crashed and
>> it didn't notice that the connection was gone (due to a server crash)
>> so it was impossible to log a disconnection error.
>> In the code there is a while loop for reconnecting when the connection
>> fails but didn't loop because my guess is that the code were stuck in
>> the "foreach".
>>
>> Am I missing something or this could be and issue?
>>
>
> Is it possible that you are preventing an exception from the socket from
> affecting the Subscription? Posting a minimal piece of code for
> demonstration may be useful.
>
>
>
> Regards
>
> Emile
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110331/1247d780/attachment.htm>