[rabbitmq-discuss] EndOfStreamException after 15 minutes of inactivity

Thu Jun 7 11:46:22 BST 2012

On 7 Jun 2012, at 10:08, Tim Watson wrote:

> On 06/06/12 22:05, Tim Watson wrote:
>> On 06/06/2012 16:18, Walter Moore wrote:
>>> Hi Tim,
>>> 
>>> I am not explicitly setting the TTL when creating the queue. Looking
>>> over the TTL docs, it looks like declaring the queue with “x-expires”
>>> set to a very large (long-long-int) number would effectively fix the
>>> issue. I’ll give that a shot this morning.
>>> 
>> 
>> The x-expires extension controls queue lifetime, whereas x-message-ttl controls the dwell time for messages before they get discarded. I'm not convinced this has anything to do with the behaviour you specified, as if it isn't currently set then nothing should be causing the queue to be removed (and as you've not said otherwise, I'm assuming that the queue survives this disconnection of the consumer).
>> 
>>> I have confirmed via logging that Heartbeat = 0 in the .Net client
>>> connection.
>>> 
>> 
>> Ok - that seemed like a logical candidate but perhaps it's a red herring. It does 'sound' like some kind of timeout/expiry is happening, but if there's no heartbeat set then perhaps that's not the case.
>> 
> 
> Hi Walter,
> 
> Can you please try *enabling* heartbeats - as their whole purpose is to ensure that the connection stays alive for long enough - and let us know if that resolves the problem?
> 

Actually I've mis-stated that. AMQP heartbeats are also used by the broker to deal with 'misbehaving' clients, but either way, they should help with your situation I hope.

> Cheers,
> Tim
> 
>>> It is probably also worth mentioning that this is occurring on an SSL
>>> connection. The exchange is manually configured via the Management
>>> plugin web interface using the following parameters:
>>> 
>>> Type: topic
>>> 
>>> Durable: true
>>> 
>>> The queue is created with the following parameters:
>>> 
>>> QueueName: <ApplicationName>.<ProcessName>.<ConsumerName>
>>> 
>>> Durable: true
>>> 
>>> Exclusive: false
>>> 
>>> AutoDelete: false
>>> 
>>> Arguments: null
>>> 
>>> The queue is bound to a number of filters of the form
>>> <ApplicationName>.<ProcessName>.<ProducerName> which can be on several
>>> different exchanges. The queue name, routing keys, and key bindings are
>>> all managed with an internally developed toolset.
>>> 
>> 
>> I'd be surprised to find that any of those (latter) things is at the root of this behaviour.
>> 
>>> Thank you,
>>> 
>>> *Description: Description: Description: INTLFCStone*
>>> 
>>> **
>>> 
>>> *Walter Moore*
>>> 
>>> Head of Transactional Data Exchange
>>> 
>>> INTL FCStone Inc
>>> 
>>> 230 S. LaSalle Street
>>> 
>>> Suite 10-500
>>> 
>>> Chicago IL 60604
>>> 
>>> *Direct 312-789-2538*
>>> 
>>> *From:*Tim Watson [mailto:tim at rabbitmq.com]
>>> *Sent:* Wednesday, June 06, 2012 2:32 AM
>>> *To:* Walter Moore
>>> *Cc:* rabbitmq-discuss at lists.rabbitmq.com
>>> *Subject:* Re: [rabbitmq-discuss] EndOfStreamException after 15 minutes
>>> of inactivity
>>> 
>>> On 06/06/12 07:59, Tim Watson wrote:
>>> 
>>> On 05/06/12 21:24, Walter Moore wrote:
>>> 
>>> Hello,
>>> 
>>> I have a long running, durable, non-auto-delete, non-exclusive, RabbitMq
>>> consumer (.Net client) that works perfectly unless a 15 minute (+-
>>> approximately 3 seconds) period of inactivity occurs. After 15 minutes
>>> of messaging inactivity a “System.IO.EndOfStreamException: SharedQueue
>>> closed” exception is thrown (and continues to be thrown for each
>>> subsequent call to <model>.BasicConsume()). Is this the expected
>>> behavior? I am using NoAck = true for each dequeue..
>>> 
>>> Hi Walter,
>>> 
>>> When you say 'non-auto-delete' do you mean to say there is not TTL
>>> specified on the messages?
>>> I also wondered if you have AMQP heartbeats enabled - see
>>> http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2012-May/020236.html for
>>> a discussion of this - as that could cause a connection to be terminated
>>> after a specified period of inactivity.
>>> 
>>> Looking further at the client code, I can see that the heartbeat is
>>> implemented on the client as well as the server. If this times out, we
>>> do indeed get an EndOfStreamException (with a message like "Heartbeat
>>> missing with heartbeat == <n> seconds") - see
>>> http://hg.rabbitmq.com/rabbitmq-dotnet-client/file/default/projects/client/RabbitMQ.Client/src/client/impl/ConnectionBase.cs#l541 
>>> for the details.
>>> 
>>> Looking at the SharedQueue class, I can see that once closed, it will
>>> continuously throw this exception whenever you try to dequeue. So this
>>> looks like a possible cause to me. Can you verify whether or not you've
>>> got AMQP heartbeat timeouts enabled?
>>> 
>>> Cheers,
>>> Tim
>>> 
>>> 
>>> I'll have a look in the .NET client code, as this doesn't seem like the
>>> exception class I'd expect to be thrown in a heartbeat timeout
>>> situation, but I thought it was worth mentioning just in case.
>>> 
>>> If possible, can you tell us exactly (scratching any private data such
>>> as the broker, queue and/or exchange names of course) what parameters
>>> you're passing to declare the queue and exchange(s) as well as the
>>> consumer set up?
>>> 
>>> Cheers,
>>> Tim
>>> 
>>> Thank you,
>>> 
>>> *Description: Description: Description: INTLFCStone*
>>> 
>>> **
>>> 
>>> *Walter Moore*
>>> 
>>> Head of Transactional Data Exchange
>>> 
>>> INTL FCStone Inc
>>> 
>>> 230 S. LaSalle Street
>>> 
>>> Suite 10-500
>>> 
>>> Chicago IL 60604
>>> 
>>> *Direct 312-789-2538*
>>> 
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> 
>>> rabbitmq-discuss mailing list
>>> 
>>> rabbitmq-discuss at lists.rabbitmq.com <mailto:rabbitmq-discuss at lists.rabbitmq.com>
>>> 
>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>> 
>>> 
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> 
>>> rabbitmq-discuss mailing list
>>> 
>>> rabbitmq-discuss at lists.rabbitmq.com <mailto:rabbitmq-discuss at lists.rabbitmq.com>
>>> 
>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>> 
>> 
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> 
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss