[rabbitmq-discuss] Often failed to produce message due to TIMEOUT WAITING FOR ACK

[Wong Kam Hoong]

Hi  Matthias,

What's the general health of your rabbit when this happens?
The RabbitMQ nodes seems fine during the incident happened:

*Master Node Usage*
Memory : 300M
CPU %  : Average 30%

*Slave Node Usage*
Memory : 130M
CPU %  : Average 30%

Plenty of free memory and disk?
Yes, there are plenty of free memory and disk space in both servers.

Are you using mirrored/HA queues at all?
Yes, I'm using HA in all queues.

Could there possibly be a spike, i.e. lots of messages getting published in
a short space of time?
No, I checked there are only few messages during that time, e.g. 2012-09-26
03:28:32 => 4 messages

No strange errors in the logs?
After the incident happened RabbitMQ did logged warning, e.g.

*RabbitMQ Log*
*
*
=WARNING REPORT==== 26-Sep-2012::03:29:25 ===
closing AMQP connection <0.8661.18> (192.168.0.100:46151 ->
192.168.0.100:5672):
connection_closed_abruptly

=INFO REPORT==== 26-Sep-2012::03:29:37 ===
accepting AMQP connection <0.8897.18> (192.168.0.100:43836 ->
192.168.0.100:5672)


Regards,
Wong

On Tue, Sep 25, 2012 at 2:32 PM, Matthias Radestock
<matthias at rabbitmq.com>wrote:

> Wong,
>
>
> On 25/09/12 03:46, Wong Kam Hoong wrote:
>
>> com.rabbitmq.client.**ShutdownSignalException: clean channel shutdown;
>> reason: #method<channel.close>(reply-**code=406, reply-text=TIMEOUT
>> WAITING FOR ACK, class-id=0, method-id=0)
>>          at
>> com.rabbitmq.client.impl.**ChannelN.waitForConfirms(**ChannelN.java:182)
>>
>> Based on the log, the problem seems like related to "*NACKS Received*"
>> and "*ChannelN*" code throw the TimeoutException with reply-code *406
>> *(PRECONDITION_FAILED).
>>
>
> I don't think it has anything to do with nacks - unless you are actually
> seeing some exception mentioning nacks.
>
> Looks like a straightforward timeout.
>
>  I check all RabbitMQ servers are still working fine and I believe the
>> time I set for *waitForConfirmsOrDie *(10000) should be sufficient enough.
>>
>
> 10s should indeed generally be long enough. But one can certainly envision
> scenarios where it won't be. What's the general health of your rabbit when
> this happens? Plenty of free memory and disk? No strange errors in the
> logs? Are you using mirrored/HA queues at all? Could there possibly be a
> spike, i.e. lots of messages getting published in a short space of time?
>
>
> Regards,
>
> Matthias.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120926/ebb8abc7/attachment.htm>