[rabbitmq-discuss] messages get stuck when using rabbitmq on windows, but not on osx or ubuntu

[Matthias Radestock]

Matthew,

Matthew Scott wrote:
> I'm experiencing a problem with messages getting "stalled" when serving 
> RabbitMQ with Windows XP.
> 
> The context is a distributed job processing system, wherein some message 
> payloads can be around 250-500 kB.  The clients connecting to the broker 
> are several programs written in Python using the txAMQP library and the 
> AMQP 0-8 protocol.  (*)
> 
> We can submit one job in our system, which results in several small 
> messages, and four larger messages being routed through RabbitMQ with 
> the size ranges noted above, and this succeeds.  However, when we submit 
> an identical job, one message with a larger payload is successfully 
> routed, but the second one is not.  I can see that it successfully 
> reaches the broker but never reaches the intended destination, because I 
> get this output:
> 
> C:\rabbitmq_server-1.7.1\sbin>rabbitmqctl list_connections send_oct 
> send_pend
> Listing connections ...
> 594     0
> 1183650 0
> 236655  0
> 616     0
> 610     0
> 2492155 256809    <-- here is the stuck message
> 616     0
> 616     0
> 367     0

On the surface this does look like a client problem, namely that for
some reason the client has stopped reading data from the socket, causing
the tcp flow control to stall the connection and thus data backing up in
the server's tcp buffers.

It could also be some weird networking problem in the O/S.

I suggest sticking the AMQP tracer
(http://www.rabbitmq.com/examples.html#tracer) inbetween the broker and
the client, running it on either the client or server machine (it's
worth trying both options). That will tell you what's going on at the
AMQP protocol level. If you need help interpreting the output, feel free
to post it here.

It's possible that inserting the tracer alters the behaviour, and
"fixes" the problem. If so then a lower-level tool to capture the
network traffic, such as wireshark, is your best bet.


Regards,

Matthias.