[rabbitmq-discuss] Messages not showing up

[Vukasin Toroman]

Hi everyone,

we have been using rabbitmq for a while now to route documents through a processing workflow. each step is represented by one or more processes working on one queue. the messages are small (maybe a few k) but there are quite a few of them (multiple thousand per second). This has been working fine for a while but then we have noticed that the rates (each step of the processing workflow delivers stats) do not add up. namely we seem to be losing messages (around 200 messages/sec) and this is quite discerning. The reason is that these messages do not show up in the queue and are also not visible as being recieved and un-acked. we are using non-durable queues and a combination of java and python processing clients. 

oh yes, this only happens after the system has been running for a while and a traffic spike has occurred. the rabbitmq process also seems to be gaining memory. 

we are using 2.3.1-1 version of rabbitmq-server on ubuntu 10.4 (amd64)

thank you for you help,
vukasin