[rabbitmq-discuss] Consumer stop to receive messages but continue listening queue problem.

[Gustavo Aquino]

Hi Matthew,


On Fri, Mar 12, 2010 at 9:24 AM, Matthew Sackman <matthew at lshift.net> wrote:

> Hi Gustavo,
>
> On Fri, Mar 12, 2010 at 12:00:52AM -0300, Gustavo Aquino wrote:
> > I'm running my RabbitMQ in one server, and have 2 producers in other
> server
> > posting ~2000 messages /s , and have 4 consumers in one server and other
> 4
> > in other.
>
> Ok, so we have 4kHz in, in total? Are the 8 consumers all off the same
> queue, or do they all have their own queues? If the former, then the sum
> out should also be 4kHz, whereas the latter would suggest the sum out
> should be 8 * 4kHz = 32kHz. That's faster than I've seen Rabbit running
> before, but then again you do have very meaty hardware.
>

Sorry but I don't understand why we have 4kHz in total.

All 8 consumers are listening the same queue. The hardware of Rabbit

>
> > When I start the test all running fine.. but my 8 consumers not consuming
> > the queue in same speed that we are posting, ok it is the proposal of
> test,
> > all 8 consumers are working harder consuming ~1200 /s when my doubt
> happen,
> > all the time that I run this test it happen, when my queue have a size
> about
> > 200.000 messages all consumers at one of this two consume's machine stop
> to
> > receive messages, all of them, and if I restart all consumers in this
> > machine anyone get any messages from Rabbit,all are connected but don't
> get
> > any message, now if instead of restart this consumers I stop the other
> four
> > consumers in the other machine he come back to receive messages...
> >
> > Anybody here have any feelings about what is happen ?
>
> That's very interesting. You don't say which client you're using though.
> Also, when this happens, what does Rabbit record in the logs?
>

I'm using Rabbit client in Java to consume and .Net Rabbit client to produce
the messages.



>
> > My Rabbit server have 20Gb of memory and 4 processors P6 and when this
> drop
> > happen Rabbit are consuming 80% of Proc and 20% of Memory.
>
> I have never used Power6 machines, and I have no idea how well Erlang
> (or Rabbit) runs on it, I'm afraid.
>

This is the point, I think that we will be the first in this environment, I
tried to found some experiences in Internet but don't found nothing. I'm
using Erlang R12 yes I know it's very old, but RH have only this version
compiled for ppc, I'm trying to install R13B but having some compiling
problems with ncourses, but It's other e-mail :).

We will do more some tests enabling more 2 processors, and will run
Erlang/Rabbit over 6 processors p6 4.2Ghz for long stress times.We have
severals concerns to be the first to use this environments.


> > My message size is about 180~200Kbytes
>
> Those are pretty big. If my maths above is right, that means that you're
> trying to push (4+32)*1000*200*1024 = 7372,800,000 bytes / sec, which is
> 6.8GB/s or 55Gb/s. That's enourmous. I suspect maybe my maths is wrong
> and you actually have all 8 consumers bound to the same 1 queue?
>

Sorry, I put wrong values, my fault, My message size have 180 Bytes ~
1Kbytes. And all in the same queue. Another information Rabbit always crash
with queue size get Rabbit's memory limit I try to found but don't see a way
to configure queue to drop messages after a specific size or memory limit.
It's other problem.


>
> Matthew
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100312/99690c60/attachment.htm