[rabbitmq-discuss] Channel crashes after basic.cancel_ok.

Wed May 7 22:28:43 BST 2008

BTW Edwin, I reposting this to the list, so it can be followed.

On 7 May 2008, at 21:50, Edwin Fine wrote:

> Thanks for the response, Ben. I will try to create a standalone  
> program that reproduces  the issue. I hope I can do it. The  
> hesitation comes because the circumstances under which this occurred  
> are:
> 50 queues on one <<direct>> exchange
Shouldn't be an issue.

> A producer that is pushing 150 messages/second to the queues, evenly  
> across the queue name

Shouldn't be an issue.
> Each queue is doing a basic.deliver to one of 50 Erlang consumer  
> processes (gen_event). The consumer processes all share the same  
> channel (which may not be a good idea, because when the channel  
> crashed, so did all 50 consumers. I wonder if I should have 1  
> channel for each consumer? Too heavy?).
Good question. The golden answer I guess is not too little, not too  
much ;-) Your mileage will vary.

If there is a bug, however, the channel process will bring all of the  
consumers that are linked to it. So from a *resiliency* perspective,  
you might want to have more than one channel process. Having said  
that, I would prefer to find out what the cause was and identify  
whether there is a bug that needs to fixed.

In terms of the difference this makes on overall throughput, I think  
the answer is that it depends. Using the network client you are  
receiving serially from a single client socket, so it may be 6 of one  
and half a dozen of the other. In the direct case, you may see more  
leverage effect here.

> Because the channel.flow command is not implemented, I am using  
> unsubscribe to try to do flow control, because I am finding that  
> RabbitMQ is dumping all the basic.deliver messages into the Erlang  
> message queue of each consumer gen_event process. This is totally  
> not what I want - I want the messages to stay persisted in Rabbit- 
> land until I can deliver them and tell Rabbit to release them.
At the moment, Rabbit is designed to deliver a message if it can.  
Control flow is more of a server implementation issue. Thoughts anyone?

> The reason I need flow control is because I am delivering messages  
> via HTTP/XML to a number of different Web sites. Sometimes a web  
> site may be down, often for hours, and I want to just be able to  
> pause the consumer (and producer) until messages start going through  
> again. Having the Erlang message queues filling up with thousands of  
> messages will eventually crash the VM due to OOM. I know I can poll  
> with basic.get, but the conundrum is that I want high performance  
> while the web sites are accepting input, and shut-off when they are  
> not. Any suggestions?

One strategy that springs to mind to turn the no_ack flag when sending  
the consume command to the broker. I will have to have a look to see  
what the behaviour of this, because I don't know off the top my head.  
Alternatively if you can detect the failure of a web site, can you  
actively cancel the subscription?

>
> So I may have to build a simulated version of the above to get the  
> problem to happen again. That's the hesitation - will it trigger the  
> error?
>

Let's hope so. If not, and if you feel comfortable with it, you send  
you raw code base.

> Anyway, I am probably not doing this the right way, but I have no  
> expertise in AMQP. My knowledge is of MQSeries a few years back.

I don't think this will be a problem :-) AMQP is not MQ :-)

HTH,

Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20080507/c166d442/attachment.htm 