[rabbitmq-discuss] Pika Client starts consuming before calling start_consuming

[Laing, Michael]

Then you could add a mutex in addition to rate limiting and prefetch
twiddling to insure that batch messages are not processed as long as high
priority messages are available for processing.

Good luck,

Michael


On Mon, Dec 2, 2013 at 5:25 AM, vadlamani maitreya <maitreya_vvm at yahoo.co.in
> wrote:

> Hi Michael,
>
> Thankyou for the response. The application we built infact supports only
> two priorities, ondemand and batch. The problem here is once the ondemand
> message is in queue, it must be processed irrespective of any number of
> batch messages. That is the reason for this weird logic we had to employ.
> For this usecase, I dont think the rate limiting and prefetch count
> variations would actual work. But I will try to play with the asynch
> connections for sure.
>
> regards,
> Maitreya
>
>
>
>
>
>   On Tuesday, 26 November 2013 8:32 PM, "Laing, Michael" <
> michael.laing at nytimes.com> wrote:
>  You say you want a scalable solution. Therefore with pika you should use
> a high-performance asynch connector, i.e. tornado in pika 9.13. So start
> there and follow the examples of using asynch connections - forget the
> blocking connection.
>
> Then you could try this:
>
> To do a simple priority queue I set up 2 queues and bind to both. Usually
> I have one queue that gets big bursts of messages and another, the priority
> one that only gets a few at a time. In this case I just set prefetch_count
> to 1 for both queues and let them interleave. I use acks always.
>
> To give more 'preference' to the priority queue in the above case you can
> increase its prefetch_count, increasing the likelihood that a priority
> message will be ready when the event loop rolls around.
>
> For even more preference you can 'rate limit' the non-priority queue. I do
> this by registering a short timeout after processing each non-priority
> message which does the ack in its callback.
>
> There are much more complicated ways to do this. I prefer simple.
>
> Good luck,
>
> Michael
>
> P.S. You can subscribe to a queue that is not currently bound to anything.
> Once subscribed, you can bind the queue one or more times and catch each
> bind_ok callback to know you are consuming. I'm not sure this is actually
> useful in your case.
>
>
> On Tue, Nov 26, 2013 at 1:25 AM, vadlamani maitreya <
> maitreya_vvm at yahoo.co.in> wrote:
>
> Hello Everyone,
>
> I have a small problem with RMQ and pika client and would like to ask guyz
> here for some help. The situation is as below,
>
> I am using RabbitMQ with the pika clients in a scalable solution. The
> solution requires messaging with two priorities. So every worker (through
> pika client) gets messages from the HighPriority Queue first and then goes
> to the Low Priority queue if there are no messages left in the HP Queue. I
> decide to scale the number of workers according to the number of ready
> messages and the number of consumers attached to the queue. So the
> following is the workflow i follow to achieve the above.
>
> 1) Register using  basic_consume on the queue.
> 2) Do basic_get from HP Queue
> 3)If no messages in HP Queue do a basic_get from LP Queue.
>
> I donot execute start_consuming() so the basic_consume is just for the
> number of consumers count in the RMQ server.
>
> This logic works perfectly fine with the standard Java Client. But when i
> try to do the same with the Pika Client, the moment the basic_consume is
> called the data events are processed and the callback function is executed
> on the incoming message, eventhough the start_consuming() function is not
> called.
>
> So, a couple of questions here,
>
> 1. Is there a way to register as a consumer to a queue without actually
> starting to consume from the queue (even a hack into the code is fine)
> 2. Is it bug that basic_consume() starts consuming without actually
> calling start_consuming() (the actual blocking function), and should i
> raise an issue in github regarding this ?
>
> Last but not the least, the version of the rabbitMQ server is 3.1.2 and
> the pika client, i have tried both 0.9.6 and 0.9.13 with the above scenario.
>
> Thankyou everyone,
> Maitreya
>
>
>
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20131202/f4eebd62/attachment.html>