[rabbitmq-discuss] Pika Client starts consuming before calling start_consuming

[Laing, Michael]

You say you want a scalable solution. Therefore with pika you should use a
high-performance asynch connector, i.e. tornado in pika 9.13. So start
there and follow the examples of using asynch connections - forget the
blocking connection.

Then you could try this:

To do a simple priority queue I set up 2 queues and bind to both. Usually I
have one queue that gets big bursts of messages and another, the priority
one that only gets a few at a time. In this case I just set prefetch_count
to 1 for both queues and let them interleave. I use acks always.

To give more 'preference' to the priority queue in the above case you can
increase its prefetch_count, increasing the likelihood that a priority
message will be ready when the event loop rolls around.

For even more preference you can 'rate limit' the non-priority queue. I do
this by registering a short timeout after processing each non-priority
message which does the ack in its callback.

There are much more complicated ways to do this. I prefer simple.

Good luck,

Michael

P.S. You can subscribe to a queue that is not currently bound to anything.
Once subscribed, you can bind the queue one or more times and catch each
bind_ok callback to know you are consuming. I'm not sure this is actually
useful in your case.


On Tue, Nov 26, 2013 at 1:25 AM, vadlamani maitreya <
maitreya_vvm at yahoo.co.in> wrote:

> Hello Everyone,
>
> I have a small problem with RMQ and pika client and would like to ask guyz
> here for some help. The situation is as below,
>
> I am using RabbitMQ with the pika clients in a scalable solution. The
> solution requires messaging with two priorities. So every worker (through
> pika client) gets messages from the HighPriority Queue first and then goes
> to the Low Priority queue if there are no messages left in the HP Queue. I
> decide to scale the number of workers according to the number of ready
> messages and the number of consumers attached to the queue. So the
> following is the workflow i follow to achieve the above.
>
> 1) Register using  basic_consume on the queue.
> 2) Do basic_get from HP Queue
> 3)If no messages in HP Queue do a basic_get from LP Queue.
>
> I donot execute start_consuming() so the basic_consume is just for the
> number of consumers count in the RMQ server.
>
> This logic works perfectly fine with the standard Java Client. But when i
> try to do the same with the Pika Client, the moment the basic_consume is
> called the data events are processed and the callback function is executed
> on the incoming message, eventhough the start_consuming() function is not
> called.
>
> So, a couple of questions here,
>
> 1. Is there a way to register as a consumer to a queue without actually
> starting to consume from the queue (even a hack into the code is fine)
> 2. Is it bug that basic_consume() starts consuming without actually
> calling start_consuming() (the actual blocking function), and should i
> raise an issue in github regarding this ?
>
> Last but not the least, the version of the rabbitMQ server is 3.1.2 and
> the pika client, i have tried both 0.9.6 and 0.9.13 with the above scenario.
>
> Thankyou everyone,
> Maitreya
>
>
>
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20131126/baf1318e/attachment.htm>