[rabbitmq-discuss] Ending a computation when the queue runs out of items

Mon Jun 25 12:27:28 BST 2012

Hi,

I'm wanted to use rabbitMQ in a simple scheme to multithread a
computation in python and I'm having an architecture problem. I wanted
to have a process filling the queue and a another that threw a few
threads consuming it. Than I wanter the following to happen: when the
process that fills the queue is done and the queue is empty again,
each of the threads should stop.

What I've done to this point is this: (code here: https://gist.github.com/2988023)

I have three classes (I'm using pika):
    (a) Populator that has a 'send' method that sends serialized
objects to the queue
    (b) Consumer that has a 'setup' method where I define a function
that will process each item with the basic_consume() method from the
pika channel and a method 'run' that calls the start_consuming()
method from the channel.
    (c) ConsumerPool that creates a few Consumers and a few Threads,
each Thread calling the run method from one of the consumers.

It works quite fine if I expect that my queue will be constantly
filled with stuff. But this is not what I'm expecting: I want the
process to stop when I run out of items and I don't quite see how I
can do it. I tried to call stop_consuming() but I don't think I
actually understand when or how to call it. I must stop the Threads as
well it seems.

Any help will be appreciated.

Thanks!