[rabbitmq-discuss] Why Celery?

Mon May 21 13:16:04 BST 2012

On 20 May 2012, at 19:07, mrtn wrote:

> 
> I finished reading through the rabbitmq tutorials here, and later found out people very often associate celery with rabbitmq.
> 
> From the tutorials, it seems that we don't have to use celery to process the messages in the queues (i.e. emptying them). However, someone on SO says that: 'If you don't have a celeryd process running, you'll just be adding tasks to the queue, but never emptying it',

That seems to be taken out of context, it's probably not an answer to the question
you're asking.

> which I doubt is true. In the tutorials, a consumer/worker program listens to the queue, and process messages as they arrive, without using celery.
> 
> So my questions: what's the added-value celery brings to rabbitmq, and what are the common use cases of it?

Celery is a system that responds to messages by calling Python functions,
it is heavily biased around the concept of tasks and workflows, and so may
not be suitable for all messaging scenarios.

Writing a consumer is not hard, writing one that keeps running is.
The celery worker is heavily used in production and supports many features:
http://docs.celeryproject.org/en/latest/userguide/workers.html

And then the system must also be maintained, and your sysadmin
will probably hug you for using something standard that is
documented and has a supporting community. 

The next version 2.6 will extend on the concept of subtasks, but
the documentation is only a rough draft:
http://ask.github.com/celery/getting-started/next-steps.html#subtasks

See the User Guide:
http://docs.celeryproject.org/en/latest/userguide/index.html