[rabbitmq-discuss] Why is RabbitMQ not persisting messages on a durable queue?

[Alexandru Scvorţov]

Hi,

A persistent message is one Rabbit will try to write to disk, but if the
server is killed before it gets the chance to do this, the message will
be lost.

There are a few ways to remedy this, either by using transactions or by
using confirms (see our recent blog posts).  That said, I don't know if
you can get Celery to use any of these mechanisms.

Cheers,
Alex

On Fri, Feb 18, 2011 at 01:49:46PM -0800, hekevintran wrote:
> It looks like the delivery mode is already set to 2:
> 
>     In [1]: from apps.test_app.tasks import add
> 
>     In [2]: add.delivery_mode
>     Out[2]: 2
> 
> This default cannot be changed in celery as far as I know.
> 
> On Feb 18, 1:40 pm, "Gavin M. Roy" <g... at myyearbook.com> wrote:
> > Not sure of the celery underpinnings for specifying persistent messages,
> > perhaps it's documented, but you need to set the message delivery_mode = 2
> > in the message properties when calling Basic.Publish.
> >
> >
> >
> > On Fri, Feb 18, 2011 at 4:19 PM, hekevintran <hekevint... at gmail.com> wrote:
> > > I am using RabbitMQ with Django through Celery. I am using the most
> > > basic setup:
> >
> > >    # RabbitMQ connection settings
> > >    BROKER_HOST = 'localhost'
> > >    BROKER_PORT = '5672'
> > >    BROKER_USER = 'guest'
> > >    BROKER_PASSWORD = 'guest'
> > >    BROKER_VHOST = '/'
> >
> > > I imported a Celery task and queued it to run one year later. From the
> > > iPython shell:
> >
> > >    In [1]: from apps.test_app.tasks import add
> >
> > >    In [2]: dt=datetime.datetime(2012, 2, 18, 10, 00)
> >
> > >    In [3]: add.apply_async((10, 6), eta=dt)
> > >    DEBUG:amqplib:Start from server, version: 8.0, properties:
> > > {u'information': 'Licensed under the MPL.  Seehttp://www.rabbitmq.com/',
> > > u'product': 'RabbitMQ', u'version': '2.2.0', u'copyright': 'Copyright
> > > (C) 2007-2010 LShift Ltd., Cohesive Financial Technologies LLC., and
> > > Rabbit Technologies Ltd.', u'platform': 'Erlang/OTP'}, mechanisms:
> > > ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
> > >    DEBUG:amqplib:Open OK! known_hosts []
> > >    DEBUG:amqplib:using channel_id: 1
> > >    DEBUG:amqplib:Channel open
> > >    DEBUG:amqplib:Closed channel #1
> > >    Out[3]: <AsyncResult: cfc507a1-175f-438e-acea-8c989a120ab3>
> >
> > > RabbitMQ received this message in the celery queue:
> >
> > >    $  rabbitmqctl list_queues name messages durable
> > >    Listing queues ...
> > >    KTMacBook.local.celeryd.pidbox      0       false
> > >    celery      1       true
> > >    celeryctl_KTMacBook.local   0       true
> > >    ...done.
> >
> > > I then killed RabbitMQ by hitting control-C followed by 'a' to abort.
> > > When I start the server again and check it with rabbitmqctl, it says
> > > that there are no messages in the celery queue:
> >
> > >    $  rabbitmqctl list_queues name messages durable
> > >    Listing queues ...
> > >    celery      0       true
> > >    celeryctl_KTMacBook.local   0       true
> > >    ...done.
> >
> > > The celery queue was durable. Why were the messages not persisted?
> > > What do I need to do to make the messages persistent?
> > > _______________________________________________
> > > rabbitmq-discuss mailing list
> > > rabbitmq-disc... at lists.rabbitmq.com
> > >https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> >
> >
> >
> > _______________________________________________
> > rabbitmq-discuss mailing list
> > rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss