[rabbitmq-discuss] Why is RabbitMQ not persisting messages on a durable queue?

[Alexandru Scvortov]

Also, are you running the latest version of the broker?  There was a bug
in versions of Rabbit before 2.3.0 that made the above problem worse.
RabbitMQ 2.3.1 can still lose messages if it's killed quickly enough,
but it should happen less often.

Cheers,
Alex

On Sat, Feb 19, 2011 at 12:44:17AM +0200, Alexandru Scvorţov wrote:
> Hi,
> 
> A persistent message is one Rabbit will try to write to disk, but if the
> server is killed before it gets the chance to do this, the message will
> be lost.
> 
> There are a few ways to remedy this, either by using transactions or by
> using confirms (see our recent blog posts).  That said, I don't know if
> you can get Celery to use any of these mechanisms.
> 
> Cheers,
> Alex
> 
> On Fri, Feb 18, 2011 at 01:49:46PM -0800, hekevintran wrote:
> > It looks like the delivery mode is already set to 2:
> > 
> >     In [1]: from apps.test_app.tasks import add
> > 
> >     In [2]: add.delivery_mode
> >     Out[2]: 2
> > 
> > This default cannot be changed in celery as far as I know.
> > 
> > On Feb 18, 1:40 pm, "Gavin M. Roy" <g... at myyearbook.com> wrote:
> > > Not sure of the celery underpinnings for specifying persistent messages,
> > > perhaps it's documented, but you need to set the message delivery_mode = 2
> > > in the message properties when calling Basic.Publish.
> > >
> > >
> > >
> > > On Fri, Feb 18, 2011 at 4:19 PM, hekevintran <hekevint... at gmail.com> wrote:
> > > > I am using RabbitMQ with Django through Celery. I am using the most
> > > > basic setup:
> > >
> > > >    # RabbitMQ connection settings
> > > >    BROKER_HOST = 'localhost'
> > > >    BROKER_PORT = '5672'
> > > >    BROKER_USER = 'guest'
> > > >    BROKER_PASSWORD = 'guest'
> > > >    BROKER_VHOST = '/'
> > >
> > > > I imported a Celery task and queued it to run one year later. From the
> > > > iPython shell:
> > >
> > > >    In [1]: from apps.test_app.tasks import add
> > >
> > > >    In [2]: dt=datetime.datetime(2012, 2, 18, 10, 00)
> > >
> > > >    In [3]: add.apply_async((10, 6), eta=dt)
> > > >    DEBUG:amqplib:Start from server, version: 8.0, properties:
> > > > {u'information': 'Licensed under the MPL.  Seehttp://www.rabbitmq.com/',
> > > > u'product': 'RabbitMQ', u'version': '2.2.0', u'copyright': 'Copyright
> > > > (C) 2007-2010 LShift Ltd., Cohesive Financial Technologies LLC., and
> > > > Rabbit Technologies Ltd.', u'platform': 'Erlang/OTP'}, mechanisms:
> > > > ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
> > > >    DEBUG:amqplib:Open OK! known_hosts []
> > > >    DEBUG:amqplib:using channel_id: 1
> > > >    DEBUG:amqplib:Channel open
> > > >    DEBUG:amqplib:Closed channel #1
> > > >    Out[3]: <AsyncResult: cfc507a1-175f-438e-acea-8c989a120ab3>
> > >
> > > > RabbitMQ received this message in the celery queue:
> > >
> > > >    $  rabbitmqctl list_queues name messages durable
> > > >    Listing queues ...
> > > >    KTMacBook.local.celeryd.pidbox      0       false
> > > >    celery      1       true
> > > >    celeryctl_KTMacBook.local   0       true
> > > >    ...done.
> > >
> > > > I then killed RabbitMQ by hitting control-C followed by 'a' to abort.
> > > > When I start the server again and check it with rabbitmqctl, it says
> > > > that there are no messages in the celery queue:
> > >
> > > >    $  rabbitmqctl list_queues name messages durable
> > > >    Listing queues ...
> > > >    celery      0       true
> > > >    celeryctl_KTMacBook.local   0       true
> > > >    ...done.
> > >
> > > > The celery queue was durable. Why were the messages not persisted?
> > > > What do I need to do to make the messages persistent?
> > > > _______________________________________________
> > > > rabbitmq-discuss mailing list
> > > > rabbitmq-disc... at lists.rabbitmq.com
> > > >https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> > >
> > >
> > >
> > > _______________________________________________
> > > rabbitmq-discuss mailing list
> > > rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> > _______________________________________________
> > rabbitmq-discuss mailing list
> > rabbitmq-discuss at lists.rabbitmq.com
> > https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss