[rabbitmq-discuss] Why is RabbitMQ not persisting messages on a durable queue?

Sat Feb 19 21:19:43 GMT 2011

I was running version 2.2.0. Upgrading to 2.3.1 did the trick. Thanks!

Out of curiosity what was the bug that caused persistence to fail?

On Feb 18, 2:55 pm, Alexandru Scvortov <alexan... at rabbitmq.com> wrote:
> Also, are you running the latest version of the broker?  There was a bug
> in versions of Rabbit before 2.3.0 that made the above problem worse.
> RabbitMQ 2.3.1 can still lose messages if it's killed quickly enough,
> but it should happen less often.
>
> Cheers,
> Alex
>
>
>
>
>
> On Sat, Feb 19, 2011 at 12:44:17AM +0200, Alexandru Scvorţov wrote:
> > Hi,
>
> > A persistent message is one Rabbit will try to write to disk, but if the
> > server is killed before it gets the chance to do this, the message will
> > be lost.
>
> > There are a few ways to remedy this, either by using transactions or by
> > using confirms (see our recent blog posts).  That said, I don't know if
> > you can get Celery to use any of these mechanisms.
>
> > Cheers,
> > Alex
>
> > On Fri, Feb 18, 2011 at 01:49:46PM -0800, hekevintran wrote:
> > > It looks like the delivery mode is already set to 2:
>
> > >     In [1]: from apps.test_app.tasks import add
>
> > >     In [2]: add.delivery_mode
> > >     Out[2]: 2
>
> > > This default cannot be changed in celery as far as I know.
>
> > > On Feb 18, 1:40 pm, "Gavin M. Roy" <g... at myyearbook.com> wrote:
> > > > Not sure of the celery underpinnings for specifying persistent messages,
> > > > perhaps it's documented, but you need to set the message delivery_mode = 2
> > > > in the message properties when calling Basic.Publish.
>
> > > > On Fri, Feb 18, 2011 at 4:19 PM, hekevintran <hekevint... at gmail.com> wrote:
> > > > > I am using RabbitMQ with Django through Celery. I am using the most
> > > > > basic setup:
>
> > > > >    # RabbitMQ connection settings
> > > > >    BROKER_HOST = 'localhost'
> > > > >    BROKER_PORT = '5672'
> > > > >    BROKER_USER = 'guest'
> > > > >    BROKER_PASSWORD = 'guest'
> > > > >    BROKER_VHOST = '/'
>
> > > > > I imported a Celery task and queued it to run one year later. From the
> > > > > iPython shell:
>
> > > > >    In [1]: from apps.test_app.tasks import add
>
> > > > >    In [2]: dt=datetime.datetime(2012, 2, 18, 10, 00)
>
> > > > >    In [3]: add.apply_async((10, 6), eta=dt)
> > > > >    DEBUG:amqplib:Start from server, version: 8.0, properties:
> > > > > {u'information': 'Licensed under the MPL.  Seehttp://www.rabbitmq.com/',
> > > > > u'product': 'RabbitMQ', u'version': '2.2.0', u'copyright': 'Copyright
> > > > > (C) 2007-2010 LShift Ltd., Cohesive Financial Technologies LLC., and
> > > > > Rabbit Technologies Ltd.', u'platform': 'Erlang/OTP'}, mechanisms:
> > > > > ['PLAIN', 'AMQPLAIN'], locales: ['en_US']
> > > > >    DEBUG:amqplib:Open OK! known_hosts []
> > > > >    DEBUG:amqplib:using channel_id: 1
> > > > >    DEBUG:amqplib:Channel open
> > > > >    DEBUG:amqplib:Closed channel #1
> > > > >    Out[3]: <AsyncResult: cfc507a1-175f-438e-acea-8c989a120ab3>
>
> > > > > RabbitMQ received this message in the celery queue:
>
> > > > >    $  rabbitmqctl list_queues name messages durable
> > > > >    Listing queues ...
> > > > >    KTMacBook.local.celeryd.pidbox      0       false
> > > > >    celery      1       true
> > > > >    celeryctl_KTMacBook.local   0       true
> > > > >    ...done.
>
> > > > > I then killed RabbitMQ by hitting control-C followed by 'a' to abort.
> > > > > When I start the server again and check it with rabbitmqctl, it says
> > > > > that there are no messages in the celery queue:
>
> > > > >    $  rabbitmqctl list_queues name messages durable
> > > > >    Listing queues ...
> > > > >    celery      0       true
> > > > >    celeryctl_KTMacBook.local   0       true
> > > > >    ...done.
>
> > > > > The celery queue was durable. Why were the messages not persisted?
> > > > > What do I need to do to make the messages persistent?
> > > > > _______________________________________________
> > > > > rabbitmq-discuss mailing list
> > > > > rabbitmq-disc... at lists.rabbitmq.com
> > > > >https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
> > > > _______________________________________________
> > > > rabbitmq-discuss mailing list
> > > > rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> > > _______________________________________________
> > > rabbitmq-discuss mailing list
> > > rabbitmq-disc... at lists.rabbitmq.com
> > >https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> > _______________________________________________
> > rabbitmq-discuss mailing list
> > rabbitmq-disc... at lists.rabbitmq.com
> >https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss