[rabbitmq-discuss] Dead-Lettering Queue Contents On Queue Expiration
matthias at rabbitmq.com
Mon Jan 27 14:32:54 GMT 2014
On 17/01/14 22:08, Alex Robson wrote:
> I have updated the pull request. I jumped the gun a bit in thinking that
> I had solved this with the initial pull. I still may not have resolved
> it in a way that would make sense to devs who know the code base better
> than I do. I wasn't sure if there were a better way to remove bindings
> from an expiring queue and its DLX, so I'm removing them first from the
> destination using rabbit_binding:remove_for_destination and then calling
> the exchanges remove_bindings. The latter call is necessary because with
> out it, a consistent hash exchange will retain invalid routes and
> continue to try and route messages to the expired queue.
I can see why this is necessary, but it makes a whole bunch of
assumptions about topology that work for c-h-x but not in general.
Wwhy not remove all bindings, indeed even perhaps the queue itself? i.e.
bring forward part of what happens in terminate_delete, in particular
the call to rabbit_amqqueue:internal_delete.
That is not w/o problems either since now you have a "dark" queue, i.e.
a queue that is completely invisible to routing and management. Yet that
queue is still operating - it's dead-lettering - and using the backing
queue, etc. That is likely problematic. So, for example, a client could
come along and create a new queue with the same name, which would end up
stomping on or tripping over the persistent storage of the dark queue.
Yet if you don't remove the queue record then there is nothing stopping
clients from creating new bindings for it. Including to the DLX.
So perhaps this queue record in mnesia needs to indicate some limbo
state, similar to what happens to durable queues residing on down nodes.
In that state, the existence of the queue is known but no operations can
be performed on it.
This is all getting rather tricky.
Plus we still have to deal with the issue I mentioned previously, of
messages that are in-flight between the exchange and
in-the-process-of-deletion queue getting lost. That can actually be
addressed for specific use cases, namely those where the message is
expected to reach exactly one queue - by publishing in mandatory mode
and re-sending the message when a basic.return is received. Obviously
this can create dups, but they are inevitable in most cases anyway. It's
also rather slow since it's synchronous (though that will change in the
next RabbitMQ feature release).
Trouble is, you'd also have to get the dead-lettering to publish in
mandatory mode. Which introduces all sorts of complications.
Also note, btw, that, in general, dead-lettering is not guaranteed. It
doesn't use confirms (and doing so would be very complex indeed), so
messages can get lost due to failures during dead lettering.
Summary: It's all rather complicated.
More information about the rabbitmq-discuss