[rabbitmq-discuss] A new exchange type

[Martin Sustrik]

> We have a use pattern in mind that I think would require development of a new exchange type, and a couple of conversations with Alexis would seem to reinforce this view that, currently, the functionality we have in mind doesn't exist in rabbitmq.
> 
> What we'd like to do is have an exchange type which acted like a regular topic exchange, where messages are "routed" into queues based on the topic, but in addition kept a copy of the last message sent to the exchange for each topic. So the behaviour of the exchange would be slightly different to how it is at the moment. Currently, if you create a purely topic based exchange, and publish a message to it, then some time alter attach a queue to that subject, no message is delivered until the next message is published to that topic. With the new exchange the newly connected queue would immediately receive a message that was a copy of the last message published, however long ago it was. Clearly, the timestamp on the message is important, but this architecture could, I think, be very useful. One use would be as a replacement for the proprietary "market data feed" type servers many institutions use to share internal data records. Although there is clearly a distinction between a 
"record" and a "message" it would be very simple, and for many purposes perfectly adequate, to build a simple internal feed system around a RabbitMQ message broker with this new, additional exchange type.
> 
> Anyone else see a use for it? We're currently weighing up the pros and cons of having such an exchange developed for rabbit, or continuing to use our own, internally developed but proprietary message broker, which /does/ implement last message persistence in this way.

That's the standard last-value-cache scenario. Almost everyone would 
benefit from that kind of functionality IMO. However, it's pretty 
dangerous to use without the proper caution. The number of topics is 
basically unbound (actually 256^256) meaning that you can exhaust memory 
and cause DoS by swapping to disk when application doesn't restrict 
itself to a finite set of distinct topics. Also, getting the last value 
can be pretty CPU greedy if there's a lot of topics - that can also lead 
to DoS situations.

Martin