[rabbitmq-discuss] A new exchange type

[Tony Garnock-Jones]

Alister,

That's a great insight. We've been puzzling over "last-value caching"
for a while now, and not made any great progress, largely I think
because we'd concentrated on putting it in to the *queues* rather than
the exchanges. Your observation that it's best placed in the *exchange*
is, I think, critical. Great stuff!

So given a stateful (!) exchange of the kind you describe, what could
some of the parameters be for its behaviour?

 - how entries in the cache are expired

 - how the cache is keyed

 - whether looser consistency than total across a cluster is
   worthwhile for each particular application

Anything else?

Off the top of my head, it'd be a straightforward enough implementation.
Presumably it would be activated by an argument in the exchange.declare
arguments table. A new shared mnesia table would be required, with the
same scope as the routing tables. It could either be disk- or
ram-resident. On binding creation, a scan of the cache table would be
done and the matching messages delivered to the queue before the binding
completes.

Tony



Alister Morton wrote:
> We have a use pattern in mind that I think would require development
of a new exchange type, and a couple of conversations with Alexis would
seem to reinforce this view that, currently, the functionality we have
in mind doesn't exist in rabbitmq.
> 
> What we'd like to do is have an exchange type which acted like a
regular topic exchange, where messages are "routed" into queues based on
the topic, but in addition kept a copy of the last message sent to the
exchange for each topic. So the behaviour of the exchange would be
slightly different to how it is at the moment. Currently, if you create
a purely topic based exchange, and publish a message to it, then some
time alter attach a queue to that subject, no message is delivered until
the next message is published to that topic. With the new exchange the
newly connected queue would immediately receive a message that was a
copy of the last message published, however long ago it was. Clearly,
the timestamp on the message is important, but this architecture could,
I think, be very useful. One use would be as a replacement for the
proprietary "market data feed" type servers many institutions use to
share internal data records. Although there is clearly a distinction
between a "record" and a "message" it would be very simple, and for many
purposes perfectly adequate, to build a simple internal feed system
around a RabbitMQ message broker with this new, additional exchange type.
> 
> Anyone else see a use for it? We're currently weighing up the pros
> and
cons of having such an exchange developed for rabbit, or continuing to
use our own, internally developed but proprietary message broker, which
/does/ implement last message persistence in this way.

-- 
 [][][] Tony Garnock-Jones     | Mob: +44 (0)7905 974 211
   [][] LShift Ltd             | Tel: +44 (0)20 7729 7060
 []  [] http://www.lshift.net/ | Email: tonyg at lshift.net