[rabbitmq-discuss] A new exchange type

Wed Jan 28 15:21:30 GMT 2009

Hi all,

Tony Garnock-Jones wrote:
> That's a great insight. We've been puzzling over "last-value caching"
> for a while now, and not made any great progress, largely I think
> because we'd concentrated on putting it in to the *queues* rather than
> the exchanges. Your observation that it's best placed in the *exchange*
> is, I think, critical. Great stuff!
> 
> Alister Morton wrote:
>> We have a use pattern in mind that I think would require development
> of a new exchange type, and a couple of conversations with Alexis would
> seem to reinforce this view that, currently, the functionality we have
> in mind doesn't exist in rabbitmq.

Alternatively (and please note that this is purely a theoretical speculation on my part),
is the broker itself the best place to run this extended functionality? Wouldn't it be
beneficial to have core broker implement only absolutely necessary fundamental concepts
and push the rest of functionality out to user-space? (compare with kernel-space vs
user-space in OSes).

For example, if core broker could expose AMQP events via AMQP (i.e., broker itself will be
sending messages to a certain protected exchange every time a queue is created, or
consumer connects, or consumer disconnects, etc) plus have its runtime state obtainable
via AMQP (again - send a request message to a certain exchange, get response), then I
could hopefully implement last-value-cache scenario totally in user-space. (This could
also help with monitoring, btw).

Benefits could be:
* smaller core broker (which means faster, more secure, more reliable, fewer moving parts,
more stable, rev'ed less frequently)
* flexibility

When I say flexibility, I mean, for example, when I am running a broker in the cloud,
local disk (the only medium to which core broker can persist; forget about mnesia for now
due to partitioned network) might not be the most reliable persistence medium and I might
choose to persist to something else. Can't expect core broker to integrate with AWS
SimpleDB, right?

Furthermore, flexibility can come handy for those who are like me and have not yet
advanced beyond chapter 5 of Joe Armstrong's book :), and for organizations that do not
(yet!) have Erlang expertise in house. With user-space approach, any language can be used
to write new routing logic and extend core rabbit, and it can better integrate with one's
infra.

Possible disadvantages:
* suboptimal algorithms (can be improved with community peer review)
* higher latency for certain things outside of core broker (not a big deal for me - but I
am not running a trading shop, YMMV)

In other words, please consider rabbitmq core broker as a *platform* on top of which new
functionality can be written and shared, *in* *addition* to being an *end* *product*.

- Dmitriy