[rabbitmq-discuss] [Q] best way to add a sequencer to the broker

[Ben Hood]

Chuck,

On Mon, Dec 29, 2008 at 10:09 PM, Chuck Remes <cremes.devlist at mac.com> wrote:
>> What is the scope of the stamp? Is it globally monotonic
>>
>> a) accross the entire Rabbit cluster
>> b) within a single Rabbit node
>> c) within a single AMQP channel
>> d) just through a particular queue
> I need to order the messages for the entire distributed application,
> so in this case I believe that means (a) and (b). I haven't really
> given much thought to the clustered case. Any rules of thumb when
> designing for a cluster other than "share nothing?"

Well apart from breaking the golden rule of "share nothing" in a
cluster, doing this should be straight forward.

What I mean to say is that stamping each message with a header
property is a pretty trivial extension in terms of the lines of code
you'd have to write to set the header.

And you could use mnesia to maintain a globally unique ordering key
and not have to worry about the coherency of this key.

So all in all, this is all quite doable.

The downside to synchronizing of a single piece of global state (which
is what strict global ordering implies) is that you introduce a not
insignificant bottleneck into the system.

So unless you come up with some funky way to maintain a coherent
global counter, your application will not scale.

Having said that, if scalability/throughput/latency are not issues for
you, then I would cut to the chase and go for the custom exchange I
described earlier.

> This is the best suggestion yet. I'll start with a Java client and as
> we require additional performance I'll see about contracting this out
> via LShift. While I'd love to write it myself, I long ago concluded
> that it makes more sense to farm this kind of activity out to the real
> experts.

As indicated above, setting this for each message would not be too
difficult to do. Given a mechanism to maintain a global counter, it
would be less effort to do the custom exchange that any external relay
mechanism.

HTH,

Ben