[rabbitmq-discuss] [Q] best way to add a sequencer to the broker

[Chuck Remes]

On Dec 29, 2008, at 12:29 PM, Ben Hood wrote:

> Chuck,
>
> On Mon, Dec 29, 2008 at 5:43 PM, Chuck Remes  
> <cremes.devlist at mac.com> wrote:
>> Your last sentence quoted above gets it exactly. Each publisher will
>> set the sequence number to 0 so that the republisher/sequencer  
>> service
>> can add the global number stamp to it. I guess the publishers could
>> also set it to nil; the initial value is irrelevant since the
>> Sequencer will overwrite it.
>
> Fair point. It need not be set until it passes through the broker.
> FWIW there are some standard header properties that could be misused
> for this purpose, e.g. the message_id field.

I would never dream of misusing a header field! :)

>> The use case is to stamp a global order on all messages through the
>> broker for the purposes of replay, debugging and async logging. I
>> realize that during normal operation the services will be publishing
>> messages in an arbitrary order; it doesn't matter to me. I need to
>> recreate that arbitrary order *exactly* for a replay.
>
> What is the scope of the stamp? Is it globally monotonic
>
> a) accross the entire Rabbit cluster
> b) within a single Rabbit node
> c) within a single AMQP channel
> d) just through a particular queue
>
> ?
>
> Obviously, as the scope widens the throughput will become more serial.

I need to order the messages for the entire distributed application,  
so in this case I believe that means (a) and (b). I haven't really  
given much thought to the clustered case. Any rules of thumb when  
designing for a cluster other than "share nothing?"


>> Think of the sequencer as the software equivalent of a hardware bus
>> clock. Alternately, think of the sequencer as the equivalent of  
>> adding
>> an identity column to a database table (this column contains a
>> monotonically increasing integer assigned by the server).
>
> Strict ingress ordering is going to cost more than global identity,
> which could be achieved by a guid, for example.
>
> A timestamp would be cheaper, but than would not be strictly ordered,
> just asymptotically.

Yes, this is strict ingress ordering. If it were just stamping it with  
a GUID I wouldn't have bothered you fine folks with my question.

>> I think this does solve the issue. I don't know Erlang yet so I'll
>> probably keep the Sequencer as an external Java client for the short-
>> term, but it's nice to know I can get a performance win with a  
>> rewrite
>> in Erlang. Avoiding the TCP overhead, even if it is short circuited  
>> on
>> a localhost, is a good thing.
>
> What would be more efficient would be a custom exchange implementation
> that stamps each message as it passes through, because you would avoid
> the delivery-redelivery overhead. This would also mean less moving
> parts.
>
> In contrast to the Erlang AMQP client, this would involve some coding
> in the server.

This is the best suggestion yet. I'll start with a Java client and as  
we require additional performance I'll see about contracting this out  
via LShift. While I'd love to write it myself, I long ago concluded  
that it makes more sense to farm this kind of activity out to the real  
experts.


>> As other services declare exchanges and publish to them, I would like
>> my Sequencer to dynamically detect the existence of a new Exchange so
>> it can create a queue, bind the new queue to the Exchange and receive
>> messages from it for processing and republishing.
>
> ATM there is no notification facility present in Rabbit to tell
> somebody that an exchange has been created. One could extend Rabbit to
> do this (which wouldn't be rocket science) or you could also do this
> on the application level, i.e. maintain a notification queue that the
> sequencer listens on, and have whoever creates a new exchange also
> post to this queue.

Excellent suggestion; I'll do this at the application level. Thank you!

cr