[rabbitmq-discuss] [Q] best way to add a sequencer to the broker

[Ben Hood]

Hi Chuck,

On Mon, Dec 29, 2008 at 2:04 PM, Chuck Remes <cremes.devlist at mac.com> wrote:
> 1. This code will likely start before other services, so it will
> declare the exchanges and block on them while waiting for traffic. The
> topic Consumer/Producer examples all illustrate how to do this with a
> single exchange. How do I accomplish this task with multiple
> exchanges? Do I need a separate thread and channel for each exchange?

In general, one should avoid sharing channels across threads - for
background, see here:
http://hopper.squarespace.com/blog/2008/10/6/multithreading-amqp-clients.html

> 2. Is there a better way to accomplish this task?

I'm not quite sure I fully understand the actual use case. AFAICT you
want each message to have it's own sequence number that you want to
set yourself but have some kind of intermediary increment this number.
But because this is stateless operation (because you are just
incrementing a context free number), I must conclude that I don't
really understand what you are really trying to achieve (although I've
read through your post twice - maybe I'm just being stupid). Or is
that the publisher always sets this sequence number to 0 and you want
to stamp it with a globally incremented number?

BTW I don't whether the delivery tag that is sent in the header of the
message to the consumer is of any use to you for sequencing.

> I've read some notes
> about future releases of rabbitmq allowing for "internal" clients that
> run inside the rabbitmq memory space. I think this is an ideal use of
> that capability. Am I right or barking up the wrong tree? And does
> that internal client need to be written in erlang or can it be java/c/
> whatever?

The Erlang AMQP library can run transparently in the address space of
the broker. The interface is exactly the same as the client library
that uses TCP, the only difference is that the TCP stack and the
message serialization/deserialization is avoided. This basically takes
advantage of the fact that Erlang messages natively and hence you can
unobtrusively abstract out the AMQP wire format.

The types of things that this is useful for is processing messages in
a custom fashion without a network hop and you can stick to the AMQP
model rather than having to write something that is Rabbit specific.
This also takes advantage of the fact that Erlang processes are
isolated from each other and so it is ok to run "foreign code" in the
same memory space as the broker.

But I don't know if this solves your problem, because as I stated
above, I didn't really understand it.

> 3. Is there a way to make this more dynamic so I do not have to
> declare all the exchanges up-front in this sequencer code? Ideally
> this service could detect that new exchanges were declared by other
> services and automatically subscribe to them to do the sequence
> stamping and routing.

I don't know if the terminology is right here, because you generally
don't subscribe to an exchange, rather to a queue.

In general, it doesn't matter who declares an exchange - the operation
is idempotent - see this FAQ:
http://lettuce.squarespace.com/faq/exchanges/im-interested-in-creating-an-exchange-which-entity-should-de.html

HTH,

Ben