[rabbitmq-discuss] Fwd: Exchanges, Routing, and AMQP

[Peter Silva]

reply instead of reply to all...
Thank you ben...


On Tue, Nov 11, 2008 at 6:35 AM, Ben Hood <0x6e6562 at gmail.com> wrote:

> Peter,
>
> On Tue, Nov 11, 2008 at 4:42 AM, Peter Silva <Peter.A.Silva at gmail.com>
> wrote:
>
> This discussion (as an extension to the thread Kirk started) is going
> on too long. If defering the routing type to the bindings is as
> trivial as you maintain, then please enlighten me with a proof of
> concept written in a language of your choice. It would have not have
> to be AMQP specific, just something that shows the basr fundamental
> routing lookup mechanism based on the data structures you are
> proposing.
>
> Code speaks louder than words and I would love to be proven wrong.
>

Funny you should ask!

I would really appreciate someone who understands switching having a look at
our switch.  We've only used it internally so far, but I whipped up some
.deb's in honour of your request:

http://metpx.sf.net

It's a third generation switch, build from blackbox re-engineering of the
other two generations.  There is developer documentation to get a loopback
configuration working for testing, it's very weather oriented right now.
We've been working on it for a few years, but the weather messaging
community is, well, kind of scattered, so the installed base is 1.  It's a
very enthusiastic 1, mind you :-)

concept mapping:

AMQP           MetPX
---------            -----------
Exchange      Receiver
Queue           Sender
Binding          Routing

bindings work with two verbs, accept and reject.
We differentiate between regex's and direct mappings for efficiency reasons
(use a table for the direct ones.)
and yeah, we're stuck with a linear search through regex's, which we try to
minimize.

We didn't do routing that matches to exchanges, but almost all the exchanges
use the same routing tables. It implements the rest of the routing
algorithms I have described in python.  Ruthless simplicity meant that our
first database was done using 1 file per message on a raw file system.  It
turns out we can deliver hundreds of messages per second with that storage
and FTP, which was good enough for us.

The packaging is kind of rough, internal people use it a little
differently.  We've been using debian packages for a few months, but file
layout is a little different.

The routing is actually a very small part of the code a few hundred lines at
most.  Most of it is protocol drivers.

The reason for me asking these questions is to understand how to add AMQP
support to metpx.

ready to answer questions, on or offline.



>
> > On the other hand,
> > going through long lists of regular expression evaluations can kill
> > performance.
>
> :-)


the key is minimizing the number that have to be searched for each routing
event.   If you treasure speed over flexibility, then just don't put any
regex's in your routing.

Another difference in the design point, is that metpx came from dealing with
existing exchanges, and did not have the luxury of defining it's own key
syntax.   Data producers are often limited in the names they can supply, and
consumers with what they receive (remember, we deal with files, our keys are
essentially file names.)

If we insisted on file naming conventions, we ended up having to implement
separate glue elsewhere to rename things we received into the file naming
convention the switch required.
  It made sense to add regex comprehension to the switch itself to derive
routing keys from the arbitrary file names we would receive.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20081112/2f171d84/attachment.htm