[rabbitmq-discuss] whether the weather is right.

[Ben Hood]

Peter,

On Sat, Nov 8, 2008 at 3:36 PM, Peter Silva <Peter.A.Silva at gmail.com> wrote:
> The protocol is kind of daunting to me, and rabbit
> seems to be the place where I have some vague clue as to what people are
> saying, and there is an active list.

Can you elaborate on how you think the protocol is daunting? Is it too
complicated, too vague, too verbose or something else. It is always
interesting to hear why the spec is not very well received. Or is
something like the lack of ancillary supporting documentation or
examples?

> thinking about applications in weather
> messaging.  Don't really understand how to map the two realms so far.

Pardon my ignorance, but are you talking about sending meteorological
data via messages?

> The weather messaging model is sort of pub/sub.   Every message has either a
> header or a file name which is used as a routing key.   We have a routing
> table (about 40k entries in an in-memory python dictionary/associative
> array) which lists destinations to which the message will be queued for
> transmission.

If I understand you correctly, a destination in the weather domain
could map to a queue in AMQP.

A producer would send a message with a particular routing key to an
exchange and the broker evaluates to which endpoint it needs to route
the messages to based on queue-exchange bindings (see
http://hopper.squarespace.com/blog/2008/10/7/consumer-driven-messaging.html).

> We ensure that downstream links are not overburdened (a
> radar feed is about 6 mbs, some of our clients run 4800 bps.)

Producer flow control is now in the main Rabbit source tree, hence it
will be in the upcoming release.

If you are looking for fine grained egress balancing, you might search
this list for discussions about basic.qos.

> and we take
> care of permissions (some clients are not allowed to see certain data from
> certain sources.)

We're actually removing ACLs, because Rabbit was the only broker to
actually implement them and hence they were nuked from the spec :-(

Again, this is a sought after feature, which we'd like to put back
.... there are discussions on ACLs in the archives.


> I read Kirk Wylie's October thread about publishing with interest, as it
> resonates strongly in my world.

Kirk might embrace the discussion by actually answering my last post
on the subject ;-)

> I want the source of messages to have as
> little as possible to say about how or where I (the MHS receiving messages)
> will forward them.The source should simply put a label on it to tell me
> what it is.

And I would argue that all intents and purposes is the currently the
case - Kirk is just suggesting taking this thought to the next level -
I am questioning the doability of this - but he has come up something
yet other than "yes, of course, it's doable". See my (as yet still
unanswered) post from Oct 24.

> The channel it arrives on tells me where it is coming from,
> with the two pieces of information, the broker/exchange/MHS (I'm hoping all
> three are synonymous) will make routing decisions.

The is what currently happens.

> I find routing tables a lot easier to understand than capabilities and
> ACL's, and sufficient, if not elegant.  Has anyone looked at them as a
> security mechanism?

There is a post about this that Steve Jenson started on Nov 3. Also,
just search the list for ACL and capabilities, much has been said.

HTH,

Ben