[rabbitmq-discuss] whether the weather is right.

Sat Nov 8 15:36:57 GMT 2008

Hello,

introductory letter... I have been trying to figure out a place to learn
about AMQP for the past little while, and am playing around with py-amqplib
& rabbitmq on a laptop.  The protocol is kind of daunting to me, and rabbit
seems to be the place where I have some vague clue as to what people are
saying, and there is an active list.   very impressed with rabbitmq & erlang
from what I have read so far.  thinking about applications in weather
messaging.  Don't really understand how to map the two realms so far.

The weather messaging model is sort of pub/sub.   Every message has either a
header or a file name which is used as a routing key.   We have a routing
table (about 40k entries in an in-memory python dictionary/associative
array) which lists destinations to which the message will be queued for
transmission.  Regular expressions can be used in lots of interesting ways
in this process.  We ensure that downstream links are not overburdened (a
radar feed is about 6 mbs, some of our clients run 4800 bps.) and we take
care of permissions (some clients are not allowed to see certain data from
certain sources.)

I read Kirk Wylie's October thread about publishing with interest, as it
resonates strongly in my world.  I want the source of messages to have as
little as possible to say about how or where I (the MHS receiving messages)
will forward them.  The source should simply put a label on it to tell me
what it is.  The channel it arrives on tells me where it is coming from,
with the two pieces of information, the broker/exchange/MHS (I'm hoping all
three are synonymous) will make routing decisions.

To me, unicast/direct, multi-cast/topic, broadcast/fanout data distribution
has nothing to do with how the data arrived at the switch/broker/exchange.
Neither producer, nor consumer should have anything to say about that.
Producers describe what they are producing, consumers describe what they are
interested in, the MHS people could map producers and consumers together
masked by the routing table (which defines permissions)  and that would
include mapping particular exchanges onto 'topics'/'exchanges', etc... The
aggregation role includes categorization.  For example, we have groupings of
data accoring
to interest group... climatology, hydrology, aviation, road weather.  It
isn't the publisher of the information that knows (or is even aware of) who
is interested in their data.  The client won't necessarily know that the
data exists.   The aggregator/broker/MHS/exchange operator can try to bring
the two together.

I find routing tables a lot easier to understand than capabilities and
ACL's, and sufficient, if not elegant.  Has anyone looked at them as a
security mechanism?  I imagine the task is to do some sort of use case
mapping.    They might be applicable as one security model alongside others.

If there are any other weather people out there, would be interesting to
talk.

Just looking for now... but at least I've found a place to watch!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20081108/6eed2479/attachment.htm 