[rabbitmq-discuss] request for help!

[Robert Godfrey]

Sorry - had meant this e-mail exchange to be on list (should really
learn to read the To: list before sending :-) )

-- Rob

---------- Forwarded message ----------

From: Robert Godfrey <rob.j.godfrey at gmail.com>
Date: 6 May 2010 12:35
Subject: Re: [rabbitmq-discuss] request for help!
To: Martin Sustrik <sustrik at 250bpm.com>



Oops - just noticed that this mail only went to you and not the Rabbit
list... would you be OK if I posted it back onto the list - which was
my original intent?

Som more comments in-line below:

On 6 May 2010 12:16, Martin Sustrik <sustrik at 250bpm.com> wrote:
>
> Hi Rob,
>
>> On 6 May 2010 10:11, Martin Sustrik <sustrik at 250bpm.com <mailto:sustrik at 250bpm.com>> wrote:
>>
>>>    Great you are bringing this to the public attention!
>>>
>>>    We've discussed this before, so I'm going to repeat myself for the sake
>>>    of people on the mailing list...
>>>
>>>     From my point of view there are two crucial issues with AMQP/1.0:
>>>
>>>    1. Messy layering. Almost every concept is discussed at multiple places.
>>>    It's very hard to understand the specification in its current form. (To
>>>    be fair, one piece is nicely and consistently separated. The codec.)
>>
>>
>> Would be glad to understand this comment in more detail.  At least
>> conceptually I think the layering in 1-0 is a lot better defined with
>> concerns separated than in previous versions.  Are your concerns
>> around the  model, or the documentation of it?
>
> It's much better than it used to be. Anyway, I volunteer to help with this.
> My idea is to start with moving pieces of the specification around to make
> it easier to follow without changing the wire protocol. Will get in touch with
> you shortly!
>
>>>    2. Assumption that messages are stored on the node in unordered fashion
>>>    shifts AMQP from being message queueing protocol to being database
>>>    protocol. This blurs AMQP's mission, makes it compete with SQL and
>>>    causes implementation to be extremely costly and complex. We should
>>>    follow the good old "do one thing and do it well" principle and focus on
>>>    defining MQ behaviour rather than DB behaviour.
>>
>>
>> It's not that we assume the messages are unordered at the node... but that
>> the *transport* makes *no* assumptions about message ordering within a
>> node (because it doesn't need to).  It would be perfectly valid for an
>> implementation to be built with nodes that store messages strictly in order
>>  - and that's certainly how I expect most "Queues" inside "Brokers" to work
>> (having said that we already see things in AMQP implementations such
>> as priority queues, last value queues, etc. that violate this notion of "strict"
>> ordering).
>
> It's wider than ordering alone. Node with unordered messages is basically a
> database table.

A Node with unordered messages would be some sort of Bag I would
think... Unlike a DB table there would be no natural notion of a
primary key, or necessarily any notion of a way to select the messages
(though I guess the node *may* support certain AMQP1-0 filter types).
Again, there is no requirement on implementers to write such a Node
type (unless Michael has secretly put that in the document he is
currently writing ;-) )... All that the core spec is doing is saying
is "it's perfectly valid to have nodes that give FIFO ordering, LIFO
ordering, or any other (non-)ordering you like".  This is actually
removing a burden on implementers rather than adding one (although the
requirement for the same sort of pseudo-FIFO that 0-x requires will
probably also be a requirement for AMQP 1-0 *Brokers*).

>
> However, there are also other DB concepts in the spec. Say a concept of
> non-destructive link -- which boils down to a SELECT statement. Or, say,
> it seems implied that a message cannot silently disappear from a node
> same way as record cannot silently disappear from a DB table.

There is absolutely no intent to prevent a node from silently
"disappearing" a message that it is holding.  All that
"non-destructive" means is that the act of sending the message to the
receiver does not in itself cause the message to be "disappeared".
The two major use cases this functionality is designed for are Queue
"browsing" (i.e. the ability to look at what is in your queue without
consuming the messages) and the ability to implement "Topics" as a
single entity (rather than the AMQP 0-x way of having a queue per
consumer).  The Topic case is potentially more interesting as a Topic
node would have a policy that says "when all connected consumers have
moved beyond a message in the *ordered* Topic, then the message may be
deleted".  This obviously requires some notion of ordering at the
(Topic) node.  Having implemented this functionality elsewhere, it
certainly doesn't require a database table like data-structure - all
it requires is that you hold pointers into the queue.

>
> Compare this witch 0-9 exchanges where message is dropped when it
> cannot be routed. Etc.

 My belief is that an 0-x exchange would be a perfectly valid node in AMQP.

>> Subsequent pieces of the AMQP specification will define the required
>> node types that need to be supported by applications acting as
>> "AMQP Brokers".  Michael may be able to talk more to that as that's
>> his area of responsibility :-)


hope this helps,
Rob