[rabbitmq-discuss] Transactions in AMQP

[Alexis Richardson]

Tanmay

In the spec:

AMQP supports three kinds of transactions:

   1.      Automatic transactions, in which every published message
and acknowledgement is processed as a stand-alone transaction.
   2.      Server local transactions, in which the server will buffer
published messages and acknowledgements and commit them on demand from
the client.
   3.      Distributed transactions, in which the server will
synchronise its transactions with an external transaction coordinator.

You ask:

1)  "How can this transaction be atomic and/or rolled-back since the
message is already gone?"

The answer is that the message is not already gone, but held at the
broker as part of a partially (or pre-) committed transaction.  This
is because, for the publisher to have an explicit 'abort' we are
talking about case 2, server local "TX" or case 3, "DTX".

In more detail --

In 0-8, 0-9 and 0-10, there is a notion of 'command'.  Commands are
basically control ('admin') expressions sent between the broker and
its communicating peers, for example queue creation.  Each command is
atomic (cf. txn type 1 above).

When there is no explicit transaction (TX or DTX) then messages are
processed atomically too - like commands.

In 0-8, 0-9 and 0-10, there is a notion of transactional message
*group*, or TX.  The full details of TX are in the spec.

Your question was about just such a 'group' -- it contained just one
message atom, followed by an abort/commit atom.  In pseudocode your
example could be written as a group of three: "open_txn ;  send
message M ; commit_txn".  The consumer won't see M until the broker
sees and commits (commit_txn).

Sidebar: in 0-10, commands have an internal sequencing enabling
correlation for in-order execution.  The precise relationship between
command execution and session management is new in 0-10 since sessions
are new in 0-10 (they manage ack behaviour and message order).

Your puzzlement seemed to arise due to the interplay between ack and
abort/commit.  Would the broker ack the message, knowing it to be in a
transaction (TX), before waiting for a commit message?  The answer is
subtle and to do with the architecture for 0-10.  Let's go back to the
pseudocode above.

* The producer, in the example, has a session relationship with the
broker.  In this scope, the message M can have been acked before the
broker sees 'commit_txn'.
* But all this ack means is that the broker has seen M, and the
producer does not need to resend M.  I.e. the ack is being used as a
'control frame' to manage session state between producer and broker.
We need this because the broker and producer may be connected by a
network.  If they shared a physical address space (reliably) then we
would not need this.
* The sending of this ack does not imply that the broker has yet made
M visible to consumers.  In fact, in this case M will not be visible
to consumers because M is in a server local (TX) transaction.  M will
sit in the brokers precommitted state until told to commit.  Once the
command to commit arrives, then M will be committed and the consumer
can see M.

The above analysis would also cover the case where there was more than
one message sent, say "open_txn ;  send message M ; send message N ;
send message O ; commit_txn".  In this case the consumer won't see any
messages at all, until commit.  But acks will be used under the hood,
to enable the broker and producer to manage questions like 'do I need
to resend N, even though M and O have been acked?'.

You need this architectural layering in order to scale in the presence
of multiple implementation scenarios.

Finally....

In 0-10 the notion of a distributed transaction, DTX, is defined so
that XA compliance is achieved.  You could use this for grouping
commands when sharing broker state with another transactional entity,
for instance an RDBMS.

The guys who did the DTX work did a really good job and as Carl
mentioned, you can find DTX details in the JIRA:
https://jira.amqp.org/jira/browse/AMQP-4

You also asked:

2) "Is the protocol currently assuming a reliable transport layer
(TCP, SCTP, etc) and no unreliable connection (say, UDP)? If yes, will
the later releases work on including unreliable transport layer
protocols?"

As Carl explained, the answer is:

--- 0-8 and 0-9 assume TCP
--- 0-10 sessions enable reliability in the presence of UDP (or
'provide a framework for it')
--- But the full details will provided in later releases

HTH!

alexis





On 8/28/07, Goel, Tanmay <tanmay.goel at intel.com> wrote:
>
>
>
>
>
>
> Hi,
>
>
>
> 1. I have a basic question about transactions in AMQP. What exactly does a
> transaction include/mean? For example, does it include the administrative
> features like creation of queues, exchanges, etc that will be auto-deleted
> if the client aborts the transaction. I'm not entirely clear about this, so
> I'm just trying to understand and figure out possibilities. The spec says
> that messages and acknowledgments are covered in a transaction. What would
> happen if the publisher sends a message to the broker/queue, consumer
> receives (reads it from the queue) and processes it, sends an ack to the
> broker and then the publisher decides to abort the transaction? How can this
> transaction be atomic and/or rolled-back since the message is already gone?
>
>
>
> Please help me understand this. I'm trying to understand the scope of a
> transaction.
>
>
>
> 2. Is the protocol currently assuming a reliable transport layer (TCP, SCTP,
> etc) and no unreliable connection (say, UDP)? If yes, will the later
> releases work on including unreliable transport layer protocols?
>
>
>
> Thanks,
>
>
>
> Tanmay
>
>


-- 
Alexis Richardson
+44 20 7617 7339 (UK)
+44 77 9865 2911 (cell)
+1 650 206 2517 (US)