[rabbitmq-discuss] Queue Look-ahead, et alia

[Jerry Kuch]

Hi, Paul.

It is in fact possible to write alternate storage backends
for queue contents, but it's a non-trivial amount of work
and may ultimately give you something you don't want.

The current persister used by Rabbit is explicitly quite
tuned to messaging use cases; other data stores may cater
to quite different workloads and give you performance, or
perhaps even correctness, situations you won't enjoy.

A discussion of some of the issues can be found here:

http://www.rabbitmq.com/blog/2011/01/20/rabbitmq-backing-stores-databases-and-disks/

Best regards,
Jerry

----- Original Message -----
From: "Paul M. Bell" <pbell at syncsort.com>
To: "Emile Joubert" <emile at rabbitmq.com>
Cc: "RabbitMQ List" <rabbitmq-discuss at lists.rabbitmq.com>
Sent: Tuesday, February 7, 2012 8:28:35 AM
Subject: Re: [rabbitmq-discuss] Queue Look-ahead, et alia

Merci beaucoup, Emile, for another very helpful reply. I appreciate it.

It seems that I may have a "square peg, round hole" problem in the way I am trying to use RabbitMQ. Perhaps it's not the right tool for the job. That said, I think it has much to offer and I am championing its use in-house.

Your mention of database in the context of "random access" to a queue leads me to the following questions (and here I am in way over my head): I recall that Rabbit supports different types of queue backing storage by means of "extensions." Would it be possible via such an extension to back the queue with, say, SQL storage? And might this allow, through standard SQL access patterns, random access to the queue?

Thanks again.

-Paul

-----Original Message-----
From: Emile Joubert [mailto:emile at rabbitmq.com]
Sent: Monday, February 06, 2012 8:46 AM
To: Bell, Paul M.
Cc: RabbitMQ List
Subject: Re: [rabbitmq-discuss] Queue Look-ahead, et alia

Hi Paul,

On 06/02/12 12:37, Bell, Paul M. wrote:
> 1. Is it possible for a consumer to consume specific messages from a
> queue, e.g., one that contains a specific value (like a unique
> "request ID" that is part of each message)?

No, but if you are trying to match up requests with replies then
consider making use of a dedicated reply queue, in the way RPC does. The
6th tutorial explains in greater detail:
http://www.rabbitmq.com/tutorials/tutorial-six-java.html

> 2. Is it possible for a consumer to "run a queue," e.g., visit each
> message in the queue without removing it?

In theory a consumer could fetch many messages without acknowledgement,
acknowledge only those that match a filter and requeue the remainder.
Treating a queue like this strongly discouraged. The specification says
of basic.reject:

"The client MUST NOT use this method as a means of selecting messages to
process. "

If you require random access to messages then consider using a database,
or redesigning the message flow in such a way that consumers are able to
process all messages at the head of a queue.

> 3. Perhaps related to (1): does Rabbit support a kind of queue "peek"
> logic in the manner of Queue objects in C# and elsewhere? Peek allows
> one to obtain the queue head but leave it in place. (I know from
> previous threads on this list that message will remain in Q until
> expressly ack-ed, so maybe that's the answer.....)

A consumer can fetch and requeue a message from a queue, but (unless
there are other consumers consuming from the same queue) it will not
make any progress this way. The reason is that the consumer will see the
same message over and over.

> Behind my questions is something I learned re Microsoft's Azure
> design. Its message queues can be manipulated via RESTful URIs.
> Consequently, the contents of an Azure queue can be queried, just as
> one might query data in the persistence layer.

Data structures that can be browsed and manipulated in this way are not
strictly speaking queues. For performance reasons the internal data
structures in RabbitMQ have queues that do not permit random data access
patterns.




-Emile



ATTENTION: -----

The information contained in this message (including any files transmitted with this message) may contain proprietary, trade secret or other  confidential and/or legally privileged information. Any pricing information contained in this message or in any files transmitted with this message is always confidential and cannot be shared with any third parties without prior written approval from Syncsort. This message is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any use, disclosure, copying or distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or Syncsort and destroy all copies of this message in your possession, custody or control.
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss at lists.rabbitmq.com
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss