[rabbitmq-discuss] Am I using Queues Correctly?

Mon Sep 6 20:18:23 BST 2010

To clarify a point, when a message is consumed with no-ack disabled, it
is taken off the queue (in the sense that no other consumer will receive
it), but if the consumer dies without ack'ing it, the message will get
re-inserted into the queue and will be delivered to another consumer.

On Mon, Sep 06, 2010 at 08:01:26PM +0100, Alexandru Scvorţov wrote:
> > When I receive a message from a queue does that remove it from the queue or
> > just mark is as read?
> 
> It depends.  When you basic.get or basic.consume a message from the
> queue, you have to say whether you do it in no-ack mode or not.  If
> you enable no-ack, ``there no need to acknowledge the messages'', so
> consuming it also dequeues it.  If you set no-ack to false, consuming
> the message just marks it as delivered; it's not dequeued until you
> explicitly basic.ack it; if the consumers dies before ack'ing, the
> message is re-inserted into the queue.
> 
> > Do messages get automatically stored to disk or do I
> > need to explicitly turn that on?
> 
> It depends on what you mean by ``stored to disk''.
> 
> As of 2.0.0, all messages may be persisted to disk.  That means that any
> message may get written to disk (RabbitMQ tends to do this if it's
> running low on memory).
> 
> Completely orthogonal to this, messages may be transient or persistent.
> No guarantees are made about transient messages.  Persistent messages
> that end up on durable queues are supposed to survive a broker restart;
> this means that they tend to get written to disk fairly quickly and
> are recovered on startup.  You choose whether a queue is durable or not
> when you declare it.  You choose whether a message is persistent or not
> when you publish it.
> 
> Still, you can lose persistent messages (if the broker dies before it
> has had a chance to sync the disk buffers).  Currently, the only way to
> ensure that messages are never lost is to set the channel to
> transactional mode (tx.select) and do a commit after every message.  If
> the commit succeeds, the message has been published successfully.
> 
> Hope this clears things up.
> 
> Cheers,
> Alex
> 
> On Mon, Sep 06, 2010 at 07:06:45PM +0100, Tom Martin wrote:
> > Thanks for your recommendations, I'll take a look at the topic exchange but
> > if what I've proposed sounds ok then I'll leave it as that.
> > 
> > When I receive a message from a queue does that remove it from the queue or
> > just mark is as read? Do messages get automatically stored to disk or do I
> > need to explicitly turn that on?
> > 
> > Thanks again.
> > 
> > On 6 September 2010 14:41, Vlad Alexandru Ionescu <vlad at rabbitmq.com> wrote:
> > 
> > >
> > > Apart from the mentioned 'direct' and 'topic' exchange types, there is also
> > > the 'headers' exchange (match on a number of attributes that you specify).
> > >
> > > Rgds,
> > > Vlad.
> > >
> > >
> > >
> > > On 09/06/2010 02:38 PM, Alexandru Scvorţov wrote:
> > >
> > >> Hi,
> > >>
> > >> There's nothing inherently wrong with what you describe.
> > >>
> > >> Depending on what you filter on, you could make it simpler.  If you
> > >> filter on a message name/id or something like that, you could probably
> > >> just bind the filtered queue to a direct exchange (match on the routing
> > >> key) or a topic exchange (match on something like a regexp).
> > >>
> > >> If you're filtering rules are more complicated or if they involve
> > >> inspecting the message content, what you're doing is probably the way
> > >> to go.  You could also write a custom exchange type as a plugin.
> > >>
> > >> Cheers,
> > >> Alex
> > >>
> > >> On Mon, Sep 06, 2010 at 02:03:44PM +0100, Tom Martin wrote:
> > >>
> > >>
> > >>> Hi,
> > >>>
> > >>> I'm building a system which takes incoming messages, filters them and
> > >>> stores
> > >>> them to DB. I have each of these three bits running as separate ruby
> > >>> processes, if the filter or database saving processes die then when they
> > >>> come back, new messages are waiting in the queue.
> > >>>
> > >>> I've implemented this using two separate queues: an incoming queue and
> > >>> a filtered queue. The filtering process just takes the incoming queue,
> > >>> does
> > >>> it's business and inserts it into the filtered queue. The DB storing
> > >>> process
> > >>> takes the filtered queue and stores them to DB.
> > >>>
> > >>> I'm very new to RabbitMQ, is this kind of flow correct, creating two
> > >>> different queues? or is there a better way to do it using Exchanges or
> > >>> some
> > >>> other part of RabbitMQ?
> > >>>
> > >>>
> > >>
> > >>
> > >>> _______________________________________________
> > >>> rabbitmq-discuss mailing list
> > >>> rabbitmq-discuss at lists.rabbitmq.com
> > >>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> > >>>
> > >>>
> > >> _______________________________________________
> > >> rabbitmq-discuss mailing list
> > >> rabbitmq-discuss at lists.rabbitmq.com
> > >> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> > >>
> > >>
> > >
> > 
> > 
> > -- 
> > Tom Martin
> > devart.org
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss