[rabbitmq-discuss] Queue and filtering questions

Thu Nov 17 10:27:50 GMT 2011

Matthias,

Ideally, I dont want my clients to know about each other.

" So there must be some mechanism in place that detects worker
failure. Could that not trigger an event that is broadcast to all the
consumers of the output queue, so that they can subsequently filter
out
the results of the failed worker?"

This is pretty much what I am planning to do now. Each client
subscribe to a queue, and when a worker fails (expected to be very
infrequently), then an update is broadcast to all clients saying
ignore any messages with id of XXXXXX.
The only problem I can see with this method is that we cant know when
its safe to delete the notices saying 'ignore anything with id of
XXXXX' as we cant search the messages in the system to say that there
are no more messages with id XXXXX. However, this is unlikely to be a
major problem in practice.

As for the added complexity, I was only going to do it server side if
it was possible to do simply. As such, its not a problem.

Thanks for such a simple and easy to use yet powerful messaging
system.

Andrew

On Nov 17, 9:39 pm, Matthias Radestock <matth... at rabbitmq.com> wrote:
> Andrew,
>
> On 17/11/11 02:49, Andrew Esler wrote:
>
> > The reason I want to do filtering server side is then I have easy
> > access to the info I need for filtering. If filtering is done client
> > side, then all clients have to know about each other and tasks they
> > are working on in order to be able to find the information they need
> > to make filtering decisions.
>
> It is not obvious from your earlier problem statement why there would be
> a need for clients to know about each other.
>
> You said "I would like to be able to filter messages from the output
> queue that were sent before the worker failed so they are not delivered
> twice.". So there must be some mechanism in place that detects worker
> failure. Could that not trigger an event that is broadcast to all the
> consumers of the output queue, so that they can subsequently filter out
> the results of the failed worker?
>
> I wonder though whether this complexity is really necessary. The failure
> detection can never prevent duplicates completely - after all, some of
> the results of the dying worker may already have been processed. So
> given that the system needs to cope with that anyway, is really worth
> putting a secondary mechanism in place? That would only make sense if
> the worker failure frequency is high and the cost of ordinary duplicate
> handling is high.
>
> Regards,
>
> Matthias.
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss