[rabbitmq-discuss] Binding to multiple queues in a round-robin style using the PIKA library

[Stephen Young]

Hey Emile,

I've updated to disconnect and reconnect from queues when encountering
deadlock, that has solved the problem. Thanks a lot for your help!

Cheers,
Stephen

On Feb 3, 11:29 am, Emile Joubert <em... at rabbitmq.com> wrote:
> Hi Stephen,
>
> On 03/02/12 15:49, Stephen Young wrote:
>
> > Q1...Qn are the queues I have created
> > Each consumer binds to all queues, and has the capability of parsing
> > and using data from each queue.
>
> > A request is pushed through a broker and generates a set of queue
> > entries, E1...En
> > Ej must be processed for Ej+1...En to be processed, the rest are
> > independent of the result of Ej.
> > Bundling requests together causes the system to be significantly
> > slower, as it doesn't capitalize on the heavy parallelism of running
> > E1..Ej simultaneously, and once Ej is processed, Ej+1...En
> > simultaneously.
>
> This is just a suggestion and might not be possible, but why not treat
> Ej completely separately from other entries on the publishing side?
> Process Ej perhaps using an RPC call and include the result in all
> messages Ej+1..En. Or make the result of Ej available to consumers in
> some other way if it is large. You should obtain good parallelism that way.
>
> > When I don't have these consumers, or have a single consumer managing
> > the whole system, what seems to be happening is that the consumer will
> > stop on one of the queues Qj+m, and read a request, determine that it
> > cant complete, send basic_reject, and then read again from the same
> > queue. This is not problematic all the time, but occasionally
> > generates a loop on that queue by the consumer. What i'd like to do is
> > have the consumer act like:
>
> > read queue Qi
> > determine blockage, send basic_reject
> > read queue Qk
> > process, send basic_ack
>
> I'm not familiar with Pika, but can you unsubscribe from Qi before the
> basic_reject?
>
> -Emile
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss