[rabbitmq-discuss] How to have multiple consumers for a Queue in Ruby

[Jakub Šťastný]

Rajat,

about the second example, why more instances should be a problem? Having
more subscribers in the same process doesn't have much sense, in my opinion,
since the reactor pattern doesn't block -> you don't have to wait -> you
don't need to use threads for parallelisation. Threads can be an advantage,
but I don't think it'll be a big difference. Plus Ruby has GIL which means
threads are sort of useless anyway, well, especially if you use Ruby 1.8
which still has green threads. I'd say if you want to go this route, use
Kernel#fork, but then you have to be careful about consistency of state.
Breaking logic down sounds like a much better & cleaner way to go in my
opinion.

Jakub

http://www.flickr.com/photos/jakub-stastny
http://twitter.com/botanicus



2011/2/23 Rajat Vig <rajat.vig at gmail.com>

> Thanks for the suggestions. I'd the same thing in mind about being able to
> start multiple of those processes.
>
> There are 2 different kinds of consumers
> 1. Get the message and write it to a database or send it out to someplace
> else.
> Parallelizing this would be easy. Controlling would be more of a problem
> here but manageable.
>
> 2. There's one large consumer which accepts different kinds of messages.
> Like start/stop/abandon a game (which are always singular) and messages for
> the game itself which are many.
> I don't want to run this one many times since it might just run into
> conflicts (though I can't see many currently - it does impose a rigid
> discipline in the code).
> The alternative that I'm thinking here is to internally re-route by
> (generated) random keys and have multiple subscribers in the same
> process. Would that work fine?
> It not, I think breaking it further to a consumer for all admin messages
> and multiple consumers for the game plays would be the way forward but it'll
> be a significant refactor.
>
> -Rajat
>
> 2011/2/22 Jakub Šťastný <stastny at 101ideas.cz>
>
> Hi Rajat, could you maybe provide some more info? In general I'd agree with
>> Jim, although it depends, there are probably other solutions as well (like
>> more queues -> more granular design of the app with more consumers).
>>
>> Jakub
>>
>> http://www.flickr.com/photos/jakub-stastny
>> http://twitter.com/botanicus
>>
>>
>>
>> On 22 February 2011 17:43, Rajat Vig <rajat.vig at gmail.com> wrote:
>>
>>> Hi
>>>
>>> We've written an application using the amqp gem with RabbitMQ.
>>> Currently the bottleneck we've run into is that if there are too many
>>> message incoming then processing the queue takes long.
>>> After reading some messages on the list, I think allowing multiple
>>> consumers is the way forward.
>>>
>>> How do I do that? And are there any pitfalls/gotchas?
>>>
>>> -Rajat
>>>
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-discuss at lists.rabbitmq.com
>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110223/972bb3b6/attachment.htm>