[rabbitmq-discuss] RabbitMQ Implementation
phillpafford at gmail.com
Wed Jan 30 15:53:38 GMT 2013
Here is a little about the setup I have currently. (
- REST API to push ( POST ) data into a queue
- The Queue has a Consumer that's always running and Produces to en Exchange
- The Exchange routes to several other Queues ( like 20+ )
- Each of the ( 20+ ) Queues does a specific task ( The Consumers always
runs as well )
- Cron job runs to check if all the ( 20+ ) Tasks are completed and
Produces to yet another Queue
I'm not sure I like the Consumers running all the time as each Consumer
utilizes around 300MB of Ram ( I think it's MBs, it's not in front of me at
the moment ) and I'm looking for another implementation.
M <-- Message coming from REST API
| <-- The Exchange
/ | \
/ | \ <-- bind to multiple queues ( 20+ )
Q1 Q2 Q3 <-- Each Queue is a task that must be completed
| <-- CRON runs to check if all queues above have completed
Q4 <-- Queues 1,2 and 3 must finish first before Queue 4 can start
C <-- Consumer
I the related question below it was suggested to use RPC but the problem
with this is the RPC ( to my understanding ) Will have multiple instances.
This is a resource intense process as is and I think by adding the RPC
calls it will just bog down the server and then become unresponsive (
Please correct me if I'm wrong ).
Another approach was to use the Aggregator pattern
Which looks exactly what I need but I found the documentation limited. Has
anyone done this pattern?
My question is I'm not happy with how it's currently implemented and I'm
looking for ways to improve the process. I'm looking to either get rid of
the CRON, Implement a new Pattern and Not have the Consumers run all the
The process currently also only supports a single instance of each
consumer. It can have multiple consumers but how we have implemented it we
only wanted one at the time.
This is implemented in PHP, Symfony2 Framework using the RabbitMQBundle
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss