[rabbitmq-discuss] How to implement the Aggregator pattern?

Fri Jun 25 19:57:51 BST 2010

Matthew,

I think you interpreted my question correctly. A and B are actually
calculations on specific messages. C requires the result of the calcs
of A and B in order to do it's work (another calculation). If I do as
you suggested, then that seems to imply that I need to manage the
messages based on the content to correctly group them. Otherwise, the
calc of C might include calc results of arbitrary messages not related
to it. Based on what I've read, this generally means that a
persistence mechanism needs to be in place in order to hold the
grouping state of messages. Does this sound accurate?

I should also mention that C may or may not be necessary based on the
message content. So the flow may look something like this:

data(x): A, B
data(y): A, B, C

So in the case of data(x), C is not needed and in the case of data(y)
C needs the results of A and B.

Thanks,
Shane

On Jun 25, 8:54 am, Matthew Sackman <matt... at rabbitmq.com> wrote:
> Hi Shane,
>
> On Wed, Jun 23, 2010 at 07:31:42AM -0700, Shane wrote:
> > I'm trying to determine the best way to implement the Aggregator
> > pattern with RabbitMQ. For example, given consumers A, B and C, I need
> > to run C with the results of A and B.
>
> If you have a consumer A and B, are you saying that they're actually
> doing some transformation of the message(s), publishing that, and then C
> needs to receive one message from A and one message from B before
> starting any work?
>
> If so, I suggest that A and B both publish the transformed message,
> maybe via different exchanges, but that each message goes to a different
> queue. C consumes from both queues, but sets qos prefetch to 1, and does
> not ack any message until it has received one message from each queue.
> If you want to be uber purist, you would wrap both of those acks in a
> transaction to have them treated atomically.
>
> But that's a fair guess from your original description so I might be
> solving the wrong problem here!
>
> Matthew
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss