[rabbitmq-discuss] Job Server Resource Allocation Help

[Bryan Murphy]

On Wed, Oct 7, 2009 at 4:02 PM, Alexis Richardson <
alexis.richardson at gmail.com> wrote:

> Bryan
>
> I assume each job should be completed at most once.
>

Ideally, yes.  Don't want to run 2-3 hour jobs more than necessary. :)


> You can achieve something almost like what you want by creating two
> queues.  Use one for "paying" and one for "non-paying" jobs by binding
> each queue to the (direct) exchange using the keys "paying" and
> "nonpaying" respectively.  Assign N servers to the "paying" queue.
> Let the remainder consume from both queues using some selection scheme
> (eg worker randomly chooses a queue each time a new job is needed).
>

I was thinking along those lines, but wanted to make sure I wasn't missing
something obvious.  That will require some coding on our end that I was
hoping to avoid.

Many other schemes are possible.  However trying to achieve some total
> order (eg 'first come first serve') may be more complex than you
> really need.
>
> alexis
>
>
Total order isn't necessary, we just want to make a good faith effort to
serve those who's requests came first.

I have a related question... BasicConsume has a "filter" parameter.  It's
been a few months since I read the AMQP spec.  I took a quick look online,
but couldn't find any good documentation.  What does that do (if anything)
and how do you use it?

Ideally, what I was originally fishing for was something like this:

Client:
  headers["paying"] = "true";
  bus.Send(message, headers);

Server A:
  bus.Receive(queue="jobs", filter="paying == 'true'");

Server B:
  bus.Receive(queue="jobs" /* no filter */);

Thanks,
Bryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20091007/1e704aea/attachment.htm