On Tue, Jan 4, 2011 at 2:12 PM, andrew chase <span dir="ltr"><<a href="mailto:andrew@glyde.com">andrew@glyde.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Most of your bullet points are outside the realm (IMO) of what a<br>
message system should be concerned with. In particular, it looks like<br>
most of the features you're after are more in line with what a<br>
background job processing system would handle. Those types of systems<br>
often use messaging under the covers but often also use some type of<br>
persistent data store in addition (or instead of) a messaging system.<br>
I'd suggest taking a look at a couple of these systems to see if they<br>
may be more what you're after (I know a couple ruby-based ones of the<br>
top of my head like Resque, DelayedJob, etc<br>
<a href="https://github.com/defunkt/resque" target="_blank">https://github.com/defunkt/resque</a>).<br></blockquote><div><br></div><div>Thanks Andrew,</div><div><br></div><div>I spent some time looking at those tonight and came full circle. Found a post where Resque was used as a replacement for DJ and then in the post's comments people suggesting RabbitMQ... ;) I'll keep reading, though. I have plenty to learn.</div>
<div><br></div><div>I think our organization has been in the habit of building somewhat monolithic code and are looking for a solution that does everything as we do it now. I'm concerned we will attempt to build our own if we can't find a perfect fit, which I suspect is a much harder task to get right than we think. As for scheduling, we already have a database table with the scheduled items, so maybe we only need a few producers that run from cron.</div>
<div><br></div><div><br></div><div>I had two slightly more RabbitMQ-specific questions that are hopefully more straight forward to answer. </div><div><br></div><div>When a job is re-queued due to failure to ACK is there anything that indicates to the next consumer that the message has already been delivered to a failed consumer? I need a way to recognize that a message is causing consumers to die and stop the job.</div>
<div><br></div><div>And is it a common pattern for consumers to use RabbitMQ to message back that their work is done? I was wondering if that's a good use for x-expires and/or x-message-ttl.</div><div><br></div><div>
Thanks again,</div><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div><div class="h5">On Tue, Jan 4, 2011 at 1:39 PM, Bill Moseley <<a href="mailto:moseley@hank.org">moseley@hank.org</a>> wrote:<br>
> Anyone have time to take a few of these on?<br>
> I know they are not specifically on-topic (and a bit open-ended) but we are<br>
> evaluating messaging solutions (and even discussing writing our own) so it<br>
> would be a huge help to get some feedback. I'd be happy to provide more<br>
> specifics if needed.<br>
> Thanks,<br>
><br>
> On Mon, Jan 3, 2011 at 3:13 PM, Bill Moseley <<a href="mailto:moseley@hank.org">moseley@hank.org</a>> wrote:<br>
>><br>
>> Hi,<br>
>> I'm just starting out with RabbitMQ and trying to wrap my head around a<br>
>> few concepts. I apologize if I ask something that should be obvious or I<br>
>> missed in the docs and faq. I'm hoping someone can help with a few<br>
>> scenarios, and please point me to further reading as needed.<br>
>> I'm curious about use-cases that are not directly related to RabbitMQ, but<br>
>> how to best use RabbitMQ in an web application's architecture. Thes are<br>
>> probably very common situations.<br>
>> 1) Notification of job status and completion: If the web app produces a<br>
>> request to generate an expensive PDF is there a common approach to detecting<br>
>> when the consumer is completed and the pdf is ready for downloading? Should<br>
>> I create a single-use return queue with x-expires and/or x-message-ttl on<br>
>> the queue? In a web app the web user may never come back to pick up the<br>
>> PDF.<br>
>> 2) Job sequencing: What's a common approach to multiple step processing<br>
>> that must be done serially? I.e. queue job "A" and when it's complete queue<br>
>> job "B". Doesn't really seem like consumer "A" should be responsible for<br>
>> submitting job "B". Is it better to create a consumer "AB" that submits "A"<br>
>> and waits for a message that "A" is complete and then submits job "B"?<br>
>> 3) Scheduling: I assume there's no facility for scheduling jobs to be<br>
>> delivered to consumers at some future time. Is there a better approach than<br>
>> using cron to check the database for jobs that should be queued based on<br>
>> time?<br>
>> 4) Cancel a request: We have some situations when a subsequent request<br>
>> should cancel processing of an earlier request. (Frankly, I think this is a<br>
>> design problem on our end.)<br>
>> 5) Prevent duplicate job requests: In a web environment when things break<br>
>> problems often compound by users reloading requests that caused the problem<br>
>> in the first place. I'm not sure RabbitMQ would be involved at all in this<br>
>> scenario, but I'm asking just in case. ;)<br>
>> Finally, it's great that RabbitMQ will send a message again if is is not<br>
>> ACK'd and a consumer closes its connection. But, is there any kind of max<br>
>> retry count available? That is, what's a good approach to prevent an errant<br>
>> job from killing all consumers one-by-one?<br>
>> Thanks,<br>
>> --<br>
>> Bill Moseley<br>
>> <a href="mailto:moseley@hank.org">moseley@hank.org</a><br>
><br>
><br>
><br>
> --<br>
> Bill Moseley<br>
> <a href="mailto:moseley@hank.org">moseley@hank.org</a><br>
><br>
</div></div>> _______________________________________________<br>
> rabbitmq-discuss mailing list<br>
> <a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>
> <a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
><br>
><br>
</blockquote></div><br><br clear="all"><br>-- <br>Bill Moseley<br><a href="mailto:moseley@hank.org" target="_blank">moseley@hank.org</a><br>