[rabbitmq-discuss] Web use scenarios

[Alexis Richardson]

Andrew

We agree that databases and queues are different in that queues can be
seen as data for FIFO, streamed, delivery, whereas databases support
more general queries.

Rabbit is currently FIFO which means that other models have to be
supported at the application layer, eg by using clever routing
patterns, clients with timers, and so on.

Projects like Celery are worth understanding here.

alexis


On Tue, Jan 4, 2011 at 10:12 PM, andrew chase <andrew at glyde.com> wrote:
> Most of your bullet points are outside the realm (IMO) of what a
> message system should be concerned with. In particular, it looks like
> most of the features you're after are more in line with what a
> background job processing system would handle. Those types of systems
> often use messaging under the covers but often also use some type of
> persistent data store in addition (or instead of) a messaging system.
> I'd suggest taking a look at a couple of these systems to see if they
> may be more what you're after (I know a couple ruby-based ones of the
> top of my head like Resque, DelayedJob, etc
> https://github.com/defunkt/resque).
>
> When it comes to guaranteeing job completion, request cancellation,
> etc, it's nice to use a persistent data store and if speed is your
> concern there are a host of new non-RDBMS solutions out there that
> would probably serve your needs well.
>
> Maybe other rabbitmq folks disagree?
>
> Cheers,
> Andrew
>
>
> On Tue, Jan 4, 2011 at 1:39 PM, Bill Moseley <moseley at hank.org> wrote:
>> Anyone have time to take a few of these on?
>> I know they are not specifically on-topic (and a bit open-ended) but we are
>> evaluating messaging solutions (and even discussing writing our own) so it
>> would be a huge help to get some feedback.  I'd be happy to provide more
>> specifics if needed.
>> Thanks,
>>
>> On Mon, Jan 3, 2011 at 3:13 PM, Bill Moseley <moseley at hank.org> wrote:
>>>
>>> Hi,
>>> I'm just starting out with RabbitMQ and trying to wrap my head around a
>>> few concepts.  I apologize if I ask something that should be obvious or I
>>> missed in the docs and faq.  I'm hoping someone can help with a few
>>> scenarios, and please point me to further reading as needed.
>>> I'm curious about use-cases that are not directly related to RabbitMQ, but
>>> how to best use RabbitMQ in an web application's architecture.  Thes are
>>> probably very common situations.
>>> 1) Notification of job status and completion:  If the web app produces a
>>> request to generate an expensive PDF is there a common approach to detecting
>>> when the consumer is completed and the pdf is ready for downloading?  Should
>>> I create a single-use return queue with x-expires and/or x-message-ttl on
>>> the queue?   In a web app the web user may never come back to pick up the
>>> PDF.
>>> 2) Job sequencing:  What's a common approach to multiple step processing
>>> that must be done serially?  I.e. queue job "A" and when it's complete queue
>>> job "B".  Doesn't really seem like consumer "A" should be responsible for
>>> submitting job "B".  Is it better to create a consumer "AB" that submits "A"
>>> and waits for a message that "A" is complete and then submits job "B"?
>>> 3) Scheduling: I assume there's no facility for scheduling jobs to be
>>> delivered to consumers at some future time.  Is there a better approach than
>>> using cron to check the database for jobs that should be queued based on
>>> time?
>>> 4) Cancel a request:  We have some situations when a subsequent request
>>> should cancel processing of an earlier request.  (Frankly, I think this is a
>>> design problem on our end.)
>>> 5) Prevent duplicate job requests: In a web environment when things break
>>> problems often compound by users reloading requests that caused the problem
>>> in the first place.  I'm not sure RabbitMQ would be involved at all in this
>>> scenario, but I'm asking just in case. ;)
>>> Finally, it's great that RabbitMQ will send a message again if is is not
>>> ACK'd and a consumer closes its connection.  But, is there any kind of max
>>> retry count available?  That is, what's a good approach to prevent an errant
>>> job from killing all consumers one-by-one?
>>> Thanks,
>>> --
>>> Bill Moseley
>>> moseley at hank.org
>>
>>
>>
>> --
>> Bill Moseley
>> moseley at hank.org
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>
>>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>