[rabbitmq-discuss] Using Disruptor in web app

[oscar]

Hi all,

Firstly this is awesome stuff and I want to use it in my next project,
which we be run in a web server.  I'm a little worried about trying to
architect my project for performance based around using the disruptor
and getting worse performance than normal due to not understanding it
well enough.  I would really appreciate some input.

The project will be hosted so I won't be able to simplify by having
all data in memory (I think).  It will be a fairy traditional (boring)
project, order entry processing (and other stuff).  So some screens to
view orders, products query for them edit/maintain CRUD type stuff.
So latency isn't important in itself but only as a means to support
the maximum number of users for a smallish server with between 1.7 -17
Gb of memory.

1) Is the disruptor for me?  (I hope yes)
2) When designing the ringbuffer events, how concerned should I be
that they don't change in terms of memory location as they are
mutated.  For instance if I use strings in the event than these will
be re-allocated on each mutation.  Should I use say StringBuilders or
even something like protobuf messages and have the event type as a
byte array and marshall each event into that byte array  (handling for
overflow) I see this as trading memory striding against cpu overhead
and short lived object creation (new strings etc) in the marshalling/
demarshalling process?
3) Since each event will be marshalled from an http request I'm not
sure how large to make the event.  Should it be large so that the ring
buffer has fewer slots but the chance of an oversize object (won't fit
in preallocated event slot) is very small?
4) Should I have just one input ringbuffer with an event definition
that can handle any event (http request) or multiple input ring
buffers?  I think that LMAX had just one.
5) What should the overall process flow be?  an http request will come
in and my web container thread will quickly marshall it into an event
and put in on the ring buffer.  Consumers will read it it and say
fetch a customer from the DB and update the event.  Finally my
business processor (final single threaded beast of a consumer) will
produce an http response.  What's the best/standard way of
communicating that back the original thread that orginated the
request?  via a Latch that's put onto the event?

6) Lastly is there a standard pattern / best practice architecture to
be adopted for a CRUD like web app running in a webserver using the
Disruptor.  I feel like what I'm doing is so standard that the ought
to be a right way to do it.  I'm thinking right now that I would only
need one RingBuffer being fed by the webserver request threads and
then various consumers decorating that,  or do I need another one on
the output side that the business processor is the producer to?

Hope I'm not being dense, really appreciate any advice you can give on
any of the points I've raised.