<div dir="ltr">You can use RabbitMQ to do this - there's nothing that says you can't. Whether it's the right solution depends on the complexity of the overall architecture, goals in that architecture, etc. My initial thought was "memcached" that both process and rest server publish and read from. In our case due to our requirements, we use a shoveled set of rabbit queues to transmit data changes to an "enterprise system" in a different geographical location the the source rabbit servers. This enterprise system is where the "source of truth" data is stored as well as for tracking information and other data. The AMQP consumers after updating the central database, publishes changes back out to the various locations for high performance read only use. If we hadn't had the need to do distributed data publishes from multiple locations, required support for networking failures/interrupts, wanted high performance and minimal impact to send such messages, we'd have just had local processes talking to local replicated/clustered databases instead. Our current architecture gives us a distributed HA asynchronous processing system. There are always, always questions though - how durable is the job ids? If a publishing fails and your app fails, do you just rerun, etc. etc. etc.<div>
<br></div><div><div style>Jason</div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Jun 11, 2013 at 2:06 PM, Robert DiFalco <span dir="ltr"><<a href="mailto:robert.difalco@gmail.com" target="_blank">robert.difalco@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I am writing a REST server in Java and am trying to come up with the best approach for scalability. The server is deployed on Heroku. I am new to RabbitMQ/AMQ so I need some advice.<div>
<br></div><div>From an architectural point of view I want to have one or more REST dynos running that delegate all of their work to one or more worker dynos. The REST server dynos must be stateless. </div><div><br></div><div>
I have two basic patterns to support.</div><div><br></div><div><ol><li><span style="line-height:normal">REST RPC. Basically, the client makes a call like GET /user/{id}. I delegate this call to a worker dyno that does a database lookup in the USER table for the specified ID and returns it. The REST interface then returns this. This to me seems like basic RPC. </span></li>
<li><span style="line-height:normal">REST ASYNC. In this model the client may call my REST server with a POST or a GET. Let's use the classic case of a long running task to format a graphic. I will create a JOB ID, return it to the client in an URI. The client calls the URI to poll the job until it is done and then eventually gets redirected to a URI that has the result. In this case, when I get the initial REST call I will generate a UID for it, submit it to the worker. There are several ways I can do the polling and returning of the result. A lot of that depends on how I decide to interact with the worker. Is it RPC? AKKA? AMQP? Because there may be more than one REST service dyno, the request to post the job may happen on one REST server and the polling request may be routed to another. So it must be stateless.</span></li>
</ol><div><br>So here's my question. First, should I be using RabbitMQ for this or is there a better solution? If I am using RabbitMQ must I implement the RPC and ASYNC patterns myself directly on top of RabbitMQ or are there libraries out there that create a layer of abstraction on top of RabbitMQ so I can do both of these simply? If I'm going to use RabbitMQ I'm hoping to use CloudAMQP on Heroku.</div>
</div><div><br></div><div>Thanks!</div><span class="HOEnZb"><font color="#888888"><div><br></div><div>Robert</div></font></span><br>_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br>Jason McIntosh<br><a href="http://mcintosh.poetshome.com/blog/">http://mcintosh.poetshome.com/blog/</a><br>573-424-7612
</div>