<br><br><div class="gmail_quote">On Mon, Feb 14, 2011 at 7:12 AM, Jonathan Brisbin <span dir="ltr"><<a href="mailto:jbrisbin@vmware.com">jbrisbin@vmware.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Would anyone like to express an interest in a particular topic for a<br>
potential RabbitMQ cookbook project I might be working on? We're<br>
trying to come up with an outline and I want to make sure it's<br>
targeted to what people actually want to read about, so if you've got<br>
an idea about some topic you'd like to see covered, please let me know<br>
off list.<br>
<br>
Some of the topics will be:<br>
<br>
Interoperability (STOMP, Celery, etc...)<br>
Performance tuning<br>
Plugin authoring<br>
Configuration<br>
[Put your topic here]<br>
<br></blockquote><div><br></div><div>These are a few of the topics that have come up at my company while evaluating RabbitMQ -- some of which I've asked about on this list. I'm not sure if all would fit into "cookbook" type of examples. Most are about how to operate RabbitMQ in a production environment.</div>
<div><ul><li>High Availability: How to set up RabbitMQ such that a server failure will not disrupt service.<br>For example, HAProxy or hardware load balancer.<br><br></li><li>Client connection management: Ensuring that clients gracefully handle connection failures.<br>
<br></li><li>Message durability: Scenarios when/why queue durability is appropriate.<br><br></li><li>Message replication: When/why/how to replicate messages across brokers (e.g. Pacemaker example).<br>And if messages are replicated how to prevent multiple consumers from processing the same message.<br>
<br></li><li>Clustering: As others have noted, how to scale RabbitMQ.<br>How to add new servers into an active cluster and how to remove a server (and dealing with queues on the server that is removed).<br></li></ul><div>
<br></div><div>As you might note, uptime is important to our company and those are all about keeping RabbitMQ up and available. Oddly, at our company the message durability/replication issue consumed the most discussion time -- that is, making sure that a message produced is always processed[1]. (I think a better/cleaner approach is to design with message loss in mind and when needed, producers will re-send a message.)</div>
</div><div><br></div><div><ul><li>Topic routing: This is covered in the tutorial, but it would be great to see some cookbook examples that demonstrate a few real-world scenarios.<br>The power of topic routing was a feature that was difficult for some to grasp at our company. Aggregating logs and tracking specific log types (e.g. error watchdog) could be one example.<br>
<br></li><li>Debugging: When things go wrong (and they always do) steps to figure out why RabbitMQ is not working, what happened, and how to fix.<br>For example: figuring out if Mnesia DB is corrupt and need to remove. This is all about the 2am IT on-call person trying to solve a problem by following a "play book" of steps to get things running again.<br>
<br></li><li>Management of consumers: monitoring queue sizes, allocating resources to workers, tools available to monitor consumers across multiple machines, threaded consumers vs. forking model.<br><br></li><li>RPC in a web environment: This may be outside the scope for RabbitMQ, but our use cases have involved the need for RPC style of calls, but in a web-environment this is often meant polling by the web client (e.g. via AJAX) because it is not desirable to have a web process wait on a RPC response. But, in a polling web environment you cannot rely on the message ever being consumed. What I've done is passed the unique return-queue name in the AJAX request and used auto_delete and x-expires on the temporary queue to have it removed if the web client fails to fetch the response.<br>
I'm not convinced that is the best approach.</li></ul></div><div><br></div><div><br></div><div><br></div><div>[1] Message durability was considered so important by some at our company that we now send all messages to a single consumer that stores them in Postgresql with Slony replication before being passed onto consumers. I would not recommend that approach -- and indeed with experience we've seen messages that are that critical need to be tracked by the application's database anyway so making the messages durable adds little benefit.</div>
</div><br>-- <br>Bill Moseley<br><a href="mailto:moseley@hank.org" target="_blank">moseley@hank.org</a><br>