[rabbitmq-discuss] ANN New RabbitMQ Logstash plugin is ready for the final review

Thu Jul 4 15:16:17 BST 2013

Some time ago, I've started working on a new version of the RabbitMQ plugin
for Logstash.

It is now ready for the final review. If you are a Logstash user who wants to use
RabbitMQ as either Logstash input or output, please have a look:

https://github.com/logstash/logstash/pull/533

The new plugin solves a number of issues:

 * Events over 128K in size caused the output to fail
 * Throughput of both input and output was far below expectations on JRuby
 * Input configuration was needlessly complicated and confusing

finally, the previous plugin was effectively unmaintained.

To solve these problems, this new version does the following:

 * Uses Bunny 0.9 [1] on MRI
 * Uses HotBunnies 2.0 [2] on JRuby
 * [In near future] Reduces input configuration to what's really essential

This version network and RabbitMQ node recovery and makes ease of
operations a priority.

Because Logstash is primarily used with JRuby, you will likely see significant
(x5-6 times for the output, x2-3 times for input) throughput improvements
with HotBunnies. In addition, by tuning RabbitMQ channel settings and consumer
thread pool sizes, input throughput can be further increased by at least double
digit %.

I'd be happy to hear what you think: just leave a comment to the pull request.

Thanks!

1. http://rubybunny.info
2. http://hotbunnies.info
--
MK