[rabbitmq-discuss] Shovel restart options

Thu Dec 13 12:54:21 GMT 2012

I am using the shovel plugin to move messages from several queues on one broker to corresponding exchanges on another, and for the most part this works beautifully.

I would like to be able to reconfigure the shovel plugin without restarting the message broker, however, as I want as high uptime on the broker as possible (and in some scenarios I do not want to mandate use of persistent messages (performance, I/O subsystem already under pressure, etc.).

Semantically, on the broker, I want to be able to do the following (given a set of terms bound to Shovel):

application:stop(rabbitmq_shovel).
application:set_env(rabbitmq_shovel, shovels, Shovels).
application:start(rabbitmq_shovel).

(I can also persist the changes in rabbitmq.config for subsequent restarts)

As far as I can see, my options are to:

1. Invoke the above commands through rabbitctl eval 'term', or a sequence thereof;
2. Implement a plugin which invokes the above, and communicate to the plugin through some mechanism (mochiweb, AMQP, etc).
3. Deploy the shovel plugin in a separate abstract machine (beam) and simply restart that VM. 

The first option has obvious drawbacks and is almost a non-starter (though of course it is the first thing I have prototyped :) -- the default limit on a command line argument on most of the platforms I am using is something like 250k, and I have no control over the size of the Shovels term.

The second is better, as the plumbing is all there to do it, and it seems like the most robust solution.

The third is viable, but it requires a separate point of management.

Have others on this list come up with any alternatives?

Thanks,

Fred Dushin