[rabbitmq-discuss] Proposed change to shovel
fred at dushin.net
Fri Apr 19 14:20:59 BST 2013
On Apr 18, 2013, at 11:23 AM, Simon MacMullen <simon at rabbitmq.com> wrote:
> On 18/04/13 16:04, Fred Dushin wrote:
>> I would like to consider a change to the shovel plugin, to allow the
>> shovel to failover more predictably against the list of broker endpoints
>> in the rabbitmq_shovel configuration. Essentially, the idea is to try
>> each endpoint in order of the declaration, and to use the first endpoint
>> to which a connection can be made, instead of selecting an endpoint at
> Why? How is this better than the random selection?
In my case, I am pushing messages across WANs, potentially to different data centers, where some are treated as backups in the case where primaries go down. So having a list of destinations provides some resiliency in the case of network or hardware failure. I think in cases like this, operators need some predictability in where the messages are being sent.
>> This does slightly violate the spirit of Erlang's "let it fail"
>> philosophy, and I am not sure if the random endpoint selection in the
>> current code is really more designed for clustered environments.
> Yeah, that's the intention.
>> Perhaps it would make more sense to make the iterative failover
>> strategy an "opt-in" feature, if not simply for the sake of
> We could do that. But I'd like to see a reason for this feature to exist.
I agree that is adds complexity to a fairly tight bit of code, hence increases test complexity, imposes an maintenance burden, etc. So I can certainly understand the reluctance to fold it in. I am mostly curious if other users have the same set of requirements, or whether instead the source and destination lists are really more used in clustering scenarios. For all I know, I am abusing the use-case for which the multi-endpoint feature was designed.
> Cheers, Simon
> Simon MacMullen
> RabbitMQ, VMware
More information about the rabbitmq-discuss