[rabbitmq-discuss] Proposed change to shovel

[Laing, Michael P.]

We use shovels and clustering extensively and prefer the current behavior.

Our architecture is 'headless' - every component is primary - our testing
and limited production rollout have demonstrated very resilient, adaptive
behavior.

-Michael

On 4/19/13 9:20 AM, "Dushin Fred" <fred at dushin.net> wrote:

>
>On Apr 18, 2013, at 11:23 AM, Simon MacMullen <simon at rabbitmq.com> wrote:
>
>> On 18/04/13 16:04, Fred Dushin wrote:
>>> I would like to consider a change to the shovel plugin, to allow the
>>> shovel to failover more predictably against the list of broker
>>>endpoints
>>> in the rabbitmq_shovel configuration.  Essentially, the idea is to try
>>> each endpoint in order of the declaration, and to use the first
>>>endpoint
>>> to which a connection can be made, instead of selecting an endpoint at
>>> random.
>> 
>> Why? How is this better than the random selection?
>
>In my case, I am pushing messages across WANs, potentially to different
>data centers, where some are treated as backups in the case where
>primaries go down.  So having a list of destinations provides some
>resiliency in the case of network or hardware failure.  I think in cases
>like this, operators need some predictability in where the messages are
>being sent.
>
>> 
>>> This does slightly violate the spirit of Erlang's "let it fail"
>>> philosophy, and I am not sure if the random endpoint selection in the
>>> current code is really more designed for clustered environments.
>> 
>> Yeah, that's the intention.
>> 
>>>  Perhaps it would make more sense to make the iterative failover
>>> strategy an "opt-in" feature, if not simply for the sake of
>>> backwards-compatibility.
>> 
>> We could do that. But I'd like to see a reason for this feature to
>>exist.
>
>I agree that is adds complexity to a fairly tight bit of code, hence
>increases test complexity, imposes an maintenance burden, etc.  So I can
>certainly understand the reluctance to fold it in.  I am mostly curious
>if other users have the same set of requirements, or whether instead the
>source and destination lists are really more used in clustering
>scenarios.  For all I know, I am abusing the use-case for which the
>multi-endpoint feature was designed.
>
>-Fred
>
>
>> 
>> Cheers, Simon
>> 
>> -- 
>> Simon MacMullen
>> RabbitMQ, VMware
>
>_______________________________________________
>rabbitmq-discuss mailing list
>rabbitmq-discuss at lists.rabbitmq.com
>https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss