[rabbitmq-discuss] Performance Hit when using Federation

[Eric Cozzi]

Wouldn't republishing more so affect latency, not throughput? I'd expect 
that the federated brokers should be able to republish the messages 
nearly as quickly as they received them. Sure there is going to be 
processing time to perform the network read, queue and re-send the 
message, but that shouldn't affect 'how many' messages a federated 
broker can send in a second?

One of the use cases that we're look at is being able to make an RPC 
call where the client and server are potentially on disparate, 
non-routable networks. Essentially, there would only be a few gateway 
nodes that would span both networks. We're looking at installing 
RabbitMQ on these gateway nodes, and at other key points in our system. 
Thus, we need to be able to support RPC across several intermediate 
RabbitMQ brokers. Federation seemed a natural fit, but is there 
another/better option? The shovel plugin doesn't look to me like it 
would easily fill our need, if I'm not mistaken.

As for having a mode where federation optimized speed over reliability, 
that might be interesting to us.

Eric

On 01/30/2013 11:59 AM, Simon MacMullen wrote:
> On 30/01/13 17:26, Eric Cozzi wrote:
>> Hello all,
>>
>> I am investigating RabbitMQ for a project where we would need to
>> federate multiple brokers. From my testing, I'm seeing acceptable
>> throughput results as long as the server and client are bot talking to
>> the same RabbitMQ server instance. However, as soon as I have the Server
>> and Client talking to different servers using a federated exchange, I'm
>> only seeing 1/4 of the throughput. Is this expected?
>
> Well, federation does require that the message be published on the 
> source broker, consumed and republished by the federation plugin and 
> finally consumed for real. So I would expect 1/2 the throughput for a 
> simple Producer -> Broker A -> Broker B -> Consumer case.
>
> Where federation *helps* throughput is when most of your messages can 
> be local. Ultimately if every message has to go everywhere, you have a 
> scaling problem anyway :-)
>
> However, I wouldn't expect 1/4 the throughput... except that 
> federation does transfer messages using confirms and acks. If your 
> publishers and consumers are *not* using confirms / acks then they 
> could go faster enough that the federation step in the middle would be 
> that much more expensive. Does that sound like what might be happening?
>
> And would anyone be interested in an optional mode for federation to 
> disable confirms and acks, going faster at the expense of guaranteed 
> delivery?
>
> Cheers, Simon
>