[rabbitmq-discuss] Understanding Federation
James Carr
james.r.carr at gmail.com
Mon Feb 27 21:13:05 GMT 2012
FYI, here is the configuration I am using on the downstream server:
[
{rabbitmq_federation,
[ {exchanges, [[{exchange, "usage"},
{virtual_host, "/"},
{type, "topic"},
{durable, true},
{auto_delete, false},
{internal, false},
{upstream_set, "remotes"}]
]},
{upstream_sets, [{"remotes", [[{connection, "upstream-server"},
{exchange, "usage"},
{max_hops, 2}]
]}
]},
{connections, [{"upstream-server", [{host, "dc2rabbitmq1"},
{protocol, "amqp"},
{port, "5672"},
{virtual_host, "/"},
{username, "guest"},
{password, "guest"},
{mechanism, default},
{prefetch_count, 1000},
{reconnect_delay, 5},
{heartbeat, 1},
{queue_expires, 30000},
{message_ttl, 10000}
]}
]},
{local_username, "guest"},
{local_nodename, "rabbit"}
]
}
].
It creates the local exchange on the downstream node just fine, but no
action on the upstream (dc2rabbitmq1). A look at the error logs
reveals the below messages, but I don't completely grok erlang yet. :S
=ERROR REPORT==== 27-Feb-2012::15:01:20 ===
** Generic server <0.1963.0> terminating
** Last message in was connect
** When Server state == {state,amqp_network_connection,
{state,undefined,undefined,undefined,undefined,
undefined,false},
<0.1962.0>,undefined,
{amqp_params_network,<<"guest">>,<<"guest">>,
<<"/">>,"dc2rabbitmq1","5672",0,0,1,infinity,
none,
[#Fun<amqp_auth_mechanisms.plain.3>,
#Fun<amqp_auth_mechanisms.amqplain.3>],
[],[]},
undefined,undefined,
#Fun<amqp_connection_sup.0.39273983>,
#Fun<amqp_connection_sup.2.54430129>,false}
** Reason for termination ==
** {function_clause,[{inet_tcp,getserv,["5672"]},
{gen_tcp,connect1,4},
{gen_tcp,connect,4},
{amqp_network_connection,do_connect,5},
{amqp_gen_connection,handle_call,3},
{gen_server,handle_msg,5},
{proc_lib,init_p_do_apply,3}]}
=ERROR REPORT==== 27-Feb-2012::15:01:20 ===
** Generic server <0.1949.0> terminating
** Last message in was {'$gen_cast',maybe_go}
** When Server state == {not_started,
{{upstream,
{amqp_params_network,<<"guest">>,
<<"guest">>,<<"/">>,"dc2rabbitmq1",
"5672",0,0,1,infinity,none,
[#Fun<amqp_auth_mechanisms.plain.3>,
#Fun<amqp_auth_mechanisms.amqplain.3>],
[],[]},
<<"usage">>,1000,2,5,none,10000,
"upstream-server"},
{resource,<<"/">>,exchange,<<"usage">>}}}
** Reason for termination ==
** {{function_clause,[{inet_tcp,getserv,["5672"]},
{gen_tcp,connect1,4},
{gen_tcp,connect,4},
{amqp_network_connection,do_connect,5},
{amqp_gen_connection,handle_call,3},
{gen_server,handle_msg,5},
{proc_lib,init_p_do_apply,3}]},
{gen_server,call,[<0.1963.0>,connect,infinity]}}
=WARNING REPORT==== 27-Feb-2012::15:01:20 ===
Channel (<0.1958.0>): Unregistering confirm handler <0.1949.0> because
it died. Reason: {{function_clause,
[{inet_tcp,
getserv,
["5672"]},
{gen_tcp,
connect1,
4},
{gen_tcp,
connect,
4},
{amqp_network_connection,
do_connect,
5},
{amqp_gen_connection,
handle_call,
3},
{gen_server,
handle_msg,
5},
{proc_lib,
init_p_do_apply,
3}]},
{gen_server,
call,
[<0.1963.0>,
connect,
infinity]}}
Any help would be appreciated.
Thanks,
James
On Mon, Feb 27, 2012 at 2:12 PM, James Carr <james.r.carr at gmail.com> wrote:
> Sorry to hijack an existing thread, but I'm delving into using
> federation myself (after a 3 month break of even working with
> rabbitmq) and have a couple questions.
>
> 1) Does federation need to be installed on both upstream and downstream?
> 2) I have a configuration that matches what Denis has but it only
> creates the exchange on the downstream server and does nothing
> upstream (only difference between his and mine is the remote hostname,
> of course). What am I missing?
>
>
> Thanks,
> James
>
>
> On Thu, Feb 23, 2012 at 10:47 AM, Simon MacMullen <simon at rabbitmq.com> wrote:
>> On 23/02/12 16:36, MELIQUE Denis (MORPHO) wrote:
>>>
>>> Thank you Simon.
>>>
>>> This makes it much clearer for me.
>>> I was first surprised to see this queue always empty on my test bed,
>>> but the reason is (correct me if I'm wrong)
>>> that my system was not loaded enough, and consumers got messages as soon
>>> as they came in.
>>
>>
>> You're partly right. Your system was not loaded enough, but the messages in
>> that queue will be consumed by the downstream exchange, not (directly) by
>> your consumers.
>>
>>
>>> So every message consumed from the downstream is first written on disk.
>>
>>
>> No. The queue is durable, but that just determines whether it survives a
>> restart. Transient messages can still through it, and remain transient.
>>
>> Cheers, Simon
>>
>>
>>> This might be a disadvantage for performance.
>>> And gives an advantage to Shovel, as you can configure it with transient
>>> queues.
>>>
>>> Denis
>>>
>>> -----Original Message-----
>>> From: Simon MacMullen [mailto:simon at rabbitmq.com]
>>> Sent: Thursday, February 23, 2012 12:11 PM
>>> To: MELIQUE Denis (MORPHO)
>>> Cc: rabbitmq-discuss at lists.rabbitmq.com
>>> Subject: Re: [rabbitmq-discuss] Understanding Federation
>>>
>>> On 23/02/12 10:56, MELIQUE Denis (MORPHO) wrote:
>>>>
>>>> I’m doing some testing with federation plug-in to try to understand it.
>>>
>>>
>>> Cool!
>>>
>>> <snip>
>>>
>>>> What are these Exchange and Queues used for ?
>>>
>>>
>>> The queue is there to buffer messages that are published to the upstream
>>> exchange and have not yet been consumed from the downstream exchange.
>>>
>>> It is bound to the internal exchange with the union of all the bindings
>>> that are in use downstream, so we only transfer messages that are going
>>> to be of use downstream.
>>>
>>> The internal exchange is there so that whenever we (re)establish a
>>> federation link we can set up an exact set of bindings
>>> mostly-atomically, by creating a new internal exchange, creating all the
>>> bindings to it, then binding it to the upstream exchange.
>>>
>>>> Are they reserved for federation internal use ?
>>>
>>>
>>> Yes.
>>>
>>>> When do messages go into these queues ?
>>>
>>>
>>> As mentioned, the bindings are pushed upstream so messages from upstream
>>> go into these queues if they will be routed once they get downstream.
>>>
>>>> Do they have to be durable or is there a way to configure them as
>>>> transient ?
>>>
>>>
>>> At the moment they have to be durable. Is this a problem? The idea is
>>> that you can set the queue TTL if you do not want them to hang around
>>> forever.
>>>
>>>> I would be very interested by a brief description of how federation
>>>> works internally.
>>>
>>>
>>> Hopefully this answers your question.
>>>
>>> Cheers, Simon
>>>
>>
>>
>> --
>> Simon MacMullen
>> RabbitMQ, VMware
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
More information about the rabbitmq-discuss
mailing list