[rabbitmq-discuss] Understanding Federation
James Carr
james.r.carr at gmail.com
Mon Feb 27 21:48:22 GMT 2012
Nvm, I got it working.
On Mon, Feb 27, 2012 at 3:13 PM, James Carr <james.r.carr at gmail.com> wrote:
> FYI, here is the configuration I am using on the downstream server:
>
> [
> {rabbitmq_federation,
> [ {exchanges, [[{exchange, "usage"},
> {virtual_host, "/"},
> {type, "topic"},
> {durable, true},
> {auto_delete, false},
> {internal, false},
> {upstream_set, "remotes"}]
> ]},
> {upstream_sets, [{"remotes", [[{connection, "upstream-server"},
> {exchange, "usage"},
> {max_hops, 2}]
> ]}
> ]},
> {connections, [{"upstream-server", [{host, "dc2rabbitmq1"},
> {protocol, "amqp"},
> {port, "5672"},
> {virtual_host, "/"},
> {username, "guest"},
> {password, "guest"},
> {mechanism, default},
> {prefetch_count, 1000},
> {reconnect_delay, 5},
> {heartbeat, 1},
> {queue_expires, 30000},
> {message_ttl, 10000}
> ]}
> ]},
> {local_username, "guest"},
> {local_nodename, "rabbit"}
> ]
> }
> ].
>
>
> It creates the local exchange on the downstream node just fine, but no
> action on the upstream (dc2rabbitmq1). A look at the error logs
> reveals the below messages, but I don't completely grok erlang yet. :S
>
> =ERROR REPORT==== 27-Feb-2012::15:01:20 ===
> ** Generic server <0.1963.0> terminating
> ** Last message in was connect
> ** When Server state == {state,amqp_network_connection,
> {state,undefined,undefined,undefined,undefined,
> undefined,false},
> <0.1962.0>,undefined,
> {amqp_params_network,<<"guest">>,<<"guest">>,
> <<"/">>,"dc2rabbitmq1","5672",0,0,1,infinity,
> none,
> [#Fun<amqp_auth_mechanisms.plain.3>,
> #Fun<amqp_auth_mechanisms.amqplain.3>],
> [],[]},
> undefined,undefined,
> #Fun<amqp_connection_sup.0.39273983>,
> #Fun<amqp_connection_sup.2.54430129>,false}
> ** Reason for termination ==
> ** {function_clause,[{inet_tcp,getserv,["5672"]},
> {gen_tcp,connect1,4},
> {gen_tcp,connect,4},
> {amqp_network_connection,do_connect,5},
> {amqp_gen_connection,handle_call,3},
> {gen_server,handle_msg,5},
> {proc_lib,init_p_do_apply,3}]}
>
> =ERROR REPORT==== 27-Feb-2012::15:01:20 ===
> ** Generic server <0.1949.0> terminating
> ** Last message in was {'$gen_cast',maybe_go}
> ** When Server state == {not_started,
> {{upstream,
> {amqp_params_network,<<"guest">>,
> <<"guest">>,<<"/">>,"dc2rabbitmq1",
> "5672",0,0,1,infinity,none,
> [#Fun<amqp_auth_mechanisms.plain.3>,
> #Fun<amqp_auth_mechanisms.amqplain.3>],
> [],[]},
> <<"usage">>,1000,2,5,none,10000,
> "upstream-server"},
> {resource,<<"/">>,exchange,<<"usage">>}}}
> ** Reason for termination ==
> ** {{function_clause,[{inet_tcp,getserv,["5672"]},
> {gen_tcp,connect1,4},
> {gen_tcp,connect,4},
> {amqp_network_connection,do_connect,5},
> {amqp_gen_connection,handle_call,3},
> {gen_server,handle_msg,5},
> {proc_lib,init_p_do_apply,3}]},
> {gen_server,call,[<0.1963.0>,connect,infinity]}}
>
> =WARNING REPORT==== 27-Feb-2012::15:01:20 ===
> Channel (<0.1958.0>): Unregistering confirm handler <0.1949.0> because
> it died. Reason: {{function_clause,
>
> [{inet_tcp,
>
> getserv,
>
> ["5672"]},
>
> {gen_tcp,
>
> connect1,
>
> 4},
>
> {gen_tcp,
>
> connect,
>
> 4},
>
> {amqp_network_connection,
>
> do_connect,
>
> 5},
>
> {amqp_gen_connection,
>
> handle_call,
>
> 3},
>
> {gen_server,
>
> handle_msg,
>
> 5},
>
> {proc_lib,
>
> init_p_do_apply,
>
> 3}]},
>
> {gen_server,
>
> call,
>
> [<0.1963.0>,
>
> connect,
>
> infinity]}}
>
> Any help would be appreciated.
>
> Thanks,
> James
>
>
> On Mon, Feb 27, 2012 at 2:12 PM, James Carr <james.r.carr at gmail.com> wrote:
>> Sorry to hijack an existing thread, but I'm delving into using
>> federation myself (after a 3 month break of even working with
>> rabbitmq) and have a couple questions.
>>
>> 1) Does federation need to be installed on both upstream and downstream?
>> 2) I have a configuration that matches what Denis has but it only
>> creates the exchange on the downstream server and does nothing
>> upstream (only difference between his and mine is the remote hostname,
>> of course). What am I missing?
>>
>>
>> Thanks,
>> James
>>
>>
>> On Thu, Feb 23, 2012 at 10:47 AM, Simon MacMullen <simon at rabbitmq.com> wrote:
>>> On 23/02/12 16:36, MELIQUE Denis (MORPHO) wrote:
>>>>
>>>> Thank you Simon.
>>>>
>>>> This makes it much clearer for me.
>>>> I was first surprised to see this queue always empty on my test bed,
>>>> but the reason is (correct me if I'm wrong)
>>>> that my system was not loaded enough, and consumers got messages as soon
>>>> as they came in.
>>>
>>>
>>> You're partly right. Your system was not loaded enough, but the messages in
>>> that queue will be consumed by the downstream exchange, not (directly) by
>>> your consumers.
>>>
>>>
>>>> So every message consumed from the downstream is first written on disk.
>>>
>>>
>>> No. The queue is durable, but that just determines whether it survives a
>>> restart. Transient messages can still through it, and remain transient.
>>>
>>> Cheers, Simon
>>>
>>>
>>>> This might be a disadvantage for performance.
>>>> And gives an advantage to Shovel, as you can configure it with transient
>>>> queues.
>>>>
>>>> Denis
>>>>
>>>> -----Original Message-----
>>>> From: Simon MacMullen [mailto:simon at rabbitmq.com]
>>>> Sent: Thursday, February 23, 2012 12:11 PM
>>>> To: MELIQUE Denis (MORPHO)
>>>> Cc: rabbitmq-discuss at lists.rabbitmq.com
>>>> Subject: Re: [rabbitmq-discuss] Understanding Federation
>>>>
>>>> On 23/02/12 10:56, MELIQUE Denis (MORPHO) wrote:
>>>>>
>>>>> I’m doing some testing with federation plug-in to try to understand it.
>>>>
>>>>
>>>> Cool!
>>>>
>>>> <snip>
>>>>
>>>>> What are these Exchange and Queues used for ?
>>>>
>>>>
>>>> The queue is there to buffer messages that are published to the upstream
>>>> exchange and have not yet been consumed from the downstream exchange.
>>>>
>>>> It is bound to the internal exchange with the union of all the bindings
>>>> that are in use downstream, so we only transfer messages that are going
>>>> to be of use downstream.
>>>>
>>>> The internal exchange is there so that whenever we (re)establish a
>>>> federation link we can set up an exact set of bindings
>>>> mostly-atomically, by creating a new internal exchange, creating all the
>>>> bindings to it, then binding it to the upstream exchange.
>>>>
>>>>> Are they reserved for federation internal use ?
>>>>
>>>>
>>>> Yes.
>>>>
>>>>> When do messages go into these queues ?
>>>>
>>>>
>>>> As mentioned, the bindings are pushed upstream so messages from upstream
>>>> go into these queues if they will be routed once they get downstream.
>>>>
>>>>> Do they have to be durable or is there a way to configure them as
>>>>> transient ?
>>>>
>>>>
>>>> At the moment they have to be durable. Is this a problem? The idea is
>>>> that you can set the queue TTL if you do not want them to hang around
>>>> forever.
>>>>
>>>>> I would be very interested by a brief description of how federation
>>>>> works internally.
>>>>
>>>>
>>>> Hopefully this answers your question.
>>>>
>>>> Cheers, Simon
>>>>
>>>
>>>
>>> --
>>> Simon MacMullen
>>> RabbitMQ, VMware
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-discuss at lists.rabbitmq.com
>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
More information about the rabbitmq-discuss
mailing list