[rabbitmq-discuss] Understanding Federation
pmccarty
pmccarty at rarewire.com
Mon Nov 12 21:16:10 GMT 2012
What was the problem? I'm running in to the exact same thing.
The docs on this topic somehow manage to be extremely verbose while never
seeming to tell me what I need to know to actually make things work. I
could explain the successful bits of my attempt so far in a minute or
two—it's *extremely* simple—but it's taken me hours to get here.
On Monday, February 27, 2012 3:48:22 PM UTC-6, James Carr wrote:
>
> Nvm, I got it working.
>
> On Mon, Feb 27, 2012 at 3:13 PM, James Carr <james.... at gmail.com<javascript:>>
> wrote:
> > FYI, here is the configuration I am using on the downstream server:
> >
> > [
> > {rabbitmq_federation,
> > [ {exchanges, [[{exchange, "usage"},
> > {virtual_host, "/"},
> > {type, "topic"},
> > {durable, true},
> > {auto_delete, false},
> > {internal, false},
> > {upstream_set, "remotes"}]
> > ]},
> > {upstream_sets, [{"remotes", [[{connection, "upstream-server"},
> > {exchange, "usage"},
> > {max_hops, 2}]
> > ]}
> > ]},
> > {connections, [{"upstream-server", [{host,
> "dc2rabbitmq1"},
> > {protocol, "amqp"},
> > {port, "5672"},
> > {virtual_host, "/"},
> > {username, "guest"},
> > {password, "guest"},
> > {mechanism, default},
> > {prefetch_count, 1000},
> > {reconnect_delay, 5},
> > {heartbeat, 1},
> > {queue_expires, 30000},
> > {message_ttl, 10000}
> > ]}
> > ]},
> > {local_username, "guest"},
> > {local_nodename, "rabbit"}
> > ]
> > }
> > ].
> >
> >
> > It creates the local exchange on the downstream node just fine, but no
> > action on the upstream (dc2rabbitmq1). A look at the error logs
> > reveals the below messages, but I don't completely grok erlang yet. :S
> >
> > =ERROR REPORT==== 27-Feb-2012::15:01:20 ===
> > ** Generic server <0.1963.0> terminating
> > ** Last message in was connect
> > ** When Server state == {state,amqp_network_connection,
> >
> {state,undefined,undefined,undefined,undefined,
> > undefined,false},
> > <0.1962.0>,undefined,
> > {amqp_params_network,<<"guest">>,<<"guest">>,
> >
> <<"/">>,"dc2rabbitmq1","5672",0,0,1,infinity,
> > none,
> > [#Fun<amqp_auth_mechanisms.plain.3>,
> > #Fun<amqp_auth_mechanisms.amqplain.3>],
> > [],[]},
> > undefined,undefined,
> > #Fun<amqp_connection_sup.0.39273983>,
> > #Fun<amqp_connection_sup.2.54430129>,false}
> > ** Reason for termination ==
> > ** {function_clause,[{inet_tcp,getserv,["5672"]},
> > {gen_tcp,connect1,4},
> > {gen_tcp,connect,4},
> > {amqp_network_connection,do_connect,5},
> > {amqp_gen_connection,handle_call,3},
> > {gen_server,handle_msg,5},
> > {proc_lib,init_p_do_apply,3}]}
> >
> > =ERROR REPORT==== 27-Feb-2012::15:01:20 ===
> > ** Generic server <0.1949.0> terminating
> > ** Last message in was {'$gen_cast',maybe_go}
> > ** When Server state == {not_started,
> > {{upstream,
> > {amqp_params_network,<<"guest">>,
> > <<"guest">>,<<"/">>,"dc2rabbitmq1",
> > "5672",0,0,1,infinity,none,
> > [#Fun<amqp_auth_mechanisms.plain.3>,
> >
> #Fun<amqp_auth_mechanisms.amqplain.3>],
> > [],[]},
> > <<"usage">>,1000,2,5,none,10000,
> > "upstream-server"},
> > {resource,<<"/">>,exchange,<<"usage">>}}}
> > ** Reason for termination ==
> > ** {{function_clause,[{inet_tcp,getserv,["5672"]},
> > {gen_tcp,connect1,4},
> > {gen_tcp,connect,4},
> > {amqp_network_connection,do_connect,5},
> > {amqp_gen_connection,handle_call,3},
> > {gen_server,handle_msg,5},
> > {proc_lib,init_p_do_apply,3}]},
> > {gen_server,call,[<0.1963.0>,connect,infinity]}}
> >
> > =WARNING REPORT==== 27-Feb-2012::15:01:20 ===
> > Channel (<0.1958.0>): Unregistering confirm handler <0.1949.0> because
> > it died. Reason: {{function_clause,
> >
> > [{inet_tcp,
> >
> > getserv,
> >
> > ["5672"]},
> >
> > {gen_tcp,
> >
> > connect1,
> >
> > 4},
> >
> > {gen_tcp,
> >
> > connect,
> >
> > 4},
> >
> > {amqp_network_connection,
> >
> > do_connect,
> >
> > 5},
> >
> > {amqp_gen_connection,
> >
> > handle_call,
> >
> > 3},
> >
> > {gen_server,
> >
> > handle_msg,
> >
> > 5},
> >
> > {proc_lib,
> >
> > init_p_do_apply,
> >
> > 3}]},
> >
> > {gen_server,
> >
> > call,
> >
> > [<0.1963.0>,
> >
> > connect,
> >
> > infinity]}}
> >
> > Any help would be appreciated.
> >
> > Thanks,
> > James
> >
> >
> > On Mon, Feb 27, 2012 at 2:12 PM, James Carr <james.... at gmail.com<javascript:>>
> wrote:
> >> Sorry to hijack an existing thread, but I'm delving into using
> >> federation myself (after a 3 month break of even working with
> >> rabbitmq) and have a couple questions.
> >>
> >> 1) Does federation need to be installed on both upstream and downstream?
> >> 2) I have a configuration that matches what Denis has but it only
> >> creates the exchange on the downstream server and does nothing
> >> upstream (only difference between his and mine is the remote hostname,
> >> of course). What am I missing?
> >>
> >>
> >> Thanks,
> >> James
> >>
> >>
> >> On Thu, Feb 23, 2012 at 10:47 AM, Simon MacMullen <si... at rabbitmq.com<javascript:>>
> wrote:
> >>> On 23/02/12 16:36, MELIQUE Denis (MORPHO) wrote:
> >>>>
> >>>> Thank you Simon.
> >>>>
> >>>> This makes it much clearer for me.
> >>>> I was first surprised to see this queue always empty on my test bed,
> >>>> but the reason is (correct me if I'm wrong)
> >>>> that my system was not loaded enough, and consumers got messages as
> soon
> >>>> as they came in.
> >>>
> >>>
> >>> You're partly right. Your system was not loaded enough, but the
> messages in
> >>> that queue will be consumed by the downstream exchange, not (directly)
> by
> >>> your consumers.
> >>>
> >>>
> >>>> So every message consumed from the downstream is first written on
> disk.
> >>>
> >>>
> >>> No. The queue is durable, but that just determines whether it survives
> a
> >>> restart. Transient messages can still through it, and remain transient.
> >>>
> >>> Cheers, Simon
> >>>
> >>>
> >>>> This might be a disadvantage for performance.
> >>>> And gives an advantage to Shovel, as you can configure it with
> transient
> >>>> queues.
> >>>>
> >>>> Denis
> >>>>
> >>>> -----Original Message-----
> >>>> From: Simon MacMullen [mailto:si... at rabbitmq.com <javascript:>]
> >>>> Sent: Thursday, February 23, 2012 12:11 PM
> >>>> To: MELIQUE Denis (MORPHO)
> >>>> Cc: rabbitmq... at lists.rabbitmq.com <javascript:>
> >>>> Subject: Re: [rabbitmq-discuss] Understanding Federation
> >>>>
> >>>> On 23/02/12 10:56, MELIQUE Denis (MORPHO) wrote:
> >>>>>
> >>>>> I’m doing some testing with federation plug-in to try to understand
> it.
> >>>>
> >>>>
> >>>> Cool!
> >>>>
> >>>> <snip>
> >>>>
> >>>>> What are these Exchange and Queues used for ?
> >>>>
> >>>>
> >>>> The queue is there to buffer messages that are published to the
> upstream
> >>>> exchange and have not yet been consumed from the downstream exchange.
> >>>>
> >>>> It is bound to the internal exchange with the union of all the
> bindings
> >>>> that are in use downstream, so we only transfer messages that are
> going
> >>>> to be of use downstream.
> >>>>
> >>>> The internal exchange is there so that whenever we (re)establish a
> >>>> federation link we can set up an exact set of bindings
> >>>> mostly-atomically, by creating a new internal exchange, creating all
> the
> >>>> bindings to it, then binding it to the upstream exchange.
> >>>>
> >>>>> Are they reserved for federation internal use ?
> >>>>
> >>>>
> >>>> Yes.
> >>>>
> >>>>> When do messages go into these queues ?
> >>>>
> >>>>
> >>>> As mentioned, the bindings are pushed upstream so messages from
> upstream
> >>>> go into these queues if they will be routed once they get downstream.
> >>>>
> >>>>> Do they have to be durable or is there a way to configure them as
> >>>>> transient ?
> >>>>
> >>>>
> >>>> At the moment they have to be durable. Is this a problem? The idea is
> >>>> that you can set the queue TTL if you do not want them to hang around
> >>>> forever.
> >>>>
> >>>>> I would be very interested by a brief description of how federation
> >>>>> works internally.
> >>>>
> >>>>
> >>>> Hopefully this answers your question.
> >>>>
> >>>> Cheers, Simon
> >>>>
> >>>
> >>>
> >>> --
> >>> Simon MacMullen
> >>> RabbitMQ, VMware
> >>> _______________________________________________
> >>> rabbitmq-discuss mailing list
> >>> rabbitmq... at lists.rabbitmq.com <javascript:>
> >>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq... at lists.rabbitmq.com <javascript:>
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20121112/d93ec138/attachment.htm>
More information about the rabbitmq-discuss
mailing list