[rabbitmq-discuss] Messages lost with RabbitMQ Federation

Thu Jul 5 12:08:23 BST 2012

I have 2 RabbitMQ Nodes. One node is on a US server and one is on a UK
server. They are both connected to the same VPN.

I'm consuming the messages from the local nodes in both in the US and UK
with:

AMQP.start('amqp://guest:guest@localhost:25672') do |connection, open_ok|
  channel  = AMQP::Channel.new(connection)
  exchange = AMQP::Exchange.new(channel, "x-federation", "xanview",
:durable => true,
    :arguments => {"upstream-set" => "my-upstreams", "type" => "topic",
"durable" => "true"})
  channel.queue("testqueue_uk", :durable => true) do |queue| # testqueue_us
in the US
    queue.bind(exchange, :routing_key => '#')
    queue.subscribe do |metadata, payload|
      i+=1
      puts "#{i}: Recieved message: #{payload.inspect}."
    end
  end
end

I'm producing messages locally to the local nodes in both the US and UK
with:

EventMachine.run do
  AMQP.connect('amqp://guest:guest@localhost:25672') do |connection,
open_ok|
    channel = AMQP::Channel.new(connection)
    exchange = AMQP::Exchange.new(channel, "x-federation", "xanview",
:durable => true, :arguments => {"upstream-set" => "my-upstreams", "type"
=> "topic", "durable" => "true"})
    EventMachine.add_periodic_timer(0.1) do
      $i+=1
      puts "#{$i}: publishing msg"
      $exchange.publish("#{$i} this is a test message from the US",
:routing_key => "some.topic", :persistent => true )
    end
  end
end

This all seems to work reliably until I try to kill the connection: I
shutdown the VPN and see messages being queued in queues called:
"federation: xanview -> node2" (on the UK server) and "federation: xanview
-> node1" (on the US server) as expected.

However when I restart the VPN, the UK server has received 43737 messages
while the US only received 43708 messages. Where did 29 messages disappear?

Both node configs look like this with the only exception of the US
connecting to federation on 10.8.0.22 and the UK connecting to federation
on 10.8.0.33:

[
  {rabbit, [
    {tcp_listeners, [25672]}
  ]},
  {rabbitmq_mochiweb, [
    {listeners,        [{mgmt, [{port, 55555}]}]},
    {default_listener, [{port, 55550}]},
    {contexts,         [{rabbit_mgmt, mgmt}]}
  ]},

{rabbitmq_federation,
   [ {exchanges, [[{exchange,     "xanview"},
                   {virtual_host, "/"},
                   {type,         "topic"},
                   {durable,      true},
                   {auto_delete,  false},
                   {internal,     false},
                   {upstream_set, "my-upstreams"}]
                 ]},
     {upstream_sets, [{"my-upstreams", [[{connection,
"upstream-server"}]]}]},
     {connections, [{"upstream-server", [{host,            "10.8.0.22"},
                                         {protocol,        "amqp"},
                                         {port,            25672},
                                         {virtual_host,    "/"},
                                         {username,        "guest"},
                                         {password,        "guest"},
                                         {mechanism,       default},
                                         {prefetch_count,  1000},
                                         {reconnect_delay, 5},
                                         {ha_policy,       "all"}
                                        ]}
                   ]},
     {local_username, "guest"},
     {local_nodename, "node2"}
   ]
  }

].

Any ideas why messages are lost? What could be causing it?

Roman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120705/691b3103/attachment.htm>