[rabbitmq-discuss] Memory leak when using shovels?

Travis hcoyote at ghostar.org
Tue Dec 20 16:42:22 GMT 2011


Hello,

Are there any known memory leaks when using shovels with 2.6.1?  We're
seeing a slow leak that grows over 6-8 days on a few instances of
rabbitmq that are using the shovel plugin and we're not exactly sure
why.  I'm not sure if this is related to the memory leak we previously
reported a few weeks ago that was fixed in 2.7.1

These instances are handling between 5 and 7 million messages a day.

The shovel config (and rabbit config) we're using is:


{rabbitmq_shovel,
   [{shovels,
     [{engine_my_prod_shovel,
       [{sources,      [{brokers,
["amqp://my-user-prod:rabbitmq-my-prod@localhost/my-prod"]},
                        {declarations,
                           ['queue.declare',
                            {'queue.bind',
                                   [{exchange, <<"my-exchange">>},
                                    {queue,    <<"my-queue">>}]}
                           ]}]},
        {destinations, [{brokers, [

"amqp://my-user-prod:rabbitmq-my-prod@aus-svc3/my-prod?heartbeat=5",

"amqp://my-user-prod:rabbitmq-my-prod@aus-svc4/my-prod?heartbeat=5"
                            ]},
                        {declarations,
                           [{'exchange.declare',
                                   [{exchange, <<"my-exchange">>},
                                    {type, <<"direct">>},
                                    durable]},
                            {'queue.bind',
                                   [{exchange, <<"my-exchange">>},
                                    {queue,    <<"my-queue">>}]}

                           ]}]},
        {queue, <<"my-queue">>},
        {prefetch_count, 1000},
        {ack_mode, on_confirm},
        {publish_properties, [{delivery_mode, 2}]},
        {publish_fields, [{exchange, <<"my-exchange">>},
                          {routing_key, <<"">>}]},
        {reconnect_delay, 5}
       ]}
      ]
    }]
}
].


I have the output of the following if it's helpful (though I won't
email it to the list; it's about 5M)

io:format("~p", [lists:reverse(lists:sort([{process_info(Pid, memory),
Pid, process_info(Pid)} || Pid <- processes()]))]).


Travis
-- 
Travis Campbell
travis at ghostar.org


More information about the rabbitmq-discuss mailing list