[rabbitmq-discuss] rabbitmq-c memory leak?

[Abhay Rajure]

Dushin Fred <fred at ...> writes:

> 
> 
> I have written a simple test program using the SimpleAqmpClient library
(which in turn uses the rabbitmq-c
> library), which seems to illustrate a memory leak, of sorts.
> 
> The program is a simple main, which pre-allocates a string message, and
then calls BasicPublish on a
> channel in a loop, all single threaded.  Effectively:
> 
>         auto envelope = BasicMessage::Create(random_data(message_size));
>         for (unsigned i = 0;  i < num_messages;  ++i)
>         {
>             channel->BasicPublish(exchange_name, routing_key, envelope);
>         }
> 
> The problem is that the memory image of this process seems to grow without
bounds.  The memory all seems to be
> allocated in amqp_pool_alloc (as expected), when calling wait_frame_inner
-> amqp_handle_input.  The
> memory is not orphaned -- i.e., it seems to get cleaned up at shutdown, so
it doesn't show up as a leak in a leak
> detector tool, such a Instruments/DTrace, or valgrind, or purify, etc. 
But I don't see where the memory is
> every de-allocated, which will spell doom for an application that is
designed to never terminate.
> 
> I am attaching a screenshot from an Instruments run, to give some context.
 The code is here:
> 
> https://github.com/fadushin/sandbox/blob/master/cpp/rabbitmq/sender.cpp
> 
> This was run against RabbitMQ 3.2.4, using rabbitmq-c 0.5.0 and
SimpleAmqpClient 2.3, compiled using
> Clang 5.0 on OS X.9 (Mavericks).
> 
> Has anyone seen this behavior?  Am I missing something in my use of
BasicPublish, that is not telling the
> rabbitmq-c library to free memory in the "amqp pool"?  It doesn't seem to
make a difference whether I am
> actively pulling messages off the queue from a separate process, or even
whether there is a queue bound to
> the exchange to which I am publishing messages.  (Not that I would expect
it to)
> 
> -Fred
> 
> encl.
> 
> 
> 
> 
> I have written a simple test program using the SimpleAqmpClient library
(which in turn uses the rabbitmq-c
> library), which seems to illustrate a memory leak, of sorts.
> 
> The program is a simple main, which pre-allocates a string message, and
then calls BasicPublish on a
> channel in a loop, all single threaded.  Effectively:
> 
>         auto envelope = BasicMessage::Create(random_data(message_size));
>         for (unsigned i = 0;  i < num_messages;  ++i)
>         {
>             channel->BasicPublish(exchange_name, routing_key, envelope);
>         }
> 
> The problem is that the memory image of this process seems to grow without
bounds.  The memory all seems to be
> allocated in amqp_pool_alloc (as expected), when calling wait_frame_inner
-> amqp_handle_input.  The
> memory is not orphaned -- i.e., it seems to get cleaned up at shutdown, so
it doesn't show up as a leak in a leak
> detector tool, such a Instruments/DTrace, or valgrind, or purify, etc. 
But I don't see where the memory is
> every de-allocated, which will spell doom for an application that is
designed to never terminate.
> 
> I am attaching a screenshot from an Instruments run, to give some context.
 The code is here:
> 
> https://github.com/fadushin/sandbox/blob/master/cpp/rabbitmq/sender.cpp
> 
> This was run against RabbitMQ 3.2.4, using rabbitmq-c 0.5.0 and
SimpleAmqpClient 2.3, compiled using
> Clang 5.0 on OS X.9 (Mavericks).
> 
> Has anyone seen this behavior?  Am I missing something in my use of
BasicPublish, that is not telling the
> rabbitmq-c library to free memory in the "amqp pool"?  It doesn't seem to
make a difference whether I am
> actively pulling messages off the queue from a separate process, or even
whether there is a queue bound to
> the exchange to which I am publishing messages.  (Not that I would expect
it to)
> 
> -Fred
> 
> encl.
> 
> 


Seems like the issue is in SimpleAmqpClient's
ChannelImpl::MaybeReleaseBufferOnChannel() method, it fails to call
amqp_maybe_release_buffers_on_channel() because by the time it is called, it
can't find channel in m_open_channels(bug?).

Try this --

In File SimpleAmqpClient/src/ChannelImpl.cpp,
call amqp_maybe_release_buffers_on_channel() in ReturnChannel().

void ChannelImpl::ReturnChannel(amqp_chanel_t channel)
{
   +amqp_maybe_release_buffers_on_channel(m_connection, channel);
    m_free_channels.push(channel);
}

-Abhay