[rabbitmq-discuss] C client and thread safety

[Tony Garnock-Jones]

Hi Jim,

Jim Irrer wrote:
> I'm working with the RabbitMQ C client, and had a few questions
> about thread safety.

In general, there is no thread awareness built in to librabbitmq at all. It is
currently the responsibility of applications using the library to ensure that
multiple threads don't screw up the library's state.

That said, there is no longer any global state at all in the library, so if you
stick to one connection (and thus one socket) per thread, you should be fine.

> Must a socket returned by amqp_open_socket be exclusively
> associated with one connection, or can it be used for multiple
> connections?

One connection. Each AMQP connection needs exactly one socket to communicate over.

> Must a socket returned by amqp_open_socket be exclusively
> used by one thread, or can it be used by multiple threads?

The connection associated with the socket must be used by one thread *at a
time*. The locking can get quite complicated when you take into account the way
librabbitmq currently deals with incoming events, so sticking with one
connection for each thread makes life much easier.

> Must a connection returned by amqp_new_connection be exclusively
> used to read from or write to one channel/queue, or is using multiple ok,
> even in a multi-threaded environment.

Multiple are OK, but again, each connection is best kept to a single thread.
Think of multiple channels as separate *transaction* and *error reporting*
contexts; they don't buy much else in current versions of the AMQP protocol :-)

> If any of these can be used by multiple threads, is there a performance
> advantage in reusing the same socket or connection, or is using multiple
> ones better?

Using a single connection with multiple channels means that TCPs
congestion-avoidance counts all AMQP traffic as a single stream, whereas using
multiple connections with a single channel each means that TCP will count each
stream separately. Which is more appropriate to your application depends on the
structure of your network: the former might be more appropriate for a WAN link,
the latter for a LAN or a datacentre, etc.

> It looks like channels (created with amqp_channel_open) are identified
> by (small?)
> integers chosen by the application and are associated with the
> connection.  This
> means that it is the responsibility of the application to choose unique
> numbers,
> properly close them, etc..   It would seem cleaner to pass a connection as a
> single parameter to amqp_channel_open and have it return a pointer to a
> channel,
> and the list of channels would be maintained in the connection.

Yes, there is a lot of per-channel state management the library currently
doesn't do. A set of "active" channel numbers might not be too hard to manage.

> Also, it looks like amqp_new_connection uses 'calloc' to allocate
> memory for a connection, but amqp_connection_close does not do
> a 'free' to avoid the memory leak.  Maybe that was intentional, but if
> not it should be changed.

The amqp_connection_close() function causes an AMQP Connection.Close RPC to
happen on the open socket, but does not then tear down any of the application
state: that's what amqp_destroy_connection() is for. The relevant pairings are
roughly

For local connection state management (calloc/free):
  amqp_new_connection
  amqp_destroy_connection

For remote connection state management (Connection.Open/Tune/Close, once local
connection state exists):
  amqp_login
  amqp_connection_close
...plus interactions required on receipt of a Connection.Close method frame
from the peer.

Cheers,
  Tony