[rabbitmq-discuss] Pika disconnects

Sat Oct 8 13:50:45 BST 2011

On Sat, Oct 8, 2011 at 13:06, John Reuning <john at ibiblio.org> wrote:
> Marek,
>
> Your note about the event loop and socket buffers makes sense.  The
> application in question is multi threaded, but only one thread
> accesses the pika objects.  Code in the main thread creates several
> objects in separate threads.  One of these second-layer threads
> watches for events in the application and publishes messages to
> rabbitmq.

Assuming that the same thread does:
 * event loop
 * consuming
 * publishing

It should be fine :)

> The current implementation has another threaded object spawned by the
> event watcher, which contains the pika event loop and connection to
> rabbitmq. The event watcher is the only object/thread that calls
> basic_publish on the pika channel object. Do problems occur when
> merely crossing a thread boundary using pika?

It doesn't matter which thread handles pika connection.
But it matters a lot that when you're doing anything
related to amqp connection, no other thread should
be doing anything on that connection.
(including waiting in event loop).

In other words:
 - you can't have blocking event loop in one thread
 - and publish from other thread.

If you want to publish, you need to stop event loop,
publish, and restart event loop. It doesn't matter
which thread does what, but it matters that only
one thread accesses pika stuff at a time.

(I hope that Gavin, pika maintainer, will fix me if I'm wrong)

Cheers,
    Marek

> Or should this approach
> work as long as the pika objects are only ever used by one thread?
>
> Thanks,
>
> -John
>
> On Sat, Oct 8, 2011 at 6:03 AM, Marek Majkowski <majek04 at gmail.com> wrote:
>> John,
>>
>> Pika doesn't support multi threading. Wrapping single basic_publish
>> isn't good enough - the side effect of this command (network buffer)
>> is going to affect different thread (the thread with event loop).
>>
>> I'm afraid there isn't a simple workaround. You can try to wrap
>> everything (including event loop) in a lock - and thus allow only
>> one thread at a time to use connection/network socket/buffers.
>> But doing that will be rather hard (how to stop event loop
>> if other thread wants to publish?)
>>
>> Or, maybe a better solution would be to use multiple connections,
>> one connection per thread.
>>
>> Does it sound reasonable?
>>
>>
>> Cheers,
>>   Marek
>>
>