[rabbitmq-discuss] Pika - how to properly publish from a periodic background thread?

[Marek Majkowski]

On Thu, Sep 22, 2011 at 21:26, Aaron Voelker <aaron at contextlogic.com> wrote:
> I'm using Python 2.6.6, Ubuntu 10.10, and Pika 0.9.5.
>
> I have a situation where I want to publish to a queue from a
> background thread at a periodic interval. The main thread is consuming
> on some different callback.
>
> I assumed (incorrectly) that wrapping my callback and publish in a
> mutex would solve this problem. I ran into some weird behaviour before
> I realized that maybe this isn't the correct approach.
>
> What happens is that the worker I publish to from the background
> thread will send work back to the callback for the main thread.
> Normally, this is fine, but in weird situations (race condition?),
> after a few hours of running my application, it will get Pika into an
> unrecoverable state. Some sample code which duplicates this problem,
> along with the resulting error messages, is included in this message
> in case someone happens to Google for this error message (which got me
> next to nowhere).
>
> What is the correct way to do this? Note that both the callback and
> the scheduler have to have shared access to the same variables (hence
> the mutex), and performance does NOT matter (the scheduler runs once
> per minute, and doesn't do too much.

What about just opening two separate connections, one in each thread?

I'm not sure how "reactors" work in Pika, but I guess there should be a
way to have two separate event loops.

Cheers,
  Marek