[rabbitmq-discuss] Using Pika and PyQtGraph (Qt event loop and BlockingConnection)

[Ask Solem]

On May 13, 2014, at 7:50 PM, Femto Trader <femto.trader at gmail.com> wrote:

> Hello,
> 
> thanks for your reply.
> 
> I don't know Tornado at all.
> 
> I'm not a very talented developer....
> Talking to me of "eventlet on pyqt" or "gevent on pyqt" looks like poetry to me ;-)

Hah! While admitting to your own limitations is a sign of intelligence, resigning from a daunting
challenge is not a virtue.

It’s foreign now because you don’t know the terms, something a quick Google search would resolve.

PyQT uses an event loop to make multiple things happen at once, e.g. playing music while drawing the progress
bar of a file copy operation.  Your BlockingConnection is blocking this event loop so nothing can happen while you’re
reading from the RabbitMQ connection socket.

Every event driven network library like Twisted, Eventlet, Gevent, Tornado etc comes with a different event loop
implementation, but eventlet and gevent is special: it uses greenlets to yield control back
to the event loop so you can write normal, imperative style Python code.

Read about eventlet here: http://eventlet.net

So if you can replace eventlet’s event loop with pyqt, then you can use any existing (eventlet compatible) Python code
with pyqt.  I did a quick google search and found ‘eventlet-pyqt’[0] that will let you do exactly this, but I don’t
know how recent this project is and there could be something better out there now.

In short: Using eventlet you can magically turn your BlockingConnection into a non-blocking one,
so if you can hook eventlet into pyqt you have solved your problem.

[0] https://code.google.com/p/eventlet-pyqt/



> 
> Luke also said me something else:
> "One solution that will likely work for you is to move your MQ code to a new thread, and send a signal back to the GUI thread whenever a message arrives."
> 
> Can someone from RabbitMQ dev team be in touch with PyQt / PySide dev teams
> and also PyQtGraph devs in order to have a very basic sample example because displaying realtime
> data coming from a messaging queue could be useful.

This seems like a fringe use case and I doubt there's a big audience for such a tutorial.
Doing so using threads would be a bad practice at that, so if there was an audience
it would make more sense to invest in a proper “pika on the pyqt event loop” extension.

> 
> One basic example could be:
> one python script connect to RabbitMQ and send a random number
> an other script receive data and plot them into a PyQtGraph.
> 
> I'm not very familiar with threading so I wonder 
> what will happen if thread that will receive data will take more time to compute data
> (do some calculations on these data) than task to display them


Python has the global interpreter lock (GIL) so you cannot perform CPU bound operations
on multiple cores unless you drop down to C and unlock the GIL.

It would be the same situation as when you use an Async I/O framework like eventlet/twisted/tornado
and any CPU bound operation will block the rest of the program.

With eventlet there is a hack to yield control back to the event loop by using
`time.sleep(0)` (eventlet monkey patches must be applied, yuo did read about eventlet already right?).
But that means you must find control points in your CPU bound operations and inject sleeps into it,
which is not exactly pretty.  Usually the solution is to use IPC and start separate OS processes for
CPU bound calculations, e.g. using Python’s multiprocessing.

> 
> 
> Femto
> 
> 
> 
> 2014-05-13 16:07 GMT+02:00 Ask Solem <ask at rabbitmq.com>:
> 
> On May 13, 2014, at 2:54 PM, Michael Klishin <mklishin at gopivotal.com> wrote:
> 
> >  On 12 May 2014 at 00:34:58, Femto (femto.trader at gmail.com) wrote:
> >>> maybe you noticed Qt have an event loop
> >>
> >> and I think it's doing odd things with pika.BlockingConnection
> >>
> >>
> >> Any help is welcome.
> >
> > I doubt that it can, although I know nothing about Qt and little about
> > Pika implementation.
> >
> > Have you tried using a different connect, e.g. Tornado?
> 
> If PyQt does indeed use an event loop then you will not be able to use a blocking connection.
> 
> You have several choices:
> 
> 1) Implement a pika connection implementation using the PyQt event loop.
> 2) Use a “eventlet on pyqt” or “gevent on pyqt” implementation like eventlet-pyqt
>     and use the eventlet or gevent connection in pika.
> 3) Run pyqt in a separate thread.  This is only an option if pyqt actually supports it, which may
>     very well not be the case.
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140514/1162e076/attachment.html>