Hi, <br><br>I'm testing rabbitmq pubsub pattern with pika 0.9.5. Basically what I do are the following: <br><br>1. publish message in a tornado application<br>2. consume message in a standalone script<br>3. ab test the tornado application to see how messages goes<br>
<br>basic parameters used:<br><br>1. exchange_type = fanout<br>2. auto_delete = False<br>3. durable = True<br>4. delivery_mode = 2<br>5. no_ack = True<br>6. SimpleReconnectionStrategy for reconnection<br><br>When I'm testing the server die scenario. I found three issues:<br>
<br>ISSUE 1:<br>=======<br>Stop rabbitmq server, tornado application raises the following exception and become not responsive:<br><br>2011-04-23 11:48:53,872 - ERROR - TornadoConnection: Socket Error on 9: 104<br>2011-04-23 11:48:53,872 - ERROR - Exception in I/O handler for fd 9<br>
Traceback (most recent call last):<br> File "/usr/local/lib/python2.6/dist-packages/tornado-1.1-py2.6.egg/tornado/ioloop.py", line 272, in start<br> self._handlers[fd](fd, events)<br> File "/usr/local/lib/python2.6/dist-packages/tornado-1.1-py2.6.egg/tornado/stack_context.py", line 128, in wrapped<br>
callback(*args, **kwargs)<br> File "/usr/local/lib/python2.6/dist-packages/pika-0.9.5-py2.6.egg/pika/adapters/base_connection.py", line 137, in _handle_events<br> self._handle_error(error)<br> File "/usr/local/lib/python2.6/dist-packages/pika-0.9.5-py2.6.egg/pika/adapters/base_connection.py", line 106, in _handle_error<br>
error_code = error[0] # Python <= 2.5<br>TypeError: 'NoneType' object is unsubscriptable<br>2011-04-23 11:48:53,886 - ERROR - Exception in callback <functools.partial object at 0x9e500f4><br>Traceback (most recent call last):<br>
File "/usr/local/lib/python2.6/dist-packages/tornado-1.1-py2.6.egg/tornado/ioloop.py", line 333, in _run_callback<br> callback()<br> File "/usr/local/lib/python2.6/dist-packages/tornado-1.1-py2.6.egg/tornado/stack_context.py", line 128, in wrapped<br>
callback(*args, **kwargs)<br> File "/home/ken/workspace/QvodGround2/utils/messaging.py", line 53, in connect<br> reconnection_strategy=srs)<br> File "/usr/local/lib/python2.6/dist-packages/pika-0.9.5-py2.6.egg/pika/adapters/tornado_connection.py", line 33, in __init__<br>
reconnection_strategy)<br> File "/usr/local/lib/python2.6/dist-packages/pika-0.9.5-py2.6.egg/pika/adapters/base_connection.py", line 50, in __init__<br> reconnection_strategy)<br> File "/usr/local/lib/python2.6/dist-packages/pika-0.9.5-py2.6.egg/pika/connection.py", line 170, in __init__<br>
self._connect()<br> File "/usr/local/lib/python2.6/dist-packages/pika-0.9.5-py2.6.egg/pika/connection.py", line 228, in _connect<br> self.parameters.port or spec.PORT)<br> File "/usr/local/lib/python2.6/dist-packages/pika-0.9.5-py2.6.egg/pika/adapters/tornado_connection.py", line 39, in _adapter_connect<br>
BaseConnection._adapter_connect(self, host, port)<br> File "/usr/local/lib/python2.6/dist-packages/pika-0.9.5-py2.6.egg/pika/adapters/base_connection.py", line 58, in _adapter_connect<br> self.socket.connect((host, port))<br>
File "<string>", line 1, in connect<br>error: [Errno 111] Connection refused<br><br>ISSUE2:<br>=======<br>I tested 800 messages, with slow consumer. while it runs, stop rabbitmq-server with Ctrl+C, consumer is able to continuing consuming message and finish all 800 messages, then it EXIT.<br>
<br>It's very weird indeed, because the server already gone. Is there any incoming message buffer in pika? How many messages can be buffered? Are those message stored in memory? Is it possible to configure the memory usage?<br>
<br>ISSUE3:<br>=======<br>In my use case, it's very important the tornado application wouldn't die for any reason. If there is anything bad happened to the rabbitmq-server or connection, messaging system can be down-graded as a not functioning part, which can go back automatically when server issues resolved. The long running consuming process shouldn't die or exit when server died or connection broken. Is this requirement sensible enough? <br>
<br>I hope SimpleReconnectionStrategy works for my case, which is not. Is there anything wrong in my application code? Or I just roll my own one, any suggestion on how to do it?<br><br>I appreciate any help.<br><br>Best regards!<br>
can<br><br>PS: partial tornado application code can be found at gist: <a href="https://gist.github.com/938517">https://gist.github.com/938517</a> , consumer script: <a href="https://gist.github.com/938226">https://gist.github.com/938226</a><br>
<br>