<div dir="ltr">Hi Roy,<div><br></div><div>Typical job processing time is 24 hours. I set up heartbeat_interval = 432000 (5 days) . When I tried this earlier, connection dropped off approximately after 10 mins. </div>
<div><br></div><div>I tried with heartbeat=0 and looks like it is working. I find the connection active after 90 minutes since the message processing started.</div><div><br></div><div style>I am now testing with jobs that would take about 1-2 days to complete processing. Will update you once the testing is complete.</div>
<div style><br></div><div style>Thank you so much for your help!</div><div style><br></div><div style>Nishanth</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Mar 8, 2013 at 12:13 PM, Gavin M. Roy <span dir="ltr"><<a href="mailto:gmr@meetme.com" target="_blank">gmr@meetme.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><div class="gmail_quote"><div class="im">On Fri, Mar 8, 2013 at 2:59 PM, Nishanth Babu Achuthan <span dir="ltr"><<a href="mailto:nishanth@urbanmapping.com" target="_blank">nishanth@urbanmapping.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Roy,<div><br></div><div>I am not using time.sleep in my app.</div><div><br></div><div>Worker consumes message from the queue. It then starts processing it which takes several hours to complete. </div>
</div></blockquote><div><br></div></div><div>Perhaps this is a key point.</div><div class="im"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">
<div>In the meantime, the connection is dropped off due to timeout. Once the processing is complete, the worker tries to send ACK. While doing so, it fails with the error below and the worker shuts down.</div>
<div><br></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div>render_jobs_consumer - INFO - tableau-classic-admin0-borders-en-us-extent-101.mbtiles published for merging</div>
</div><div><div>render_jobs_consumer - INFO - tableau-classic-admin0-borders-en-us-extent-101.mbtiles rendering completed in 334.802966833. Sending acknowledgement</div></div></blockquote></div></blockquote><div><br></div>
</div><div>I think this shows it takes over 5 minutes to process a message. If heartbeats are less than that, it will time out unless your consumer app that is doing rendering is making pika process data events.</div><div>
<br></div><div>Since pika is not running in a thread or isolated runtime, the things your consumer does impact its ability to read and respond to frames from RabbitMQ.</div><div><br></div><div>Quick fix is most likely to turn heartbeats off (forcing a heartbeat=0 in your connection parameters) or set it to 4-5x your average processing time.</div>
<div class="im">
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div><div>pika.adapters.base_connection - ERROR - Read empty data, calling disconnect</div>
</div></div><div><div><div>pika.adapters.base_connection - WARNING - Unknown state on disconnect: 5</div></div></div><div><div><div>pika.connection - WARNING - Disconnected from RabbitMQ at <a href="http://192.168.2.182:5672" target="_blank">192.168.2.182:5672</a> (0): </div>
</div></div><div><div><div>pika.adapters.select_connection - INFO - Unregistering poller on fd 3</div></div></div></blockquote></div></blockquote><div><br></div></div><div>Please let us know if this addresses your issue,</div>
<div><br>Gavin </div></div>
</div></div>
<br>_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
<br></blockquote></div><br></div>