<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
mso-fareast-language:DE-CH;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="DE-CH" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I had the same issue, depending on the meg size and network layout it can publish like 1’000msg, and then pika crashes with publishing the like
10’000<sup>th</sup> message. It will continue to publish messages (no matter whether actually sent to the broker or not) until it hits the end of the buffer.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">In order to avoid this either do not publish too fast, implement the TCP_Backpressure CB to let the buffer drain, or use transactional channels
which seems to avoid this issue at all. <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Cheers<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">josh<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> rabbitmq-discuss-bounces@lists.rabbitmq.com [mailto:rabbitmq-discuss-bounces@lists.rabbitmq.com]
<b>On Behalf Of </b>Gavin M. Roy<br>
<b>Sent:</b> Sonntag, 27. Mai 2012 23:01<br>
<b>To:</b> James Dear<br>
<b>Cc:</b> rabbitmq-discuss@lists.rabbitmq.com<br>
<b>Subject:</b> Re: [rabbitmq-discuss] Pika write buffer problem<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">The warnings are just warnings, letting you know that the writes to rabbitMQ are not happening as quickly as you'd like them to do. Are you saying they are not actually getting into RabbitMQ?
<o:p></o:p></p>
</div>
<p><span style="color:#A0A0A8">On Sunday, May 27, 2012 at 4:27 PM, James Dear wrote:<o:p></o:p></span></p>
<blockquote style="border:none;border-left:solid windowtext 1.0pt;padding:0cm 0cm 0cm 8.0pt;margin-left:0cm;margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal">Hi,<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I'm having trouble getting simple examples of basic_publish to work due to the write buffer being exceeded. If I use the demo_send code from
<a href="http://pika.github.com/examples.html">here</a>, I only manage to send 5 messages before getting write buffer exceeded problems. Here's my output:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">[INFO 2012-05-27 20:30:16,863 #5451] Connecting fd 3 to
<a href="http://127.0.0.1:5672">127.0.0.1:5672</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[INFO 2012-05-27 20:30:17,869 #5451] demo_send: Connected to RabbitMQ<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[INFO 2012-05-27 20:30:17,872 #5451] demo_send: Received our Channel<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[INFO 2012-05-27 20:30:17,882 #5451] demo_send: Queue Declared<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[INFO 2012-05-27 20:30:17,883 #5451] Sending: Hello World #0: 1338147017.88308692<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[INFO 2012-05-27 20:30:17,883 #5451] Sending: Hello World #1: 1338147017.88367391<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[INFO 2012-05-27 20:30:17,884 #5451] Sending: Hello World #2: 1338147017.88425899<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[INFO 2012-05-27 20:30:17,884 #5451] Sending: Hello World #3: 1338147017.88472700<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[INFO 2012-05-27 20:30:17,885 #5451] Sending: Hello World #4: 1338147017.88527107<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">/..snip../pika/connection.py:766: UserWarning: Pika: Write buffer exceeded warning threshold at 413 bytes and an estimated 10 frames behind<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> warn(message % (self.outbound_buffer.size, est_frames_behind))<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">/..snip../pika/connection.py:766: UserWarning: Pika: Write buffer exceeded warning threshold at 447 bytes and an estimated 11 frames behind<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> warn(message % (self.outbound_buffer.size, est_frames_behind))<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">/..snip../pika/connection.py:766: UserWarning: Pika: Write buffer exceeded warning threshold at 490 bytes and an estimated 12 frames behind<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> warn(message % (self.outbound_buffer.size, est_frames_behind))<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">[INFO 2012-05-27 20:30:17,887 #5451] Sending: Hello World #5: 1338147017.88733912<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">/..snip../pika/connection.py:766: UserWarning: Pika: Write buffer exceeded warning threshold at 511 bytes and an estimated 13 frames behind<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> warn(message % (self.outbound_buffer.size, est_frames_behind))<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">/..snip../pika/connection.py:766: UserWarning: Pika: Write buffer exceeded warning threshold at 545 bytes and an estimated 14 frames behind<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> warn(message % (self.outbound_buffer.size, est_frames_behind))<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">/..snip../pika/connection.py:766: UserWarning: Pika: Write buffer exceeded warning threshold at 588 bytes and an estimated 15 frames behind<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> warn(message % (self.outbound_buffer.size, est_frames_behind))<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">etc etc<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">On the other hand, everything works fine with pika twisted and the msgs do end up on RabbitMQ with pika twisted, so it doesn't seem to be a restriction to do with RabbitMQ per se.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I've created <a href="https://gist.github.com/2815757">gists</a> for the pika native code which doesn't work and the pika twisted code that does work.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">If anyone could help, I would greatly appreciate it.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">James<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">_______________________________________________<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">rabbitmq-discuss mailing list<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<pre><o:p> </o:p></pre>
<pre>-- <o:p></o:p></pre>
<pre>----<o:p></o:p></pre>
<pre>ASG at hnet<o:p></o:p></pre>
</div>
</body>
</html>