[rabbitmq-discuss] Pika write buffer problem

Tue Jun 12 15:28:00 BST 2012

James,

Sorry to be so tardy. Are you still having problems?

The gist code you put up (I only ran the native version on my local
machine) works fine for me

(after I modified the publish to write to routing key "test" instead of
"flibble" -- flibble isn't in the demo_send code, so this must be a
misprint?)

and I can send (and see in rabbitmqctl list_queues) many messages to
queue "test" using the default exchange.

This is typical output:

[INFO     2012-06-12 15:11:17,944  #68353] demo_send: Connected to RabbitMQ
[INFO     2012-06-12 15:11:17,944  #68353] demo_send: Received our Channel
[INFO     2012-06-12 15:11:17,945  #68353] demo_send: Queue Declared
[INFO     2012-06-12 15:11:17,945  #68353] Sending: Hello World #0: 1339510277.94537807
[INFO     2012-06-12 15:11:17,945  #68353] Sending: Hello World #1: 1339510277.94578791
[INFO     2012-06-12 15:11:17,946  #68353] Sending: Hello World #2: 1339510277.94616795
[INFO     2012-06-12 15:11:17,946  #68353] Sending: Hello World #3: 1339510277.94652104
[INFO     2012-06-12 15:11:17,946  #68353] Sending: Hello World #4: 1339510277.94688296
[INFO     2012-06-12 15:11:17,947  #68353] Sending: Hello World #5: 1339510277.94723511
[INFO     2012-06-12 15:11:17,947  #68353] Sending: Hello World #6: 1339510277.94759393
[INFO     2012-06-12 15:11:17,947  #68353] Sending: Hello World #7: 1339510277.94794512
[INFO     2012-06-12 15:11:17,948  #68353] Sending: Hello World #8: 1339510277.94829702
[INFO     2012-06-12 15:11:17,948  #68353] Sending: Hello World #9: 1339510277.94865108
[INFO     2012-06-12 15:11:17,949  #68353] Disconnected from RabbitMQ at 127.0.0.1:5672

and here is the output from rabbitmqctl:

> ./rabbitmqctl list_queues name durable auto_delete messages_ready messages_unacknowledged messages

Listing queues ...
test    true    false   50  0   50
queue   false   false   0   0   0
...done.

Are you doing something different?

Note that this connected to a local machine using guest account

Steve Powell  (a happy bunny)
----------yet more definitions from the SPD----------
corrugate (n.) T.V. soap scandal.
olympic (n.) A camp road-digger.
jamboree (n.) A conserve made from French cheese.

On 2 Jun 2012, at 10:11, James Dear wrote:

> Yeah I could understand if it was happening after the thousandth, or maybe even the hundredth message, but tenth? Why would it slow down after just ten?
> 
> On May 27, 2012 10:01 PM, "Gavin M. Roy" <gmr at myyearbook.com> wrote:
> The warnings are just warnings, letting you know that the writes to rabbitMQ are not happening as quickly as you'd like them to do. Are you saying they are not actually getting into RabbitMQ?
> On Sunday, May 27, 2012 at 4:27 PM, James Dear wrote:
> 
>> Hi,
>> 
>> I'm having trouble getting simple examples of basic_publish to work due to the write buffer being exceeded. If I use the demo_send code from here, I only manage to send 5 messages before getting write buffer exceeded problems. Here's my output:
>> 
>> [INFO     2012-05-27 20:30:16,863  #5451] Connecting fd 3 to 127.0.0.1:5672
>> [INFO     2012-05-27 20:30:17,869  #5451] demo_send: Connected to RabbitMQ
>> [INFO     2012-05-27 20:30:17,872  #5451] demo_send: Received our Channel
>> [INFO     2012-05-27 20:30:17,882  #5451] demo_send: Queue Declared
>> [INFO     2012-05-27 20:30:17,883  #5451] Sending: Hello World #0: 1338147017.88308692
>> [INFO     2012-05-27 20:30:17,883  #5451] Sending: Hello World #1: 1338147017.88367391
>> [INFO     2012-05-27 20:30:17,884  #5451] Sending: Hello World #2: 1338147017.88425899
>> [INFO     2012-05-27 20:30:17,884  #5451] Sending: Hello World #3: 1338147017.88472700
>> [INFO     2012-05-27 20:30:17,885  #5451] Sending: Hello World #4: 1338147017.88527107
>> /..snip../pika/connection.py:766: UserWarning: Pika: Write buffer exceeded warning threshold at 413 bytes and an estimated 10 frames behind
>>   warn(message % (self.outbound_buffer.size, est_frames_behind))
>> /..snip../pika/connection.py:766: UserWarning: Pika: Write buffer exceeded warning threshold at 447 bytes and an estimated 11 frames behind
>>   warn(message % (self.outbound_buffer.size, est_frames_behind))
>> /..snip../pika/connection.py:766: UserWarning: Pika: Write buffer exceeded warning threshold at 490 bytes and an estimated 12 frames behind
>>   warn(message % (self.outbound_buffer.size, est_frames_behind))
>> [INFO     2012-05-27 20:30:17,887  #5451] Sending: Hello World #5: 1338147017.88733912
>> /..snip../pika/connection.py:766: UserWarning: Pika: Write buffer exceeded warning threshold at 511 bytes and an estimated 13 frames behind
>>   warn(message % (self.outbound_buffer.size, est_frames_behind))
>> /..snip../pika/connection.py:766: UserWarning: Pika: Write buffer exceeded warning threshold at 545 bytes and an estimated 14 frames behind
>>   warn(message % (self.outbound_buffer.size, est_frames_behind))
>> /..snip../pika/connection.py:766: UserWarning: Pika: Write buffer exceeded warning threshold at 588 bytes and an estimated 15 frames behind
>>   warn(message % (self.outbound_buffer.size, est_frames_behind))
>> 
>> etc etc
>> 
>> On the other hand, everything works fine with pika twisted and the msgs do end up on RabbitMQ with pika twisted, so it doesn't seem to be a restriction to do with RabbitMQ per se.
>> 
>> I've created gists for the pika native code which doesn't work and the pika twisted code that does work.
>> 
>> If anyone could help, I would greatly appreciate it.
>> 
>> James
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> 
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss