[rabbitmq-discuss] Lost message in 50k size range

Mon Mar 12 11:17:12 GMT 2012

Hi John,

I'm not familiar with the pika code, but scanning it (version 0.9.5 for python
2.6) it looks like the callback it is complaining about is one it adds itself
on close. Are you calling (or causing to be called) close twice, do you think?

--------8<----------------------------------------------
_on_close_ready (which is a callback) calls _rpc thus:

        self._rpc(0, spec.Connection.Close(self.closing[0],
                                          self.closing[1], 0, 0),
                  self._on_connection_closed,
                  [spec.Connection.CloseOk])

and the third param is another callback (_on_connection_closed)
and _rpc adds it:

        # If we were passed a callback, add it to our stack
        if callback:
            for reply in acceptable_replies:
                self.callbacks.add(channel_number, reply, callback)

It is this callback which appears to be triggering the message.
--------------------------------------------------------

Steve Powell  (a lippy bunny)
----------some more definitions from the SPD----------
vermin (v.) Treating the dachshund for roundworm.
chinchilla (n.) Cooling device for the lower jaw.
socialcast (n.) Someone to whom everyone is speaking but nobody likes.

On 8 Mar 2012, at 14:30, John Reuning wrote:

> So, it looks like this was a flushing problem.  I either wasn't
> closing the connection properly or was terminating the ioloop too
> soon.  I fixed the message send problem but am now getting a warning
> message from the callback manager.
> 
> /mnt/hgfs/devel/seedbank/seedbank/src/pika/callback.py:72:
> UserWarning: CallbackManager.add: Duplicate callback found for
> "0:Connection.CloseOk"
>  (self.__class__.__name__, prefix, key))
> 
> It gives the warning regardless of whether or not I use
> add_on_close_callback to specify my own callback function.  Any ideas
> why this happens?
> 
> Here's the backtrace to the CallbackManager.add call that generates the message.
> 
>  File "/mnt/hgfs/devel/seedbank/seedbank/src/pika/adapters/select_connection.py",
> line 124, in start
>    self.poller.start()
>  File "/mnt/hgfs/devel/seedbank/seedbank/src/pika/adapters/select_connection.py",
> line 374, in start
>    self.poll()
>  File "/mnt/hgfs/devel/seedbank/seedbank/src/pika/adapters/select_connection.py",
> line 388, in poll
>    self._handler(events[0][0], events[0][1])
>  File "/mnt/hgfs/devel/seedbank/seedbank/src/pika/adapters/base_connection.py",
> line 134, in _handle_events
>    self._handle_read()
>  File "/mnt/hgfs/devel/seedbank/seedbank/src/pika/adapters/base_connection.py",
> line 162, in _handle_read
>    self._on_data_available(data)
>  File "/mnt/hgfs/devel/seedbank/seedbank/src/pika/connection.py",
> line 589, in _on_data_available
>    frame)                 # Args
>  File "/mnt/hgfs/devel/seedbank/seedbank/src/pika/callback.py", line
> 130, in process
>    callback(*args, **keywords)
>  File "/mnt/hgfs/devel/seedbank/seedbank/src/pika/connection.py",
> line 553, in _on_channel_close
>    self._on_close_ready()
>  File "/mnt/hgfs/devel/seedbank/seedbank/src/pika/connection.py",
> line 402, in _on_close_ready
>    [spec.Connection.CloseOk])
>  File "/mnt/hgfs/devel/seedbank/seedbank/src/pika/connection.py",
> line 620, in _rpc
>    self.callbacks.add(channel_number, reply, callback)
> 
> 
> On Tue, Mar 6, 2012 at 1:13 PM, John Reuning <john at ibiblio.org> wrote:
>> In this case, lost means the consumer never gets it.  I have a test
>> setup for debugging.  Both publisher and consumer are python/pika.
>> The publisher sends 2 messages -- one is a bson encoded python string
>> of length 49144, the other is 49145.  The consumer gets the first
>> message but not the second.  It's the same behavior for messages down
>> to 1 byte and up to a few 100 KB.  The failure point seems to be at
>> this 49145 length.
>> 
>> Other useful information may be that queue durability and message
>> delivery mode settings don't affect the behavior.  Also, your
>> suggestion of publishing to a consumerless queue didn't result in a
>> queued message, as far as I can tell.
>> 
>> I'm using the management and tracing plugins, turned on tracing, and
>> turned on pika debug logging.  I can't find any errors or indication
>> of why the message disappears or where it's going.  I'm not sure I
>> know the right places to look, though.
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss