[rabbitmq-discuss] A weird case of "PRECONDITION_FAILED - unknown delivery tag"

[Razvan Dinu]

Hi Emile, thanks for your reply.

As I mentioned I'm on a python oriented environment, so can't use the
Tracer you provided directly. I look for an equivalent or if not I'll try
to use that one. However, i'm pretty sure that the ACKs are sent correctly
(there's only one line with basic_ack in my code, and from the logging
messages it looks like it is called correctly). Nevertheless I'll use the
tracer approach just to make sure.

About synchronizing the two threads that are using the channel I don't
think it's possible. I could synchronize sending the ACK messages on the
worker thread but the consumer thread can wait indefinitely for messages to
come. And I don't see where I would have the opportunity to take a lock
since once the consumer thread starts listening, we have no control over
it. We only get the callback when a message arrives. I guess the general
question here would be: what is the correct way of fetching the messages on
one thread and doing the work on another (from a synchronization point of
view)?

And to explain a bit more the "peeking at messages" scenario, here's the
reason. Imagine each task requires fetching some data from a database
(located on a different server). As always, doing bulk operations is a lot
faster. So when the worker needs to fetch data for one task, from the
database server, it will check if there are other tasks in the local queue
that also need data, and request everything in bulk. So when the 1st task
is processed, data is fetched also for task 2,3,4..10. So the processing
for those tasks will be much faster afterwards, because they'll find the
data they need in a local cache. Then task 11 will probably fetch the data
for tasks 11 to 20 and so on. It's an optimization I'm making on the client
side improve the speed.

I guess in a few words what I'm trying to implement is bulk processing on
the client side. Maybe there's a different way to do this.

Cheers,
Raz


On Wed, Aug 28, 2013 at 4:21 PM, Emile Joubert <emile at rabbitmq.com> wrote:

>
> Hi,
>
> On 28/08/13 13:30, Razvan Dinu wrote:
>
> > it's usually because of double ack-ing, ack-ing on wrong channels or
> > ack-ing messages that should not be ack-ed. I've checked at least a
> > dozen times, and it doesn't seem to be the case for me.
>
> A good way to verify that claim independently from the client code is to
> connect to the broker via the Tracer:
>
> http://www.rabbitmq.com/javadoc/com/rabbitmq/tools/Tracer.html
>
> The protocol trace will show whether messages are being acknowledged
> correctly.
>
> > I think it has something to do with ACK-ing from one thread using the
> > same channel that is used for listening on another thread.
>
> If you use multiple threads to access the same channel then you must
> make sure they are synchronised. A better solution is to provide each
> thread with its own channel.
>
> > it's because I need the worker to be able to take a peak at some of
> > the tasks in the queues.
>
> Can the criteria being peeked at be placed in a header or the routing
> key instead? Try to route those messages differently and avoid the need
> for peeking.
>
>
>
>
> -Emile
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130828/b9d5cc47/attachment.htm>