[rabbitmq-discuss] AMQPChannelException: (404, u"NOT_FOUND - no exchange '4042acd4bbec471aafc557af39ee0efe' in vhost '/'", (60, 40), 'Channel.basic_publish')

[Alex Lyakas]

Greetings all,

I am using rabbitmq server + kombu + pyamqplib on a stock Ubuntu-Precise. 
Below are the versions of the components:
rabbitmq-server 2.7.1-0ubuntu4
python-amqplib  1.0.0+ds-1
python-kombu 1.4.3-1

The issue I am seeing is the “no exchange” exception, which I hit when 
opening a new channel on an existing connection. Here is a typical crash 
stack:
TRACE: File "/usr/lib/python2.7/dist-packages/kombu/connection.py", line 
159, in channel
TRACE: chan = self.transport.create_channel(self.connection)
TRACE: File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", 
line 235, in create_channel
TRACE: return connection.channel()
TRACE: File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", 
line 144, in channel
TRACE: return Channel(self, channel_id)
TRACE: File "/usr/lib/python2.7/dist-packages/kombu/transport/pyamqplib.py", 
line 183, in init
TRACE: super(Channel, self).__init__(*args, **kwargs)
TRACE: File 
"/usr/lib/python2.7/dist-packages/amqplib/client_0_8/channel.py", line 82, 
in init
TRACE: self._x_open()
TRACE: File 
"/usr/lib/python2.7/dist-packages/amqplib/client_0_8/channel.py", line 471, 
in _x_open
TRACE: (20, 11), # Channel.open_ok
TRACE: File 
"/usr/lib/python2.7/dist-packages/amqplib/client_0_8/abstract_channel.py", 
line 97, in wait
TRACE: return self.dispatch_method(method_sig, args, content)
TRACE: File 
"/usr/lib/python2.7/dist-packages/amqplib/client_0_8/abstract_channel.py", 
line 115, in dispatch_method
TRACE: return amqp_method(self, args)
TRACE: File 
"/usr/lib/python2.7/dist-packages/amqplib/client_0_8/channel.py", line 273, 
in _close
TRACE: (class_id, method_id))
TRACE: AMQPChannelException: (404, u"NOT_FOUND - no exchange 
'4042acd4bbec471aafc557af39ee0efe' in vhost '/'", (60, 40), 
'Channel.basic_publish')

I hit this only with direct exchanges, with “auto-delete” = True.

The flow that I have is like follows:
- node A declares a direct exchange and declares a queue
- node A sends message to node B using a topic exchange, and embeds direct 
exchange name in the message
- node B parses the message, prepares the reply, declares the (same) direct 
exchange and a producer
- node B sends reply message on the producer
- node A and node B close their channels
- several seconds after that, when node B tries to create a new channel on 
the same connection (using the same channel_id), it hits this exception.

I am trying to debug this for some time, but do not have a clear way to 
repro this. It happens occasionally.

I understand, that since I open the channel with the same channel_id, I may 
receive some stale message from the server (for the same channel_id) 
regarding failure to publish. However, several things confuse me:

1) The only code, which publishes to a direct exchange is as follows:
self.exchange = kombu.entity.Exchange(name=self.exchange_name, 
**self.kwargs)
self.producer = kombu.messaging.Producer(exchange=self.exchange, 
channel=channel, routing_key=self.routing_key)
self.producer.publish(msg)
How can exchange not being present here?

2) Even though all exchanges have auto-delete=True, I see more than 3000 
existing direct exchanges on the rabbitmqctl list_exchanges output. The 
queues that used these exchanges (also auto-delete=True) do not exist 
anymore.
When auto-delete=True exchanges are deleted? Is there some timeout on their 
deletion? The documentation says that there SHOULD be a reasonable timeout. 
What is this timeout? Can it be configured? Is exchange auto-deletion 
supposed to work in my version?

3) The exchanges that I receive exceptions about do not exist indeed. How 
can I debug the rabbitmq server and understand why it decides to delete some 
exchanges, but not the other ones?

Thanks,
Alex.