[rabbitmq-discuss] "None of the specified endpoints were reachable"

[Tony Garnock-Jones]

Hi Tammo,

Tammo.Filusch at btc-ag.com wrote:
> None of the specified endpoints were reachable
> Endpoints attempted:
>   endpoint=amqp-0-8://localhost:5672, attempts=1, outcome=Normalerweise darf jede Socketadresse (Protokoll, Netzwerkadresse oder Anschluss) nur jeweils einmal verwendet werden 127.0.0.1:5672
> Stack trace:
>    bei RabbitMQ.Client.ConnectionFactory.CreateConnection(Int32 maxRedirects, AmqpTcpEndpoint[] endpoints)
>    bei RabbitMQ.Client.ConnectionFactory.CreateConnection(AmqpTcpEndpoint[] endpoints)
>    bei RabbitMQ.Client.ConnectionFactory.CreateConnection(String address)
>    bei ...SendDataViaTcp(dataRequest request) in ...\RmqServiceClient.cs:Line 46.

This error text is associated with error code 10048, WSAEADDRINUSE.

> looking at the "outcome" part of the error message, it seems as if
> the connection from the client to the broker is not closed properly
> and because of that all possible sockets are taken.

Exactly -- but it's not necessarily because the connection isn't closed
properly! (Though, of course, it could be -- please try your system's
equivalent of netstat after 1000 messages or so, and see if there are
around 1000 active connections lying around.) I suspect TCP's TIME_WAIT
state, where if many connections are rapidly opened and closed, the TCBs
are kept around for a while to let delayed segments drop out of the
network. After a while, these can build up, leading to client-side TCP
port exhaustion.

> the following code is executed for every single message. after about
> 3800 messages had been transfered and consumed successfully the above
> error message is thrown by the sending client while executing the
> first line:
> using (IConnection conn = new ConnectionFactory().CreateConnection(serverAddress))
>                 { [...] }

Over how long a period are these 3800 connections opened and closed? If
it's reasonably quickly -- a couple of minutes, say -- it could well be
TIME_WAIT causing the issue.

> are there any obvious mistakes in my pieces of code?

Well... if possible, open *one* connection, and reuse it! :-)

Regards,
  Tony
-- 
 [][][] Tony Garnock-Jones     | Mob: +44 (0)7905 974 211
   [][] LShift Ltd             | Tel: +44 (0)20 7729 7060
 []  [] http://www.lshift.net/ | Email: tonyg at lshift.net