[rabbitmq-discuss] How do you handle recovering from a faulty connection using RabbitMQ java client library?

[Peter Moberg]

Thanks for your input! Good to hear that option C is a viable approach. The
Lyra project looks interesting, will keep an eye on it.

/Peter


On Fri, Nov 1, 2013 at 3:02 AM, Mike Hadlow <mike at suteki.co.uk> wrote:

> Hi Peter,
>
> My .NET client library, EasyNetQ (http://easynetq.com) does automatic
> reconnection. We use option C with a class called PersistentConnection. An
> internal in-memory event bus then distributes connection events to the rest
> of the library so consumers get notified when they need to re-consume.
> Works well.
>
> Good luck,
> Mike
>
>
> On Fri, Nov 1, 2013 at 4:13 AM, Jonathan Halterman <jhalterman at gmail.com>wrote:
>
>> Hi Peter,
>>
>> Solution C is actually pretty reasonable. There's not much to gain from
>> using multiple connections to the same server if you're trying to guard
>> against network failures or cluster partitions. If one connection dies,
>> they all likely will. Wrapping and recovering connections/channels works
>> fine, and as Michael mentioned, you might also check out Lyra since it
>> handles the various corner cases involved in recovering resources for you.
>>
>> Cheers,
>> Jonathan
>>
>>
>> On Thu, Oct 31, 2013 at 10:28 AM, Peter Moberg <moberg.peter at gmail.com>wrote:
>>
>>> Hi,
>>>
>>> I'm interested in knowing how other people handle recovering from a
>>> faulty connection using the official RabbitMQ java client library. We are
>>> using it to connect our application servers to our RabbitMQ cluster and we
>>> have implemented a few different ways to recover from a connection failure,
>>> but non of them feel quite right.
>>>
>>> Imagine this pseudo application:
>>>
>>> public class OurClassThatStartsConsumers {
>>>     Connection conn;
>>>
>>>     public void start() {
>>>         ConnectionFactory factory = new ConnectionFactory();
>>>         factory.setUsername("someusername");
>>>         factory.setPassword("somepassword");
>>>         factory.setHost("somehost");
>>>         conn = factory.newConnection();
>>>
>>>         new Thread(new Consumer(conn.createChannel())).start();
>>>     }
>>>   }
>>> class Consumer1 implements Runnable {
>>>     public Consumer1(Channel channel) {
>>>          this.channel = channel;
>>>     }
>>>
>>>     @Override
>>>     public void run() {
>>>         while (true) {
>>>              ... consume incoming messages on the channel...
>>>             // How do we handle that the connection dies?
>>>         }
>>>     }}
>>>
>>> In the real world we have several hundreds of consumers. So what happens
>>> if the connection dies? In the above example Consumer1 can not recover,
>>> when the connection closes, the Channel also closes, a state from which we
>>> can not recover. So lets look at some ways to solve this:
>>>
>>> Solution A)
>>>
>>> Let every consumer have their own connection and register the events
>>> that triggers when the connection dies and then handle reconnecting.
>>>
>>> Pros: It works
>>>
>>> Cons:
>>>
>>>    - Since we have a lot of consumers, we probably do not want that
>>>    many connections.
>>>    - We might possible have a lot of duplicated code for reconnecting
>>>    to rabbit and handle reconnecting
>>>
>>> Solution B)
>>>
>>> Have each consumer use the same connection and subscribe to its
>>> connection failure events.
>>>
>>> Pros: Less connections than in Solution A
>>>
>>> Cons: Since the connection is closed we need to reopen/replace it. The
>>> java client library doesn't seem to provide a way to reopen the connection,
>>> so we would have to replace it with a new connection and then somehow
>>> notify all the consumers about this new connection and they would have to
>>> recreate the channels and the consumers. Once again, a lot of logic that i
>>> don't want to see in the consumer ends up there.
>>>
>>> Solution C)
>>>
>>> Wrap Connection and Channel classes is classes that handle the
>>> re-connection logic, the consumer only needs to know about the
>>> WrappedChannel class. On a connection failure the WrappedConnection with
>>> deal with re-establishing the connection and once connected the
>>> WrappedConnection will automatically create new Channels and register
>>> consumers.
>>>
>>> Pros: It work - this is actually the solution we are using today.
>>>
>>> Cons: It feels like a hack, I think this is something that should be
>>> handled more elegantly by the underlying library.
>>>
>>> Maybe there is a much better way? The API documentation does not talk
>>> that much about recovering from a faulty connection. Any input is
>>> appreciated :)
>>>
>>>
>>> Thanks,
>>>
>>> Peter
>>>
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-discuss at lists.rabbitmq.com
>>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>>
>>>
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>
>>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20131104/d3f87f8b/attachment.htm>