[rabbitmq-discuss] JAVA client / non daemon threads / shutdown hook

Thu May 24 10:41:55 BST 2012

Hi Steve,

Many thanks for your prompt response. I have two proposals about improving
it, while keeping all parties happy. Here they are:

1. Add a boolean property "daemonBackgroundThreads" in ConnectionFactory.
Based on that property the threads created by client library would be
either daemon or not. The default would be false, what wouldn't occur any
changes in behavior of existing applications.

2. Alternatively the library could be rafactored that way
that connection/publishing stuff be daemon threads, while listener stuff
are non daemon. But as I explain further it is not best solution.

I see this problem in a bit other way, from perspective of developer, who
not always can design clean solutions. Sometimes I have to wire and patch
legacy systems ;). I think it is very common task to rabbitMQ users, who
have to wire it into existing ecosystem. My case here is to allow
some proprietary web server to send some messages during request
processing. So basically I'm restricted to expose a single Java method to
be called from the web script. I can create my own threads, connections,
bookkeeping, connection failover, reconnect etc... but this all is hidden
from the interface (single method "post") exposed to web scripting. The
most ugly thing is that I have no other way of detecting that WebServer is
going to die than adding shutdown hook to VM. And shutdown hook is fired,
when all normal threads are about to die (along with daemon threads). This
scenario is probably quite common for live integration of RabbitMQ into
existing systems.

In general when dealing with messages we can have 3 basic scenarios:
1. Publisher.
2. Synchronous consumer.
3. Asynchronous consumer (listener on messages).

The idea of writing RabbitMQ clients (implied by costly and limited TCP/IP
sockets) is to share a connection "per application" and write code that
can reconnect in case of RabbitMQ node fail. So as soon as I get
the connection, I keep it globally (shared). Usually it is my
responsibility to clean on shutdown, but there are cases when I'm unable to
do this. The shutdown hooks are handy in those scenarios.

Only the 3rd scenario (asynchronous consumer) may require non daemon
threads. But it is from my experience not a live scenario. You usually do
not write application, relaying on created behind the scene "main loop".
Why? Because even if all what you do is to listen to messages, you still
have to control the application state. You must start and stop that
application as well as allow user/OS to close it. I personally for such
micro-applications use Apache Daemons and run them as Windows services. On
Linux you have jsvc to control such application. Surely "main loop" hidden
in implementation is bad idea.

On Wed, May 23, 2012 at 5:18 PM, Steve Powell <steve at rabbitmq.com> wrote:

> Bartłomiej,
>
> What a very good point. It is clear that the MainLoop thread ought to be
> a daemon thread in your circumstances, thank you for pointing this out.
> The only affect of this will be to allow the JVM to terminate if only
> daemon threads are available.
>
> However, we have considered this problem before (internal bug 21110) and
> user apps that create a connection and register some consumers (which do
> all the work) ought to be able to terminate without accidentally killing
> the connection processing.
>
> This means that there are circumstances in which the threads we create
> are required to be non-daemon, and circumstances where you would like
> them to be daemon.
>
> The hook you implemented ought to work correctly even when the MainLoop
> is not a daemon, because closing the connection ought to make the
> MainLoop thread terminate normally. If it doesn't, there is a bug. Where
> at all possible you should attempt to close an open connection as a part
> of your termination processing as there are system resources that could
> be left high-and-dry if you do not.
>
> There are other threads in the Java Client -- the executor worker
> threads used for Consumer callbacks. These are non-daemon, too. The hook
> should still work because shutting down the connection ought to shutdown
> the consumer work service, and in turn the executor (and its worker
> threads).
>
> However, I could make them daemon threads as well, in case the main app
> terminates abruptly and expects to be able to terminate uncleanly as you
> describe.
>
> Thank you for reporting this. I'll report back here on progress.
>
> Steve Powell  (a happy bunny)
> ----------yet more definitions from the SPD----------
> corrugate (n.) T.V. soap scandal.
> olympic (n.) A camp road-digger.
> jamboree (n.) A conserve made from French cheese.
>
> On 18 May 2012, at 16:27, Bartłomiej Prokop wrote:
>
> > Hi,
> >
> > I'm using JAVA client for RabbitMQ (com.rabbitmq:amqp-client:2.8.1) to
> write a "jar component" capable of sending messages for some legacy system.
> The idea is to wrap all code that maintain the connection inside my
> component. This way, the client software deals only with very simple
> methods like "post" and is not aware of any connection handling. The
> connect/reconnect code is written and hidden from the legacy system.
> >
> > The problem I have faced is that Java client creates "behind the scene"
> some threads to manage connection - like:
> >          lines 299-301 of AMQConnection class.
> >         // start the main loop going
> >         new MainLoop("AMQP Connection " + getHostAddress() + ":" +
> getPort()).start();
> >         // after this point clear-up of MainLoop is triggered by closing
> the frameHandler.
> >
> > Unfortunatelly, those threads aren't "daemon" threads. So, when main
> application ends and appropriate connection closing not occurs, the VM
> won't terminate. My approach was to add some shutdown hook to close
> RabbitMQ connections if it is live inside my "jar component". But, due to
> those non-daemon threads, VM is not going ever to be terminated and
> shutdown hooks fired.
> >
> > It is a question to RabbitMQ driver developers, if the internal threads
> could be fired as daemon threads, could it be done in future releases?
> >
> > _______________________________________________
> > rabbitmq-discuss mailing list
> > rabbitmq-discuss at lists.rabbitmq.com
> > https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>

-- 
Bart Prokop
tel. +48 509 258 502

This e-mail is intended solely for the addressee(s) and contains
confidential information. Unauthorized distribution, modification or
disclosure of its contents is unlawful. If you have received this e-mail in
error, please notify the sender immediately by return e-mail. Please then
delete the e-mail from your system and do not copy it or disclose its
contents to any person. Email transmission cannot be guaranteed to be
secure or error free as information could be intercepted, corrupted, lost,
destroyed, arrive late or incomplete, or contain viruses. The sender
therefore does not accept liability for any errors or omissions in the
contents of this message which arise as a result of email transmission.
Information, opinions or conclusions contained in this message that do not
relate to the official business of the senders employer or principal will
be understood as neither given nor endorsed by it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120524/ae8c30e5/attachment.htm>