[rabbitmq-discuss] Tutorial example suggestion: work queues tutorial could have a description of hierarchical shutdown logic

[Ann Witbrock (c)]

Hi Lowell,
thanks for the suggestion.
I agree that a tutorial on this (shutdown, auto-delete and similar parameters, and related matters) would be useful, but I don't agree that tutorial is the right place for it. I already have that in as a proposed work item. This example could well be helpful in it.

Ann

----- Original Message -----
From: "Lowell Boggs" <Lowell.Boggs at emc.com>
To: david at rabbitmq.com
Cc: rabbitmq-discuss at lists.rabbitmq.com
Sent: Thursday, 7 July, 2011 4:00:47 PM
Subject: [rabbitmq-discuss] Tutorial example suggestion: work queues tutorial could have a description of hierarchical shutdown logic

Thanks for this great advice.  

I looked for an example of how to handle shutdown and other system commands in this tutorial:

   http://www.rabbitmq.com/tutorials/tutorial-two-python.html

But not finding any advice there, I tried to come up with my own idea based on concepts that I gleaned from the rabbitmq-c interfaces themselves and the tutorial examples.  I see the benefits of your suggestion below.

I would like to suggest that the work queues tutorial is a great place to insert the important information you provided in the attached email about exchange hierarchies.

Note that it would not be necessary to provide example code for this to be a helpful addition.  If you understand the work queues logic, you can understand a description of the shutdown logic without seeing it worked out.  

Also, I have not found (maybe I just haven't looked hard enough) a good explanation of why I should prefer one exchange type over the other.  Direct seems to be doing everything I need.  I am a bit concerned about the performance impact of of using topic name patterns.  The shutdown example you give below would be a nice place in the tutorials to comment on which exchange type is best suited for what.

Thanks again!

Lowell


-----Original Message-----
From: David Wragg [mailto:david at rabbitmq.com] 
Sent: Wednesday, July 06, 2011 5:35 PM
To: Boggs Jr., Lowell
Cc: rabbitmq-discuss at lists.rabbitmq.com
Subject: Re: [rabbitmq-discuss] rabbitmq-c: amqp_exchange_declare() always sets auto-delete to no

Hi Lowell,

<Lowell.Boggs at emc.com> writes:
> Is there a reason that you cannot create an exchange with the
> auto_delete property == true?

Auto-delete on exchanges is a AMQP 0-8 feature that is deprecated in
AMQP 0-9-1.  This is why the amqp_exchange_declare function does not
expose it (ever since it was converted to AMQP 0-9-1).

> Is it safe for me as an application devloper to create a clone of this
> function that has the auto_delete property passed in?  Does the
> rabbitmq team have plans to change the way the underlying functions
> work?

It is safe, but not a good idea.

rabbitmq-server still supports AMQP 0-8, including the auto-delete
property on exchanges.  So if you copy amqp_exchange_declare and add
that parameter, it will work today.  But at some point, rabbitmq-server
may drop AMQP 0-8 compatibility, and then it would stop working.

> Is a temporary exchange even a good idea?

Not really.

> I am trying to simplify the shutdown logic of a multi-threaded
> application.  I don't want to have to invent a way of signaling my
> worker threads to shut down.  Instead, I want to send them a message
> that they can read like normal but have a quick way of identifying
> that this is a shutdown command. I am currently writing code for my
> threads that work like this:
>
> void Thread()
> {
>
>     // create the connection
>     // declare the primary command exchange
>     // declare the qrimary queue
>     // declare a "stop command" exchange
>     // declare a "stop command" queue
>     // bind everyone up and call basic_consume on the queues
>
>
>     for(;;)
>      {
>         string exchangeName;
>         string topicName;
>         string messageBody;
>
>          int error = readMessage(exchangeName, topicName, messageBody);
>
>          if(error) break;
>
>         if(exchangeName == "stop")
>            break;
>
>        processMesage(messageBody);
>
>     }
> }
>
> Note that I am having multiple threads in multiple executables on
> multiple machines all reading from the same primary command exchange
> for performance and redundancy reasons -- so I can't really use the
> topic to indicate shutdown.  I might want to shutdown a given
> executable on a given host, not the entire system.
>
> The above logic is working fine, but when I kill a process, I have to
> remember to delete the exchanges because they don't have an autoDelete
> flag.
>
> Is there a better way to handle this shutdown logic?

It sounds like you have a hierarchy of threads, processes, and machines.
And you want to send command messages (including shutdown messages) to
the various levels in that hierarchy: To a thread, to all the threads
in a process, to all the threads in all processes on a host, and to all
threads everywhere.

You can accomplish this by declaring four exchanges. Let's call them
"per-thread", "per-process", "per-host" and "everything".

Each thread declares an auto-delete queue for itself, and binds this
queue to all four exchanges, but with different binding keys.

"per-thread" should be a direct exchange.  A thread binds its queue to
this exchange  with a binding key of the form "<host_id>.<process_id>.<thread_id>".

"per-process" should be a direct exchange.  A thread binds its queue to
this exchange with with a binding key of the form
"<host_id>.<process_id>".

"per-host" should be a direct exchange.  A thread binds its queue to
this exchange with with a binding key of the form "<host_id>"

Finally, "everything" can just be a fanout exchange (and so the binding
key is not used).

In order to send a command, use the exchange for the appropriate level
in the hierarchy, and the corresponding routing key.

Because all four exchanges are long lived, your code should not need to
delete any of them.

-- 
David Wragg
Staff Engineer, RabbitMQ
VMware, Inc.
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss at lists.rabbitmq.com
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss