[rabbitmq-discuss] A cluster on a single machine

[Simon MacMullen]

On 04/05/12 17:46, David Fooks wrote:
> Hi,
>
> I'm trying to emulate the guide here:
> http://www.rabbitmq.com/clustering.html
>
> But I'm having a lot of issues! Every time I run I seem to be getting a
> different crash. Is this feature actually supported in 2.8.1?

Yes! But running a cluster on a single machine is more fiddly than 
running it on separate machines as you need to make sure the nodes don't 
tread on each other. I don't think that's your problem though.

> Is there a proper way to completely reset all of the state that rabbitmq
> creates about other nodes/the cluster? My clean up script at the moment
> looks like this:
>
> rm /var/run/rabbitmq/*
> rm -f /var/log/rabbitmq/*
> rm -rf /var/lib/rabbitmq/*

OK, that should be fine.

> Here is my start-up script (I'm running this with sudo):
>
> RABBITMQ_NODE_PORT=5672 RABBITMQ_NODENAME=rabbit1
> RABBITMQ_SERVER_START_ARGS="-rabbitmq_mochiweb listeners
> [{mgmt,[{port,55672}]}]" rabbitmq-server -detached
> RABBITMQ_NODE_PORT=5673 RABBITMQ_NODENAME=rabbit2
> RABBITMQ_SERVER_START_ARGS="-rabbitmq_mochiweb listeners
> [{mgmt,[{port,55673}]}]" rabbitmq-server -detached
> RABBITMQ_NODE_PORT=5674 RABBITMQ_NODENAME=rabbit3
> RABBITMQ_SERVER_START_ARGS="-rabbitmq_mochiweb listeners
> [{mgmt,[{port,55674}]}]" rabbitmq-server -detached
>
> ps -ef | grep rabbit1 | grep -v grep | awk '{print $2}' >
> /var/run/rabbitmq/rabbit1.pid
> ps -ef | grep rabbit2 | grep -v grep | awk '{print $2}' >
> /var/run/rabbitmq/rabbit2.pid
> ps -ef | grep rabbit3 | grep -v grep | awk '{print $2}' >
> /var/run/rabbitmq/rabbit3.pid

Ew, those ps'es are not nice. You may prefer to set RABBITMQ_PID_FILE 
for each startup, although this means you cannot use -detached (instead 
just background it with &)

> rabbitmqctl -n rabbit1 wait /var/run/rabbitmq/rabbit1.pid
> rabbitmqctl -n rabbit2 wait /var/run/rabbitmq/rabbit2.pid
> rabbitmqctl -n rabbit3 wait /var/run/rabbitmq/rabbit3.pid
>
> rabbitmqctl -n rabbit1 stop_app
> rabbitmqctl -n rabbit1 reset
> rabbitmqctl -n rabbit1 start_app
>
> rabbitmqctl -n rabbit2 stop_app
> rabbitmqctl -n rabbit2 reset
> rabbitmqctl -n rabbit2 start_app
>
> rabbitmqctl -n rabbit3 stop_app
> rabbitmqctl -n rabbit3 reset
> rabbitmqctl -n rabbit3 start_app

> rabbitmqctl -n rabbit1 status
> rabbitmqctl -n rabbit2 status
> rabbitmqctl -n rabbit3 status
>
> rabbitmqctl -n rabbit3 stop_app
> rabbitmqctl -n rabbit3 cluster rabbit1 at dfooks-ubuntu64
> rabbit2 at dfooks-ubuntu64
> rabbitmqctl -n rabbit3 start_app

But here's the problem. You need to do:

rabbitmqctl -n rabbit3 stop_app
rabbitmqctl -n rabbit3 reset
rabbitmqctl -n rabbit3 cluster rabbit1 at dfooks-ubuntu64 \
     rabbit2 at dfooks-ubuntu64
rabbitmqctl -n rabbit3 start_app

Instead you are doing something like (considering only rabbit3):

rabbitmqctl -n rabbit3 stop_app
rabbitmqctl -n rabbit3 reset
rabbitmqctl -n rabbit3 start_app
rabbitmqctl -n rabbit3 stop_app
rabbitmqctl -n rabbit3 cluster rabbit1 at dfooks-ubuntu64 \
     rabbit2 at dfooks-ubuntu64
rabbitmqctl -n rabbit3 start_app

Clustering can only take place when the state is *completely* reset - 
the fact that you are starting and stopping the app again between reset 
and cluster is recreating a blank database...

<snip>
> {merge_schema_failed, "Incompatible schema cookies.
<snip>

...which is exactly what would lead to this error message.

And yes, at some point we hope to make clustering easier to use.

Cheers, Simon

-- 
Simon MacMullen
RabbitMQ, VMware