[rabbitmq-discuss] RabbitMQ 2.5.0 cluster cannot start

Fri Jun 17 23:07:48 BST 2011

I have a three-node rabbit cluster (running on hosts kvm-alpha,
kvm-beta, and kvm-gamma) that I just upgraded to 2.5.0, and it's
misbehaving a bit.  After upgrading, I wiped out the mnesia dirs
(/var/lib/rabbitmq/mnesia) on each of the three machines, and then
attempted to start the cluster up again.  I first started rabbit on
kvm-beta, and when I did that the rabbits on alpha and gamma could not
start.  I stopped the rabbit on beta, and started it on alpha and
gamma, and those are happy talking to each other.  In both cases, the
error that I'm getting looks like this:

starting database
...BOOT ERROR: FAILED
Reason: {error,
            {unable_to_join_cluster,
                ['rabbit at kvm-alpha','rabbit at kvm-beta','rabbit at kvm-gamma'],
                {aborted,
                    {function_clause,
                        [{mnesia_schema,cs2list,
                             [{cstruct,schema,set,[],
                                  ['rabbit at kvm-gamma','rabbit at kvm-alpha'],
                                  [],0,read_write,[],[],false,schema,
                                  [table,cstruct],
                                  [],[],
                                  {{1308,347724,86957},'rabbit at kvm-alpha'},
                                  {{4,0},
                                   {'rabbit at kvm-gamma',
                                       {1308,347733,438674}}}}]},
                         {mnesia_schema,do_merge_schema,1},
                         {mnesia_tm,apply_fun,3},
                         {mnesia_tm,execute_transaction,5},
                         {mnesia_schema,schema_coordinator,3}]}}}}
Stacktrace: [{rabbit_mnesia,init_db,3},
             {rabbit_mnesia,init,0},
             {rabbit,'-run_boot_step/1-lc$^1/1-1-',1},
             {rabbit,run_boot_step,1},
             {rabbit,'-start/2-lc$^0/1-0-',1},
             {rabbit,start,2},
             {application_master,start_it_old,4}]
Erlang has closed

that one's on kvm-beta with rabbits running happily on kvm-alpha and
gamma, but it looks the same if I start on beta first than then let
alpha or gamma fail.  The cookie hash is the same for each rabbit, but
the erlang version on alpha and gamma is 5.8.3, while beta has 5.8.4.
Is that likely to be the problem?  I thought minor releases of erl
were compatible, but maybe they aren't from rabbit's POV?  If that
isn't likely to be the case, then I'd be happy to take suggestions
about what the problem probably is.

My config file is the same on all three nodes, and looks like this:

kvm-beta watched # cat /etc/rabbitmq/rabbitmq.config
[
 {rabbit, [
  {cluster_nodes, ['rabbit at kvm-alpha', 'rabbit at kvm-beta', 'rabbit at kvm-gamma']}
 ]}
].

I'd be happy to provide any other info that could be useful.