[rabbitmq-discuss] Fwd: RabbitMQ 2.8.2 Erlang crash

Wed Jun 13 12:22:10 BST 2012

One clarification:

> If what I described above matches your situation, then you can refer
> to this other post I just sent regarding those situations:
> http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2012-June/020594.html
> .

That post advocates reseting the cluster nodes to restore a "clean"
cluster. That obviously means that you'll lose any data that resides
on that node only, for example non-HA queues. So if don't want that,
you should only follow step 1 and then start the other nodes, while
making sure that the rabbit/erlang version stays the same.

Francesco.

At Wed, 13 Jun 2012 11:53:10 +0100,
Francesco Mazzoli wrote:
> 
> Hi,
> 
> > I have a problem with starting RabbitMQ service after spliting
> > cluster.  Software crash everytime during
> > "/etc/init.d/rabbitmq-server start".
> 
> What do you mean by "splitting cluster"? I'd interpret it as "the
> connection was severed between the two nodes", but maybe it's
> something different, considering that you're having problems
> *starting* nodes. In any case, clustering in rabbit does not tolerate
> partitions well. The most it can take is simply nodes going down and
> then up again, but if the data in the tables diverge, things will go
> wrong.
> 
> In your case, since you're getting a timeout in the database boot
> step, I'd guess that for some reason mnesia thinks that it has to wait
> the tables in the other nodes first. This typically happen when
> trying to start a node of an offline cluster before starting the last
> node to go down (mnesia wants the last node to go down first because
> it has the most up-to-date data). Normally rabbit will terminate with
> a friendlier error message ("please start up the last node to go down"
> or similar) but if you shut down the nodes abruptly this won't happen
> and you'll get an hanging node.
> 
> If what I described above matches your situation, then you can refer
> to this other post I just sent regarding those situations:
> http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2012-June/020594.html
> . If it doesn't, can you describe in more detail what you are doing to
> get that? The logs would help too.
> 
> Francesco.
> 
> At Sat, 9 Jun 2012 17:24:47 +0200,
> Michał Kiędyś wrote:
> > 
> > [1  <multipart/alternative (7bit)>]
> > [1.1  <text/plain; ISO-8859-2 (quoted-printable)>]
> > 
> > [1.2  <text/html; ISO-8859-2 (quoted-printable)>]
> > 
> > ---------- Forwarded message ----------
> > From: Alexis Richardson <alexis at rabbitmq.com>
> > Date: 2012/6/8
> > Subject: Re: RabbitMQ 2.8.2 Erlang crash
> > To: Michał Kiędyś / Gadu-Gadu <m.kiedys at gadu-gadu.pl>
> > Cc: rabbitmq info <info at rabbitmq.com>
> > 
> > Michał
> > 
> > Also, if you could post this to rabbitmq-discuss, that would be helpful.
> > 
> > Best wishes
> > 
> > alexis
> > 
> > On Fri, Jun 8, 2012 at 8:34 AM, Michał Kiędyś / Gadu-Gadu
> > <m.kiedys at gadu-gadu.pl> wrote:
> > > Hello
> > >
> > > I have a problem with starting RabbitMQ service after spliting cluster.
> > > Software crash everytime during "/etc/init.d/rabbitmq-server start".
> > >
> > > Log:
> > > root at arch-task-mq-8:~# rabbitmq-server
> > > Activating RabbitMQ plugins ...
> > > 6 plugins activated:
> > > * amqp_client-2.8.2
> > > * mochiweb-1.3-rmq2.8.2-git
> > > * rabbitmq_management-2.8.2
> > > * rabbitmq_management_agent-2.8.2
> > > * rabbitmq_mochiweb-2.8.2
> > > * webmachine-1.7.0-rmq2.8.2-hg
> > >
> > > ** Found 0 name clashes in code paths
> > >
> > > +---+   +---+
> > > |   |   |   |
> > > |   |   |   |
> > > |   |   |   |
> > > |   +---+   +-------+
> > > |                   |
> > > | RabbitMQ  +---+   |
> > > |           |   |   |
> > > |   v2.8.2  +---+   |
> > > |                   |
> > > +-------------------+
> > > AMQP 0-9-1 / 0-9 / 0-8
> > > Copyright (C) 2007-2012 VMware, Inc.
> > > Licensed under the MPL.  See http://www.rabbitmq.com/
> > >
> > > node           : rabbit at arch-task-mq-8
> > > app descriptor :
> > > /usr/lib/rabbitmq/lib/rabbitmq_server-2.8.2/sbin/../ebin/rabbit.app
> > > home dir       : /var/lib/rabbitmq
> > > config file(s) : /etc/rabbitmq/rabbitmq.config
> > > cookie hash    : +MKjvqvf4/0q7+0Fhc9OEw==
> > > log            : /var/log/rabbitmq/rabbit at arch-task-mq-8.log
> > > sasl log       : /var/log/rabbitmq/rabbit at arch-task-mq-8-sasl.log
> > > database dir   : /var/lib/rabbitmq/mnesia/rabbit at arch-task-mq-8
> > > erlang version : 5.8
> > >
> > > -- rabbit boot start
> > > starting file handle cache server
> > > ...done
> > > starting worker pool
> > >  ...done
> > > starting database                                                     ...
> > >
> > > BOOT FAILED
> > > ===========
> > >
> > > Timeout contacting cluster nodes: ['rabbit at arch-task-mq-7'].
> > >
> > > DIAGNOSTICS
> > > ===========
> > >
> > > nodes in question: ['rabbit at arch-task-mq-7']
> > >
> > > hosts, their running nodes and ports:
> > > - arch-task-mq-7: [{rabbit,46044}]
> > >
> > > current node details:
> > > - node name: 'rabbit at arch-task-mq-8'
> > > - home dir: /var/lib/rabbitmq
> > > - cookie hash: +MKjvqvf4/0q7+0Fhc9OEw==
> > >
> > >
> > > {"Kernel pid
> > > terminated",application_controller,"{application_start_failure,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot}}}}}"}
> > >
> > > Crash dump was written to: erl_crash.dump
> > > Kernel pid terminated (application_controller)
> > > ({application_start_failure,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot}}}}})
> > >
> > >
> > >
> > > Regards,
> > >
> > > --
> > > Michał Kiędyś
> > > web architect / Pion Aplikacji Internetowych
> > >
> > > GG Network S.A
> > > ul. Kamionkowska 45
> > > 03-812 Warszawa
> > > tel.: +48 22 427 79 00
> > > fax.: +48 22 427 74 25
> > > gg:16698
> > >
> > > Spółka zarejestrowana w Sądzie Rejonowym dla m. st. Warszawy, XIII
> > > Wydział Gospodarczy KRS pod numerem 0000264575, NIP 867-19-48-977.
> > > Kapitał zakładowy:  1 758 461,10 zł - wpłacony w całości.
> > >
> > 
> > 
> > [2  <text/plain; us-ascii (7bit)>]
> > _______________________________________________
> > rabbitmq-discuss mailing list
> > rabbitmq-discuss at lists.rabbitmq.com
> > https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss