[rabbitmq-discuss] Management UI plugin issue on 2 node cluster

Jos Boumans jos at kruxdigital.com
Mon Apr 11 23:10:22 BST 2011


Hi,

I've just upgraded my 2 node cluster from 2.4.0 to 2.4.1 on Ubuntu
10.04.2 on ec2. Both nodes have the plugins installed for the
management UI (+agent) and the stomp adapter.

The second node in the cluster won't start with the management UI
installed. If i remove the UI and leave the agent + stomp adapter,
it starts and joins the cluster as expected. This configuration
worked fine on 2.4.0.

I have a huge erlang crash dump file, but nothing that seems to
indicate anything about the management plugin. The error log shows:

   Kernel pid terminated (application_controller)  
({application_terminated,rabbit,shutdown})

and not much more than that.

The full output of a console session is below the sig showing how
it works without the UI plugin, and fails when enabling it.

The full erlang crash dump is available here:

   http://krux-temp.s3.amazonaws.com/erl_crash.dump

Can someone help me diagnose and address the problem?

I'm also found on Freenode/#rabbitmq as 'jiboumans'.

Thank you,

--

	Jos Boumans

	"Whenever you find you are on the side of the majority,
	it is time to pause and reflect." - Mark Twain



ubuntu at rabbit-a002:/usr/lib/rabbitmq/lib/rabbitmq_server-2.4.1/plugins 
$ ls
amqp_client-2.4.1.ez  rabbitmq-management-agent-2.4.1.ez   
rabbit_stomp-2.4.1.ez
mochiweb-2.4.1.ez     rabbitmq-mochiweb-2.4.1.ez           
webmachine-2.4.1.ez

ubuntu at rabbit-a002:/usr/lib/rabbitmq/lib/rabbitmq_server-2.4.1/plugins 
$ sudo rabbitmqctl status
Status of node 'rabbit at rabbit-a002' ...
[{pid,30804},
  {running_applications,
      [{webmachine,"webmachine","1.7.0"},
       {rabbit_management_agent,"RabbitMQ Management Agent","2.4.1"},
       {amqp_client,"RabbitMQ AMQP Client","2.4.1"},
       {rabbit_mochiweb,"RabbitMQ Mochiweb Embedding","2.4.1"},
       {mochiweb,"MochiMedia Web Server","1.3"},
       {inets,"INETS  CXC 138 49","5.2"},
       {rabbit_stomp,"Embedded Rabbit Stomp Adapter","2.4.1"},
       {rabbit,"RabbitMQ","2.4.1"},
       {mnesia,"MNESIA  CXC 138 12","4.4.12"},
       {os_mon,"CPO  CXC 138 46","2.2.4"},
       {sasl,"SASL  CXC 138 11","2.1.8"},
       {stdlib,"ERTS  CXC 138 10","1.16.4"},
       {kernel,"ERTS  CXC 138 10","2.13.4"}]},
  {nodes,[{disc,['rabbit at rabbit-a002','rabbit at rabbit-a001']}]},
  {running_nodes,['rabbit at rabbit-a001','rabbit at rabbit-a002']}]
...done.

ubuntu at rabbit-a002:/usr/lib/rabbitmq/lib/rabbitmq_server-2.4.1/plugins 
$ sudo mv /tmp/rabbitmq-management-2.4.1.ez .

ubuntu at rabbit-a002:/usr/lib/rabbitmq/lib/rabbitmq_server-2.4.1/plugins 
$ sudo /etc/init.d/rabbitmq-server restart
Restarting rabbitmq-server: FAILED - check /var/log/rabbitmq/ 
startup_{log, _err}
rabbitmq-server.

ubuntu at rabbit-a002:/usr/lib/rabbitmq/lib/rabbitmq_server-2.4.1/plugins 
$ cat  /var/log/rabbitmq/startup_{log,err}
Activating RabbitMQ plugins ...
*WARNING* Undefined function fdsrv:bind_socket/2
*WARNING* Undefined function fdsrv:start/0
*WARNING* Undefined function fdsrv:stop/0
*WARNING* Undefined function webmachine_resource:start_link/2
7 plugins activated:
* amqp_client-2.4.1
* mochiweb-1.3
* rabbit_stomp-2.4.1
* rabbit_management-2.4.1
* rabbit_management_agent-2.4.1
* rabbit_mochiweb-2.4.1
* webmachine-1.7.0


+---+   +---+
|   |   |   |
|   |   |   |
|   |   |   |
|   +---+   +-------+
|                   |
| RabbitMQ  +---+   |
|           |   |   |
|   v2.4.1  +---+   |
|                   |
+-------------------+
AMQP 0-9-1 / 0-9 / 0-8
Copyright (C) 2007-2011 VMware, Inc.
Licensed under the MPL.  See http://www.rabbitmq.com/

node           : rabbit at rabbit-a002
app descriptor : /usr/lib/rabbitmq/lib/rabbitmq_server-2.4.1/sbin/../ 
ebin/rabbit.app
home dir       : /var/lib/rabbitmq
config file(s) : (none)
cookie hash    : soP1A0eax4HhjRzyXNgpFA==
log            : /var/log/rabbitmq/rabbit at rabbit-a002.log
sasl log       : /var/log/rabbitmq/rabbit at rabbit-a002-sasl.log
database dir   : /var/lib/rabbitmq/mnesia/rabbit at rabbit-a002
erlang version : 5.7.4

starting file handle cache  
server                                     ...done
starting worker  
pool                                                  ...done
starting  
database                                                     ...done
starting codec correctness  
check                                      ...done
-- external infrastructure ready
starting plugin  
registry                                              ...done
starting auth mechanism cr- 
demo                                       ...done
starting auth mechanism  
amqplain                                      ...done
starting auth mechanism  
plain                                         ...done
starting statistics event  
manager                                     ...done
starting logging  
server                                               ...done
starting exchange type  
direct                                         ...done
starting exchange type  
fanout                                         ...done
starting exchange type  
headers                                        ...done
starting exchange type  
topic                                          ...done
-- kernel ready
starting alarm  
handler                                                ...done
starting node  
monitor                                                 ...done
starting cluster  
delegate                                             ...done
starting guid  
generator                                               ...done
starting memory  
monitor                                               ...done
-- core initialized
starting empty DB  
check                                               ...done
starting exchange  
recovery                                            ...done
starting management  
agent                                             ...done
starting management statistics  
database                               ...done
starting queue supervisor and queue  
recovery                          ...done
-- message delivery logic ready
starting error log  
relay                                              ...done
starting  
networking                                                   ...done
starting  
direct_client                                                ...done
starting notify cluster  
nodes                                         ...done

broker running
starting STOMP Adapter (binding to [61613])  ...done
** Found 0 name clashes in code paths
{"Kernel pid  
terminated 
",application_controller,"{application_terminated,rabbit,shutdown}"}
Erlang has closed

Crash dump was written to: erl_crash.dump
Kernel pid terminated (application_controller)  
({application_terminated,rabbit,shutdown})

ubuntu at rabbit-a002:/usr/lib/rabbitmq/lib/rabbitmq_server-2.4.1/plugins 
$ sudo rabbitmqctl status
Status of node 'rabbit at rabbit-a002' ...
Error: unable to connect to node 'rabbit at rabbit-a002': nodedown
diagnostics:
- nodes and their ports on rabbit-a002: [{rabbitmqctl31379,54031}]
- current node: 'rabbitmqctl31379 at rabbit-a002'
- current node home dir: /var/lib/rabbitmq
- current node cookie hash: soP1A0eax4HhjRzyXNgpFA==

ubuntu at rabbit-a002:/usr/lib/rabbitmq/lib/rabbitmq_server-2.4.1/plugins 
$ sudo mv rabbitmq-management-2.4.1.ez /tmp

ubuntu at rabbit-a002:/usr/lib/rabbitmq/lib/rabbitmq_server-2.4.1/plugins 
$ sudo /etc/init.d/rabbitmq-server restart
Restarting rabbitmq-server: RabbitMQ is not running
SUCCESS
rabbitmq-server.

ubuntu at rabbit-a002:/usr/lib/rabbitmq/lib/rabbitmq_server-2.4.1/plugins 
$ sudo rabbitmqctl status
Status of node 'rabbit at rabbit-a002' ...
[{pid,31513},
  {running_applications,
      [{webmachine,"webmachine","1.7.0"},
       {rabbit_management_agent,"RabbitMQ Management Agent","2.4.1"},
       {amqp_client,"RabbitMQ AMQP Client","2.4.1"},
       {rabbit_mochiweb,"RabbitMQ Mochiweb Embedding","2.4.1"},
       {mochiweb,"MochiMedia Web Server","1.3"},
       {inets,"INETS  CXC 138 49","5.2"},
       {rabbit_stomp,"Embedded Rabbit Stomp Adapter","2.4.1"},
       {rabbit,"RabbitMQ","2.4.1"},
       {mnesia,"MNESIA  CXC 138 12","4.4.12"},
       {os_mon,"CPO  CXC 138 46","2.2.4"},
       {sasl,"SASL  CXC 138 11","2.1.8"},
       {stdlib,"ERTS  CXC 138 10","1.16.4"},
       {kernel,"ERTS  CXC 138 10","2.13.4"}]},
  {nodes,[{disc,['rabbit at rabbit-a002','rabbit at rabbit-a001']}]},
  {running_nodes,['rabbit at rabbit-a001','rabbit at rabbit-a002']}]
...done.



More information about the rabbitmq-discuss mailing list