[rabbitmq-discuss] After weekly log rotatation was done, the command "rabbitmqctl rotate-logs" never stop

[Nate]

Hi Matthias,

    I use pstack to dig the erlang procces, The log seems like the erlang
already crash??!. Are there any other hint can help to track this issue?

process name:
/usr/lib/erlang/erts-5.9/bin/beam.smp -- -root /usr/lib/erlang -progname
erl -- -home /var/lib/rabbitmq -- -pa
/usr/lib/rabbitmq/lib/rabbitmq_server-3.0.1/sbin/../ebin -noshell
-noinput -hidden -sname rabbitmqctl8972 -s rabbit_control_main -nodename
rabbit at rabbitmq1 -extra rotate_logs

Thread 4 (Thread 0x40800940 (LWP 16722)):
#0  0x00002b03c5245fc2 in select () from /lib64/libc.so.6
#1  0x00000000005b3281 in erts_milli_sleep ()
#2  0x000000000045585a in system_cleanup ()
#3  0x0000000000455e0c in erl_exit ()
#4  0x000000000054c946 in signal_dispatcher_thread_func ()
#5  0x00000000005aec19 in thr_wrapper ()
#6  0x00002b03c4d5a73d in start_thread () from /lib64/libpthread.so.0
#7  0x00002b03c524cd1d in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x41001940 (LWP 16723)):
#0  0x00002b03c4d5eee9 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1  0x00000000005ad0d9 in ethr_cond_wait ()
#2  0x0000000000476587 in sys_msg_dispatcher_wait ()
#3  0x00000000004ef7a1 in erts_thr_progress_register_managed_thread ()
#4  0x000000000047a3d1 in sys_msg_dispatcher_func ()
#5  0x00000000005aec19 in thr_wrapper ()
#6  0x00002b03c4d5a73d in start_thread () from /lib64/libpthread.so.0
#7  0x00002b03c524cd1d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x41007940 (LWP 16724)):
#0  0x00002b03c4d5eee9 in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1  0x00000000005ad0d9 in ethr_cond_wait ()
#2  0x000000000054d76f in child_waiter ()
#3  0x00000000005aec19 in thr_wrapper ()
#4  0x00002b03c4d5a73d in start_thread () from /lib64/libpthread.so.0
#5  0x00002b03c524cd1d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x2b03c54d0af0 (LWP 16710)):
#0  0x00002b03c4d614c4 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00002b03c4d5ce1a in _L_lock_1034 () from /lib64/libpthread.so.0
#2  0x00002b03c4d5ccdc in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00000000004bb8be in erts_get_timeval ()
#4  0x00000000004efbe4 in erts_thr_progress_fatal_error_block ()
#5  0x00000000004fa562 in erl_crash_dump_v ()
#6  0x0000000000455eab in erl_exit ()
#7  0x00000000004ba9d5 in get_tolerant_timeofday ()
#8  0x00000000004baffa in erts_get_emu_time ()
#9  0x00000000004a5d41 in erl_create_process ()
#10 0x000000000045709b in erl_start ()
#11 0x0000000000436169 in main ()


2013/5/16 Matthias Radestock-3 [via RabbitMQ] <
ml-node+s1065348n26760h25 at n5.nabble.com>

> Nate,
>
> On 16/05/13 12:36, Nate wrote:
> >         Yes, I ran it as root and nothing return to me.... However, when
> > I use SIGABRT the process will be deleted. My OS is centos 5.5.
>
> There must be something seriously wrong with that Erlang VM process if
> it is not reacting to SIGUSR1.
>
> I suggest you "walk up the call stack" of the ps output you posted
> initially and manually run the log rotation commands, to check whether
> any of them get stuck, e.g.
>
> /bin/sh /usr/sbin/rabbitmqctl rotate_logs
> /bin/sh /etc/init.d/rabbitmq-server rotate-logs
> /bin/sh /sbin/service rabbitmq-server rotate-logs
> sh -c ? /sbin/service rabbitmq-server rotate-logs > /dev/null? ?
> (hmm, what's with those question marks? are they placeholders for
> something?)
> /usr/sbin/logrotate /etc/logrotate.conf
> /bin/sh /etc/cron.daily/logrotate
> /bin/bash /usr/bin/run-parts /etc/cron.daily
>
> Matthias.
> _______________________________________________
> rabbitmq-discuss mailing list
> [hidden email] <http://user/SendEmail.jtp?type=node&node=26760&i=0>
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
> ------------------------------
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://rabbitmq.1065348.n5.nabble.com/Re-After-weekly-log-rotatation-was-done-the-command-rabbitmqctl-rotate-logs-never-stop-tp26607p26760.html
>  To unsubscribe from Re: After weekly log rotatation was done, the command
> "rabbitmqctl rotate-logs" never stop, click here<http://rabbitmq.1065348.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=26607&code=bmF0ZXdhbmcwNDI1QGdtYWlsLmNvbXwyNjYwN3w0NTY2Njg2Njk=>
> .
> NAML<http://rabbitmq.1065348.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>




--
View this message in context: http://rabbitmq.1065348.n5.nabble.com/Re-After-weekly-log-rotatation-was-done-the-command-rabbitmqctl-rotate-logs-never-stop-tp26607p27006.html
Sent from the RabbitMQ mailing list archive at Nabble.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130527/a4e8cfa5/attachment.htm>