[rabbitmq-discuss] Strange failures (Generic server <xxx> terminating) from RabbitMQ
Matt Pietrek
mpietrek at skytap.com
Wed Jul 17 00:26:54 BST 2013
We're seeing a bunch of failure messages from our Ruby clients to RabbitMQ
3.0.2. From the client side, we see 541 "Internal Error"
On the server side logs, we see a bunch of errors. Normally I can at least
hazard a guess what's wrong, but I have no idea with this:
=ERROR REPORT==== 16-Jul-2013::22:43:15 ===
** Generic server <0.4358.0> terminating
** Last message in was {can_send,<0.4355.0>,true}
** When Server state == {lim,1,<0.4359.0>,false,[],1}
** Reason for termination ==
** {function_clause,[{orddict,update,
[<0.4355.0>,#Fun<rabbit_limiter.3.113433752>,[]],
[{file,"orddict.erl"},{line,153}]},
{rabbit_limiter,limit_queue,2,[]},
{rabbit_limiter,handle_call,3,[]},
{gen_server2,handle_msg,2,[]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,227}]}]}
Or like this:
--------------------
=ERROR REPORT==== 16-Jul-2013::22:44:15 ===
** Generic server <0.4563.0> terminating
** Last message in was {'$gen_cast',
{method,
{'basic.consume',0,<<"web-tasks-mpietrek">>,
<<"web-tasks-mpietrek-1374014655000-264079359112">>,
false,false,false,false,[]},
none,noflow}}
** When Server state == {ch,running,rabbit_framing_amqp_0_9_1,41,<0.4540.0>,
<0.4560.0>,<0.4540.0>,
<<"192.168.78.13:57671 -> 192.168.78.50:5672">>,
{lstate,<0.4561.0>,true,false},
none,2,
{[],[]},
{user,<<"guest">>,
[administrator],
rabbit_auth_backend_internal,
{internal_user,<<"guest">>,
<<78,56,17,4,226,151,136,14,156,84,34,25,101,29,104,
78,125,130,202,207>>,
[administrator]}},
<<"/">>,<<"web-tasks-mpietrek">>,
{dict,1,16,16,8,80,48,
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
{{[],[],[],[],[],[],[],
[[<0.4558.0>|
{resource,<<"/">>,queue,
<<"web-tasks-mpietrek">>}]],
[],[],[],[],[],[],[],[]}}},
{dict,3,16,16,8,80,48,
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
{{[],[],[],[],[],[],[],
[[<0.4558.0>|#Ref<0.0.0.83887>]],
[],[],[],[],
[[<2783.2507.0>|#Ref<0.0.0.83888>]],
[[<2782.2563.0>|#Ref<0.0.0.83889>]],
[],[]}}},
{dict,0,16,16,8,80,48,
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},
{set,0,16,16,8,80,48,
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},
{dict,0,16,16,8,80,48,
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},
{set,1,16,16,8,80,48,
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
{{[],[],[],[],[],[],[],
[<0.4558.0>],
[],[],[],[],[],[],[],[]}}},
<0.4539.0>,
{state,fine,5000,#Ref<0.0.0.86733>},
false,1,
{{0,nil},{0,nil}},
[],
[{<<"consumer_cancel_notify">>,bool,true}],
none}
Or like this:
--------------------
=ERROR REPORT==== 16-Jul-2013::22:44:15 ===
** Generic server <0.4561.0> terminating
** Last message in was {can_send,<0.4558.0>,true}
** When Server state == {lim,1,<0.4563.0>,false,[],1}
** Reason for termination ==
** {function_clause,[{orddict,update,
[<0.4558.0>,#Fun<rabbit_limiter.3.113433752>,[]],
[{file,"orddict.erl"},{line,153}]},
{rabbit_limiter,limit_queue,2,[]},
{rabbit_limiter,handle_call,3,[]},
{gen_server2,handle_msg,2,[]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,227}]}]}
=ERROR REPORT==== 16-Jul-2013::22:44:15 ===
** Generic server <0.4558.0> terminating
** Last message in was {basic_consume,false,<0.4563.0>,<0.4561.0>,true,
<<"web-tasks-mpietrek-1374014655000-264079359112">>,
false,none,
{'basic.consume_ok',
<<"web-tasks-mpietrek-1374014655000-264079359112">>}}
** When Server state == {q,
{amqqueue,
{resource,<<"/">>,queue,<<"web-tasks-mpietrek">>},
true,false,none,[],<0.4558.0>,[],[],
[{vhost,<<"/">>},
{name,<<"ha-web-tasks-mpietrek">>},
{pattern,<<"web-tasks-mpietrek">>},
{definition,[{<<"ha-mode">>,<<"all">>}]},
{priority,0}],
[]},
none,true,rabbit_mirror_queue_master,
{state,
{resource,<<"/">>,queue,<<"web-tasks-mpietrek">>},
<0.4566.0>,<0.4565.0>,rabbit_variable_queue,
{vqstate,
{0,{[],[]}},
{0,{[],[]}},
{delta,undefined,0,undefined},
{0,{[],[]}},
{13,
{[{msg_status,12,
<<114,86,130,79,196,29,163,190,216,213,230,38,
174,139,39,60>>,
{basic_message,
{resource,<<"/">>,exchange,<<"skytap">>},
[<<"web-tasks-mpietrek.mpietrek.default">>],
{content,60,
{'P_basic',<<"application/octet-stream">>,
undefined,undefined,2,0,undefined,undefined,
undefined,undefined,undefined,undefined,
undefined,undefined,undefined},
<<152,0,24,97,112,112,108,105,99,97,116,105,
111,110,47,111,99,116,101,116,45,115,116,
114,101,97,109,2,0>>,
rabbit_framing_amqp_0_9_1,
[<<"{\"payload\":{\"object\":\"BAhvOh1EZWxldGVFeHBpcmVkVXBsb2Fkc1Rhc2sGOhJAZ3JhY2VfcGVyaW9k\\nbzocQWN0aXZlU3VwcG9ydDo6RHVyYXRpb24HOgtAdmFsdWVpAwB1EjoLQHBh\\ncnRzWwZbBzoJZGF5c2kT\\n\"},\"_serialization_type\":\"marshal\",\"_timestamp\":\"2013-07-16T22:43:55Z\",\"_message_type\":\"request\",\"_unique_id\":\"app1:req:wfe-em-init.80690eb0d0940130f02f000c291bd1d8.2.372:336\",\"caller_context\":\"wfe-em-init.80690eb0d0940130f02f000c291bd1d8.2.373\",\"action\":\"async-task\",\"_mq_protocol_version\":1}">>]},
<<114,86,130,79,196,29,163,190,216,213,230,38,
174,139,39,60>>,
true},
true,false,true,true,
{message_properties,undefined,false}},
{msg_status,11,
<<12,29,215,111,220,221,165,115,106,150,132,130,
118,94,193,26>>,
{basic_message,
{resource,<<"/">>,exchange,<<"skytap">>},
[<<"web-tasks-mpietrek.mpietrek.default">>],
{content,60,
{'P_basic',<<"application/octet-stream">>,
undefined,undefined,2,0,undefined,undefined,
undefined,undefined,undefined,undefined,
undefined,undefined,undefined},
<<152,0,24,97,112,112,108,105,99,97,116,105,
111,110,47,111,99,116,101,116,45,115,116,
114,101,97,109,2,0>>,
rabbit_framing_amqp_0_9_1,
<rest of error omitted>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130716/7f373738/attachment.htm>
More information about the rabbitmq-discuss
mailing list