[rabbitmq-discuss] messages stuck using shovel
Andy Gram
agram at blackridge.us
Fri Nov 1 02:42:09 GMT 2013
And I should have provided this from devmid:
[{ident_refresh,
{running,
{source,
{amqp_params_network,<<"guest">>,<<"guest">>,<<"/">>,
"192.168.168.212",undefined,0,0,0,infinity,none,
[#Fun<amqp_uri.7.32597394>,#Fun<amqp_uri.7.32597394>],
[],[]}},
{destination,
{amqp_params_direct,<<"guest">>,none,<<"/">>,rabbit at devmid,none,
[]}}},
{{2013,10,31},{17,32,2}}},
{ident_pub,
{running,
{source,
{amqp_params_direct,<<"guest">>,none,<<"/">>,rabbit at devmid,none,
[]}},
{destination,
{amqp_params_network,<<"guest">>,<<"guest">>,<<"/">>,
"192.168.168.212",undefined,0,0,0,infinity,none,
[#Fun<amqp_uri.7.32597394>,#Fun<amqp_uri.7.32597394>],
[],[]}}},
{{2013,10,31},{17,32,2}}},
{ident_sub,
{running,
{source,
{amqp_params_network,<<"guest">>,<<"guest">>,<<"/">>,
"192.168.168.212",undefined,0,0,0,infinity,none,
[#Fun<amqp_uri.7.32597394>,#Fun<amqp_uri.7.32597394>],
[],[]}},
{destination,
{amqp_params_direct,<<"guest">>,none,<<"/">>,rabbit at devmid,none,
[]}}},
{{2013,10,31},{17,32,2}}},
{refresh_req,
{running,
{source,
{amqp_params_direct,<<"guest">>,none,<<"/">>,rabbit at devmid,none,
[]}},
{destination,
{amqp_params_network,<<"guest">>,<<"guest">>,<<"/">>,
"192.168.168.212",undefined,0,0,0,infinity,none,
[#Fun<amqp_uri.7.32597394>,#Fun<amqp_uri.7.32597394>],
[],[]}}},
{{2013,10,31},{17,32,2}}}]
...done.
IP address of devtop is 192.168.168.212. Here's additional info from regarding connections from devtop;
+-------+-----------------------------------------------+---------+----------------+----------+-----------+-----------------+------------------+-----------------+---------------+------------------+-------------------+--------------------+-----------------+-----------+------+------------+----------+----------+----------+----------+-----------+-------+------------+----------+------------------+--------------+---------+---------+-------+
| vhost | name | type | auth_mechanism | channels | frame_max | host | last_blocked_age | last_blocked_by | node | peer_cert_issuer | peer_cert_subject | peer_cert_validity | peer_host | peer_port | port | protocol | recv_cnt | recv_oct | send_cnt | send_oct | send_pend | ssl | ssl_cipher | ssl_hash | ssl_key_exchange | ssl_protocol | state | timeout | user |
+-------+-----------------------------------------------+---------+----------------+----------+-----------+-----------------+------------------+-----------------+---------------+------------------+-------------------+--------------------+-----------------+-----------+------+------------+----------+----------+----------+----------+-----------+-------+------------+----------+------------------+--------------+---------+---------+-------+
| / | 127.0.0.1:53029 -> 127.0.0.1:5672 | network | PLAIN | 2 | 32768 | 127.0.0.1 | infinity | none | rabbit at devtop | | | | 127.0.0.1 | 53029 | 5672 | AMQP 0-9-1 | 81 | 1888 | 69 | 2001 | 0 | False | | | | | running | 600 | guest |
| / | 127.0.0.1:53030 -> 127.0.0.1:5672 | network | PLAIN | 2 | 32768 | 127.0.0.1 | infinity | none | rabbit at devtop | | | | 127.0.0.1 | 53030 | 5672 | AMQP 0-9-1 | 67 | 926 | 66 | 964 | 0 | False | | | | | running | 600 | guest |
| / | 192.168.168.142:38100 -> 192.168.168.212:5672 | network | PLAIN | 1 | 131072 | 192.168.168.212 | infinity | none | rabbit at devtop | | | | 192.168.168.142 | 38100 | 5672 | AMQP 0-9-1 | 63 | 975 | 63 | 934 | 0 | False | | | | | running | 600 | guest |
| / | 192.168.168.142:41887 -> 192.168.168.212:5672 | network | PLAIN | 1 | 131072 | 192.168.168.212 | infinity | none | rabbit at devtop | | | | 192.168.168.142 | 41887 | 5672 | AMQP 0-9-1 | 64 | 951 | 64 | 1159 | 0 | False | | | | | running | 600 | guest |
| / | 192.168.168.142:51473 -> 192.168.168.212:5672 | network | PLAIN | 1 | 131072 | 192.168.168.212 | infinity | none | rabbit at devtop | | | | 192.168.168.142 | 51473 | 5672 | AMQP 0-9-1 | 64 | 967 | 63 | 924 | 0 | False | | | | | running | 600 | guest |
| / | 192.168.168.142:58133 -> 192.168.168.212:5672 | network | PLAIN | 1 | 131072 | 192.168.168.212 | infinity | none | rabbit at devtop | | | | 192.168.168.142 | 58133 | 5672 | AMQP 0-9-1 | 60 | 870 | 60 | 867 | 0 | False | | | | | running | 600 | guest |
| / | 192.168.168.146:36800 -> 192.168.168.212:5672 | network | PLAIN | 1 | 131072 | 192.168.168.212 | infinity | none | rabbit at devtop | | | | 192.168.168.146 | 36800 | 5672 | AMQP 0-9-1 | 64 | 967 | 63 | 924 | 0 | False | | | | | running | 600 | guest |
| / | 192.168.168.146:37081 -> 192.168.168.212:5672 | network | PLAIN | 1 | 131072 | 192.168.168.212 | infinity | none | rabbit at devtop | | | | 192.168.168.146 | 37081 | 5672 | AMQP 0-9-1 | 63 | 975 | 63 | 934 | 0 | False | | | | | running | 600 | guest |
| / | 192.168.168.146:40113 -> 192.168.168.212:5672 | network | PLAIN | 1 | 131072 | 192.168.168.212 | infinity | none | rabbit at devtop | | | | 192.168.168.146 | 40113 | 5672 | AMQP 0-9-1 | 63 | 951 | 64 | 1159 | 0 | False | | | | | running | 600 | guest |
| / | 192.168.168.146:49883 -> 192.168.168.212:5672 | network | PLAIN | 1 | 131072 | 192.168.168.212 | infinity | none | rabbit at devtop | | | | 192.168.168.146 | 49883 | 5672 | AMQP 0-9-1 | 61 | 1067 | 61 | 888 | 0 | False | | | | | running | 600 | guest |
+-------+-----------------------------------------------+---------+----------------+----------+-----------+-----------------+------------------+-----------------+---------------+------------------+-------------------+--------------------+-----------------+-----------+------+------------+----------+----------+----------+----------+-----------+-------+------------+----------+------------------+--------------+---------+---------+-------+
From: rabbitmq-discuss-bounces at lists.rabbitmq.com [mailto:rabbitmq-discuss-bounces at lists.rabbitmq.com] On Behalf Of Andy Gram
Sent: Thursday, October 31, 2013 11:51 AM
To: Discussions about RabbitMQ
Subject: Re: [rabbitmq-discuss] messages stuck using shovel
I probably should have mentioned that I am using RabbitMQ R3.1.4 with Erlang R14B04 running on linux.
From: rabbitmq-discuss-bounces at lists.rabbitmq.com<mailto:rabbitmq-discuss-bounces at lists.rabbitmq.com> [mailto:rabbitmq-discuss-bounces at lists.rabbitmq.com] On Behalf Of Andy Gram
Sent: Thursday, October 31, 2013 11:10 AM
To: rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>
Subject: [rabbitmq-discuss] messages stuck using shovel
I have a configuration of 3 RabbitMQ nodes; devtop, devmid, devbot. I am implementing a star configuration with devtop as the center of the start. This is a pubsub application over a WAN. Any node can submit a new item to devtop and my publisher application will publish to all nodes, including itself. I am using shovel to instantiate a queue for each node that is bound to a fanout exchange on devtop. The published messages are stuck on devtop as below;
+-------+--------------------------------+-------------+-----------+---------+------------------------+---------------------+--------+----------+----------------+-------------------------+---------------+--------+---------+
| vhost | name | auto_delete | consumers | durable | exclusive_consumer_tag | idle_since | memory | messages | messages_ready | messages_unacknowledged | node | policy | status |
+-------+--------------------------------+-------------+-----------+---------+------------------------+---------------------+--------+----------+----------------+-------------------------+---------------+--------+---------+
| / | amq.gen-QXyScDVGCoTv8bhcbrMOrA | False | 1 | False | | 2013-10-31 10:32:25 | 9456 | 1 | 0 | 1 | rabbit at devtop | | running |
| / | amq.gen-Ya03CVYcY-CEVKOYcSt0Xw | False | 1 | False | | 2013-10-31 10:32:25 | 9456 | 1 | 0 | 1 | rabbit at devtop | | running |
| / | new_ident_queue | False | 1 | True | | 2013-10-31 10:32:28 | 14312 | 0 | 0 | 0 | rabbit at devtop | | running |
| / | pub_idents | False | 0 | True | | 2013-10-31 10:15:52 | 13760 | 0 | 0 | 0 | rabbit at devtop | | running |
| / | refresh_idents | False | 2 | True | | 2013-10-31 10:32:18 | 14192 | 0 | 0 | 0 | rabbit at devtop | | running |
| / | refresh_queue | False | 1 | True | | 2013-10-31 10:32:18 | 13976 | 0 | 0 | 0 | rabbit at devtop | | running |
+-------+--------------------------------+-------------+-----------+---------+------------------------+---------------------+--------+----------+----------------+-------------------------+---------------+--------+---------+
The status on devmid is as follows;
+-------+-----------------+-------------+-----------+---------+------------------------+---------------------+--------+----------+----------------+-------------------------+---------------+--------+---------+
| vhost | name | auto_delete | consumers | durable | exclusive_consumer_tag | idle_since | memory | messages | messages_ready | messages_unacknowledged | node | policy | status |
+-------+-----------------+-------------+-----------+---------+------------------------+---------------------+--------+----------+----------------+-------------------------+---------------+--------+---------+
| / | ident_updates | False | 0 | True | | 2013-10-31 17:32:03 | 21696 | 0 | 0 | 0 | rabbit at devmid | | running |
| / | new_ident_queue | False | 1 | True | | 2013-10-31 17:32:03 | 21912 | 0 | 0 | 0 | rabbit at devmid | | running |
| / | refresh_idents | False | 0 | True | | 2013-10-31 17:32:03 | 21696 | 0 | 0 | 0 | rabbit at devmid | | running |
| / | refresh_queue | False | 1 | True | | 2013-10-31 17:32:03 | 21912 | 0 | 0 | 0 | rabbit at devmid | | running |
+-------+-----------------+-------------+-----------+---------+------------------------+---------------------+--------+----------+----------------+-------------------------+---------------+--------+---------+
I am expecting shovel from devmid to move the messages from devtop queue to ident_updates queue on devmid. My shovel configuration follows:
{rabbitmq_shovel,
[ {shovels, [ {ident_pub,
[ {sources, [ {broker, "amqp://"}
, {declarations, [
{'queue.declare',
[{queue, <<"new_ident_queue">> }
, durable
]}
]}
]}
, {destinations, [ {broker, "amqp://192.168.168.212"}
, {declarations, [
{'queue.declare',
[{queue, <<"new_ident_queue">> }
, durable
]}
]}
]}
, {queue, <<"new_ident_queue">>}
, {prefetch_count, 1}
, {ack_mode, on_confirm}
, {publish_properties, [ {delivery_mode, 2} ]}
, {reconnect_delay, 5}
]},
{ident_sub,
[ {destinations, [ {broker, "amqp://"}
, {declarations, [
{'queue.declare',
[{queue, <<"ident_updates">> }
, durable
]}
]}
]}
, {sources, [ {broker, "amqp://192.168.168.212"}
, {declarations, [ {'exchange.declare',
[ {exchange, <<"pub_idents">>}
, {type, <<"fanout">>}
]}
, {'queue.declare',
[{queue, <<>> }
, exclusive
]}
, {'queue.bind',
[ {exchange, <<"pub_idents">>}
, {queue, <<>>}
]}
]}
]}
, {queue, <<>>}
, {prefetch_count, 1}
, {ack_mode, on_confirm}
, {publish_properties, [ {delivery_mode, 2} ]}
, {reconnect_delay, 5}
]},
...
My shovel is stuck!! Any help would be appreciated.
Thanks,
Andy
This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
Andy Gram
BlackRidge Technology Inc.
www.blackridge.us<http://www.blackridge.us/>
408-892-4044
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20131101/369939ce/attachment.htm>
More information about the rabbitmq-discuss
mailing list