<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Hey all, I was hoping I could get some direction here that might help us out. We’ve got a RabbitMQ production cluster up, and for the majority of our environments it works great. However, we have a protected set of environments that from
within it the connections keep dying. If we go to the load balanced VIP, about 25 packets or so get exchanged, then there’s a 200ms lull to acknowledge a packet from Rabbit, a quick resend/reack, and then nothing until the connection dies five minutes later.
If we try to go past the VIP straight to a node, it fails outright (we did update the firewall rules to point to the single node). We’ve pulled the app out of this environment and it works fine outside, but within it seems to always exhibit this behavior.
I got one of our network guys to break down the route the packets have to take:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">“Client Service is behind 2 stateful Cisco ASA firewalls. They connect to a F5 LBed Rabbit MQ server that works fine when not connecting form the behind the FW. Firewall ports open are TCP 5672-5673 and 15672. Issue we having is related
to the connection on TCP 5672. Under working conditions, the client spins up 3 TCP connection to the server on 5672. On the problem client, it spins up an initial 3 and immediately spins up additional connections. Upon spinning up the additional connections
it leaves the previous connections open, which are eventually reset after 5 min.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thus the main issue is what is the client seeing that causing it to restart the connection. The failed connections are very repeatable, it gets about 14 packets per side. With the last packets being a ctag ### from the server, followed
by a delayed ack from the client.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Did try adding some overrides for the protocol on the FWs that inline. Added the below.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">class-map RabbitMQ<o:p></o:p></p>
<p class="MsoNormal">description Disable connection timeouts and other for RabbitMQ<o:p></o:p></p>
<p class="MsoNormal">match port tcp eq 5672<o:p></o:p></p>
<p class="MsoNormal">policy-map global_policy<o:p></o:p></p>
<p class="MsoNormal">class RabbitMQ<o:p></o:p></p>
<p class="MsoNormal"> set connection random-sequence-number disable<o:p></o:p></p>
<p class="MsoNormal"> set connection timeout embryonic 0:00:20 tcp 0:00:00<o:p></o:p></p>
<p class="MsoNormal">“<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Any ideas on what in that sequence could be hosing us up?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
<p class="MsoNormal">Kale<o:p></o:p></p>
</div>
<p>This communication contains information that is confidential,<br>
proprietary in nature, and may also be attorney-client privileged<br>
and/or work product privileged. It is for the exclusive use of the<br>
intended recipient(s). If you are not the intended recipient(s) or<br>
the person responsible for delivering it to the intended<br>
recipient(s), please note that any form of dissemination,<br>
distribution or copying of this communication is strictly<br>
prohibited and may be unlawful. If you have received this<br>
communication in error, please immediately notify the sender by replying<br>
to this message and delete this email immediately. Thank you for your cooperation. </p>
<p>Please be advised that neither Altegrity, its affiliates, its employees<br>
or agents accept liability for any errors, omissions or damages<br>
caused by delays of receipt or by any virus infection in this<br>
message or its attachments, or which may otherwise arise as a<br>
result of this e-mail transmission.</p></body>
</html>