<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Thanks for the feedback guys, sorry for the slow reply back.<div><br></div><div>I took a look on the machine. It's running on an Amazon Linux instance on EC2 (based on CentOS loosely). I don't see anything at all related to oom-killer in logs. Nor anything about rabbit. I looked in dmesg, and everything in /var/log.</div><div><br></div><div>Amazon doesn't do memory monitoring with AWS CloudWatch (major downside, blech), so I can't see if there was any memory contention, but we've never seen any issues on those machines. There's no CPU or disk spikes during that time to indicate anything else traumatic happening on the machine in question.</div><div><br></div><div>One long-shot idea at the application layer: The daemons we run to consume those queues call "declare" on the relevant exchanges and queues on startup. This has always been a convenient way to ensure the queues are alive before we start writing/reading them. In the case where RabbitMQ crashes on us, the daemons start the cycle (they die, their parent process restarts them, they die again, etc.). </div><div><br></div><div>Is there any case in which WHILE rabbit was starting up, it might accept a connection, which would instruct it to create the "expiring-queue", and then when it went to restore the persistent messages from disk, it would give up since the queue already existed? Any sort of race condition that could bite us if there are clients thrashing away during start? Or will RabbitMQ fail to accept any connections until it's cleanly started up?</div><div><br></div><div>-Will</div><div><br></div><div><br><div><div><div>On May 21, 2012, at 5:02 PM, Matthias Radestock wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>On 21/05/12 21:50, Francesco Mazzoli wrote:<br><blockquote type="cite">I have trouble believing that it is actually dying silently with no<br></blockquote><blockquote type="cite">information in the logs.<br></blockquote><br>iirc we've seen this in the past with things like 'oom killer'. It is<br>probably worth checking the system logs.<br><br><blockquote type="cite">In the meantime I'm going to do the obvious and suggest to upgrade to<br></blockquote><blockquote type="cite"> 2.8.2. We fixed several ugly bugs related to DLX (one of which was<br></blockquote><blockquote type="cite">particularly easy to get) and they might be related to your problem.<br></blockquote><br>I don't think any of the DLX bugs pre 2.8.2 would have brought down an<br>entire rabbit, just individual queues.<br><br>Matthias.<br></div></blockquote></div><br><div>
<span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div><div>________________</div><div>Will Koffel</div><div>CTO, Thumb™</div><div>51 E 12th St., 4th Floor</div><div>New York, NY 10003</div><div>Office: (212) 673-8650</div><div>Mobile: (617) 575-WILL</div><div>@thumb</div><div><a href="http://www.thumb.it/">www.thumb.it</a></div></div></div><div><br></div></div></span></div></span><br class="Apple-interchange-newline"></div></span><br class="Apple-interchange-newline"></div></span><br class="Apple-interchange-newline"></span><br class="Apple-interchange-newline">
</div>
<br></div></div></body></html>