<div dir="ltr"><div>Circling back after an earlier question about this problem. We've upgraded to 3.1.3-1. We've moved the mnesia files to a dedicated 10k sata drive via RABBITMQ_MNESIA_BASE, a new drive that nothing else is using, on a xen virtual machine that is only running rabbitmq. We generally deal with about 20 messages per second.</div>
<div><br></div><div>But we're seeing pretty heavy disk activity, with iowait 30-40%, and iostat numbers like this:</div><div><br></div><div><font face="courier new, monospace">Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util</font></div>
<div><font face="courier new, monospace">xvdbp1 0.00 60.00 0.00 143.40 0.00 1452.80 10.13 0.88 6.14 4.22 60.48</font></div><div><br></div><div>Is this expected? That seems excessive, and I worry that the high iowait could cause us problems as we grow, so I was hoping to get some pointers to things to look at that to improve that. Any information, tips, or suggestions would be much appreciated!</div>
<div><br></div><div>There's nothing in the sasl logs, and the rabbitmq logs are nothing but these:</div><div><br></div><div><div><font face="courier new, monospace">INFO REPORT==== 11-Jul-2013::09:25:55 ===</font></div>
<div><font face="courier new, monospace">accepting STOMP connection <0.14174.238> (...)</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">=INFO REPORT==== 11-Jul-2013::09:26:12 ===</font></div>
<div><font face="courier new, monospace">closing STOMP connection <0.13749.238> (...)</font></div></div><div><br></div><div>These are the queues we have in there now:</div><div><br></div><div>$ rabbitmqctl list_queues name durable arguments pid messages_ready </div>
<div>Listing queues ...</div><div>article.added_to_content_bundle<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.239.0><span class="" style="white-space:pre"> </span>0</div>
<div>institution.added_to_content_bundle<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.215.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.article_downloads<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.221.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.article_downloads.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.265.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.article_downloads.unaggregated<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.274.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.article_downloads.unaggregated.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.294.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.bulk_email<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.273.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.bulk_email.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.287.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.certify-existing-expedites<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.307.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.certify-existing-expedites.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.253.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.create-sw-site<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.291.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.create-sw-site.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.282.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.dcnetwork_follow<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.300.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.dcnetwork_follow-error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.289.0><span class="" style="white-space:pre"> </span>229</div>
<div>production.dcnetwork_follow.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.232.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.dcnetwork_update<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.219.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.dcnetwork_update.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.301.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.email<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.223.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.email.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.241.0><span class="" style="white-space:pre"> </span>1</div>
<div>production.expresso-archive-pending-articles<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.305.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.expresso-archive-pending-articles.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.299.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.expresso-copy-article<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.244.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.expresso-copy-article.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.242.0><span class="" style="white-space:pre"> </span>1</div>
<div>production.general-errors<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.250.0><span class="" style="white-space:pre"> </span>150</div>
<div>production.generate-usage-report<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.235.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.generate-usage-report.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.222.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.guest_access_report<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.238.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.guest_access_report.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.281.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.logged_request<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.302.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.logged_request.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.245.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.migrate-article-editor-notes<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.14254.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.migrate-article-editor-notes.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.14255.0><span class="" style="white-space:pre"> </span>96</div>
<div>production.mq-solr-update-article<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.256.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.mq-solr-update-article.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.275.0><span class="" style="white-space:pre"> </span>185</div>
<div>production.mq-solr-update-article.low-priority<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.262.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.mq-solr-update-article.low-priority.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.259.0><span class="" style="white-space:pre"> </span>3</div>
<div>production.publish-context<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.295.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.publish-context.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.268.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.purge-stamped-files<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.14256.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.purge-stamped-files.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.14257.0><span class="" style="white-space:pre"> </span>12</div>
<div>production.record_download<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.257.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.record_download.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.285.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.saved_search_results<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.286.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.saved_search_results.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.230.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.solr-update-article<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.217.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.stamp-pdf<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.225.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.stamp-pdf.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.251.0><span class="" style="white-space:pre"> </span>398</div>
<div>production.stamp-pdf.low-priority<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.276.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.stamp-pdf.low-priority.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.236.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.test<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.233.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.test.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.254.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.withdraw-for-author<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.296.0><span class="" style="white-space:pre"> </span>0</div>
<div>production.withdraw-for-author.error<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.308.0><span class="" style="white-space:pre"> </span>0</div>
<div>test<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.279.0><span class="" style="white-space:pre"> </span>0</div>
<div>user.added_to_content_bundle<span class="" style="white-space:pre"> </span>true<span class="" style="white-space:pre"> </span>[]<span class="" style="white-space:pre"> </span><rabbit@rabbitmq.1.237.0><span class="" style="white-space:pre"> </span>0</div>
<div>...done.</div><div><br></div><div><br></div>-- <br>Kevin M. Goess<br>Software Engineer<br>Berkeley Electronic Press<br><a href="mailto:kgoess@bepress.com" target="_blank">kgoess@bepress.com</a><br>
<br>510-665-1200 x179<br><a href="http://www.bepress.com" target="_blank">www.bepress.com</a><br>
<br>bepress: sustainable scholarly publishing <br>
</div>