[rabbitmq-discuss] Meez status.

[matt]

Ok, I think we *might* be out of the woods.  Earlier in the day Alexis  
& Co. told us that publishing on the channel might not be thread safe  
in our application.  We made a change earlier in the day, but it looks  
like it never really made it to production.  Verified this evening  
about an hour ago and now things are running better w/ the sync block  
around on the publish calls.  We think that the frames were getting  
clobbered and as a result killing the connection and giving us socket  
closed exceptions.  We saw this at one point while running the trace:

  Exception in thread "Thread-4"  
com.rabbitmq.client.UnexpectedFrameError
  at com.rabbitmq.client.impl.AMQCommand 
$Assembler.handleFrame(AMQCommand.java:291)
  at com.rabbitmq.tools.Tracer$DirectionHandler.doFrame(Tracer.java:158)
  at com.rabbitmq.tools.Tracer$DirectionHandler.run(Tracer.java:170)
  at java.lang.Thread.run(Thread.java:595)

This was our biggest traffic day in history - a ton of which is coming  
from AOL.  In the last hour or so we've pushed about 48k requests  
through a single broker and the payload is about 5k each.

That said, things are looking better.  Now we have sluggish publish  
times because everyone's blocked on the publish.  However, I think we  
can just create more channels and spread the load that way.  We also  
have a subset of export requests that should get processed first  
before others, so we'll be leveraging the priority level in the  
protocol.  Real excited about that.

Many thanks to the rabbitMQ team.  You guys have a responsiveness  
that's unheard of!

-matt

--
Matt Darling
Director, Server Engineering
Meez
182 2nd Street, 5th Floor
San Francisco, CA 94105
(m) 415.505.4623
matt at corp.meez.com



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20071108/4041391b/attachment.htm