[rabbitmq-discuss] Weird Crash (91MB message over STOMP) [Reproducible]

Sun Aug 9 23:56:29 BST 2009

Darien,

Darien Kindlund wrote:
> This particular bug makes me wonder about a much larger issue: In
> general, how *should* RabbitMQ behave when it's fed an astronomically
> large message size?
> 
> Let's say that RabbitMQ is smart, in that if the message is to big to
> fit into RAM, it will attempt to page the rest of the message onto
> disk ... but what happens when the disk is completely full and there's
> simply no space left?
> 
> Ideally, I'm thinking (in this scenario) RabbitMQ should gracefully
> return an error message to the producer, discard the message, and
> somehow terminate the producer's send operation.  I'm not sure if this
> is exactly how it's handled already -- or if this behavior is
> consistent across all the different connector transports (e.g., HTTP,
> STOMP).
> 
> I'm not an AMQP spec expert, but if the AMQP spec requires that all
> producers supply the message size in the header (ahead of time), then
> RabbitMQ could easily perform sanity checks on the maximum allowable
> message size.  A RabbitMQ admin could specify this configuration
> option in the /etc/rabbitmq/rabbitmq.conf file... or RabbitMQ could
> approximate a maximal message size by looking at the total RAM + a
> portion of the unused swap.  That way RabbitMQ could inspect the
> message size and pre-emptively reject the producer's message before
> running into the out-of-RAM/diskspace issue.

In theory that's all possible in AMQP, which requires the content-length 
to be transmitted at the beginning, an even STOMP, as long as producers 
include the optional content-length header there.

One complication is that not all messages do in fact need to be stored - 
there may be no queues to send the message to, or the queues may have 
only auto-ack consumers (in which case the message could in theory be 
"streamed through").

RabbitMQ doesn't implement anything fancy here; if a message doesn't fit 
into available virtual memory then the broker will die.

RabbitMQ can still transmit large messages though. The way to do that is 
to get the producer to chop up the original message into smaller ones, 
and for the consumer to reassemble them. This has the added advantage of 
making it easier to deal with failures by performing partial 
retransmissions - something that isn't possible if messages are treated 
as indivisible entities.

Regards,

Matthias.