[rabbitmq-discuss] Can RabbitMQ handle big messages?
irmo.manie at gmail.com
Mon Mar 12 12:51:17 GMT 2012
RabbitMQ should actually not be used for big file transfers or only
with great care and fragmenting the files into smaller separate
When running a single broker instance, you'd still be safe, but in a
clustered setup, very big messages will break the cluster.
Clustered nodes are connected via 1 tcp connection, which must also
transport a (erlang) heartbeat. If your big message takes more time to
transfer between nodes than the heartbeat timeout (anywhere between
~20-45 seconds if I'm correct), the cluster will break and your
message is lost.
The preferred architecture for file transfer over amqp is to just send
a message with a link to a downloadable resource and let the file
transfer be handle by specialized protocol like ftp :-)
On Mon, Mar 12, 2012 at 1:26 PM, Emile Joubert <emile at rabbitmq.com> wrote:
> Hi Zabrane,
> On 10/03/12 18:34, Jerry Kuch wrote:
>> Hi, Zabrane: In theory the AMQP protocol IIRC allows crazy large
>> message payloads (2^64 bytes I believe). In practice though, that's
>> madness since you end up with potential copying and buffering along the
>> way that could make a broker very unhealthy.
> I have successfully processed messages as large as 2Gb using RabbitMQ,
> where 2Gb was about 5% of the total RAM. If the ratio between message
> size and total RAM stays low then you can send even larger messages, up
> to the limit Jerry mentioned.
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
More information about the rabbitmq-discuss