[rabbitmq-discuss] Large Message Support / Stability

[Alexander Schatten]

Michael,

thanks for your insight. That is indeed an interesting approach. Our app consumers are most likely not as widely distributed as yours, but the encrypted bucket idea is an interesting one indeed. 

However, keeping payload external has further drawbacks, e.g. when can I delete the payload, if at all (that is, how would I know when the last guest has arrived to eat lunch before I remove the buffet *g*)

From a "simplicity" point of view, I would prefer to dump the stuff into the message; however, my gut feeling calls "trouble"!!

thanks a lot for sharing your experiences.


Kind regards,


Alexander Schatten

On 30.07.2013, at 23:17, "Laing, Michael" <michael.laing at nytimes.com> wrote:

> We have taken your alternative approach and write large payloads to S3. They go to encrypted private buckets and we generate signed URLs to allow access by downstream components. We employ content distribution networks for some payloads.
> 
> As they are general purpose, we have retooled our messaging pipelines to pass opaque messages, generally compressed binary. It is expected that any content necessary for filtering and routing will be promoted to AMQP headers where it is easily accessible without requiring access to the message body. This has worked quite well in practice with our internal clients.
> 
> Like you, we gather a diverse set of inputs. However we distribute payloads globally, sometimes to millions of endpoints, and that affects our approach.
> 
> Michael Laing
> NYTimes
> 
--
==========================================
Dr. Alexander Schatten
==========================================
http://www.schatten.info
http://sichten.blogspot.com
Follow me at 
https://alpha.app.net/alex_buzz
http://twitter.com/alex_buzz
==========================================
"Die Selektion hat uns Anschauungsformen gemäß den Aufgaben in noch höchst einfachen Lebensbereichen eingebaut. Und mit Anschauungen von gestern unterwerfen wir uns eine Welt von morgen.", Rupert Riedl