[rabbitmq-discuss] For speed, use Shovel or not?

[Jon Brisbin]

Sorry. Brain dead. Totally forgot to include the link that was sitting in my clipboard:

https://github.com/jbrisbin/vpc-utils

Sorry for the noise.

jb

On Apr 28, 2011, at 1:01 PM, Gene Campbell wrote:

> Thank you for the response.    Great one for a FAQ.  The confirmation of batching and compression not being in Shovel is something we until now we couldn't confirm.  (Therefore it's usefulness to us just dropped a bit.)
> 
> The value of being able to ACK back fast, and let a client continue on, while giving some comfort going forward that the message will actually be consumed somewhere after some length of time.  For example, if you can stand 1 minute of latency from publish to consume, but you're client needs an ACK, it will need to store potentially large amounts of data (if your link goes down) until it gets the ACK (link comes back up and message consumed.)  With shovel, it can get a quicker ACK.
> 
> Thanks
> Gene
> 
> On Thu, Apr 28, 2011 at 5:22 AM, Matthew Sackman <matthew at rabbitmq.com> wrote:
> Hi,
> 
> On Sun, Apr 24, 2011 at 02:13:13PM -0700, ristretto.rb wrote:
> > Suppose there are two servers connected across a WAN (read internet),
> > sending transient messages, with all else held constant what is the
> > faster method in terms of messages / sec:  Two brokers and a Shovel,
> > one broker with a remote publisher, or one broker with a remote
> > consumer.  (The last two cases might be really just one case.)
> >
> > msg -> X -> Q -> Shovel -> X -> Q -> Consume
> > vs
> > msg ---------------------------> X -> Q -> Consume
> >
> > What if durability is required?  Does this change the decision?
> 
> Regardless of whether you're using the shovel or not, over the WAN link
> will flow AMQP msgs over TCP/IP. There is no compression or any batching
> used by shovel, so your throughput and latency won't be different in any
> of the combinations you suggest.
> 
> However, you might wish for, eg the publisher to very quickly be able to
> publish a msg and get a confirmation back that the msg has been
> published and accepted by the broker. If the latency of this operation
> is important then it makes sense to have the broker near the publisher.
> 
> The next thing to consider is whether it's actually going to make any
> difference to the consumer whether or not it's got a broker very near
> it. For example, is it going to be doing any synchronous operations
> regularly. If not (i.e. it's just receiving msgs and acking them), then
> there probably isn't much difference whether it's connected to a local
> broker or a remote one.
> 
> Matthew
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> 
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss


Thanks!

Jon Brisbin

http://jbrisbin.com
Twitter: @j_brisbin


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110428/0f673ae4/attachment.htm>