Matthew,<div><br><br><div class="gmail_quote">On Fri, Mar 12, 2010 at 11:58 AM, Matthew Sackman <span dir="ltr"><<a href="mailto:matthew@lshift.net">matthew@lshift.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="im">On Fri, Mar 12, 2010 at 11:36:31AM -0300, Gustavo Aquino wrote:<br>
> This is my biggest concern.<br>
<br>
</div>It's the nature of AMQP.<br>
<div class="im"><br>
> Look about this scenario I have one Exchange and one persistent queue.I'm<br>
> posting transient messages to this Exchange (to get better performance) if<br>
> my Exchanges messages are in buffer and server crash I will be loose all<br>
> messages in buffer and it is a big problem.... I cannot lose any messages,<br>
<br>
</div>Not true. There is no messaging product available (nor will there ever<br>
be) that can guarantee it can't lose messages. Do you have infinite disk<br>
space? Is there no event that can cause power loss? Even in the event of<br>
the sun exploding you want to ensure no message loss?<br></blockquote><div><br></div><div>If sun exploding money will be the minor problem of us. ;-)</div><div><br></div><div>We don't have infinite disk space, but the primary proposal is monitoring to be able to be pro active and do something before problem happen.</div>
<div><br></div><div>Look,for example I'm monitoring queue size, and look that we are�coming�to our limit, one way to guarantee the messages inside server is stopping producers and redirect producers to other server, so in theory I can guarantee that messages inside server1 will be consumed and server not loose messages or crash.</div>
<div><br></div><div><br></div><div>Or how to guarantee that one message posted will be consumed ? consume don't know about the message posted.</div><div>�</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
The fact is, message loss is unavoidable. You can take many steps to<br>
avoid this, by using transactions and buffering messages in the client<br>
so that if the connection to the broker dies you can reconnect and<br>
republish, but at that point you risk duplicates, so you need<br>
application logic on the consumer side to watch for duplicates and<br>
remove them as necessary.<br>
<div class="im"><br>
> this messages are business it represent money, so if I loose one we loose<br>
> money and It cannot happen, in other way I need to be the fastest we<br>
> need overrides in nanoseconds or 1 millisecond is the limit, so post<br>
> messages in persistent queue is out of the scope, or �have one way to be<br>
> fast with persistent ?<br>
<br>
</div>Right, so you can't afford the delay of writing to disk, and yet you<br>
want to ensure messages can't be lost in the event of a crash, or<br>
comet strike. You may wish then to explore multiple brokers, publish, in<br>
transactions to duplicate resources (queues) on every broker, and then<br>
simultaneously consume from all of them, deduplicating as you go. Thus<br>
you don't need to write to disk, and by shear strength of numbers, you<br>
should be able to survive crashes. However, really these various brokers<br>
should be in different data centres, probably on different continents,<br>
so the 1ms max delay may be challenging...<br></blockquote><div><br></div><div>How to duplicate resources inside multiples brokers using Rabbitmq ? I saw that Rabbit don't have a default way to do a Cluster HA, your based proposal is HP. �</div>
<div>�</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<font color="#888888"><br>
Matthew<br>
</font></blockquote></div><br></div>