[rabbitmq-discuss] RabbitMQ HA with Redis

Fri Apr 16 13:05:21 BST 2010

Anyone ?

On Thu, Apr 15, 2010 at 11:38 AM, Gustavo Aquino
<aquino.gustavo at gmail.com>wrote:

> Alexis,
>
> On Thu, Apr 15, 2010 at 7:51 AM, Alexis Richardson <
> alexis.richardson at gmail.com> wrote:
>
>> Gustavo,
>>
>> On Wed, Apr 14, 2010 at 9:35 PM, Gustavo Aquino
>> <aquino.gustavo at gmail.com> wrote:
>> > Alexis,
>> > Why not one HA OOTB can be provided ?
>>
>> As I mentioned, we are about to release an HA Guide which provides our
>> recommendation for HA today, that we can stand behind.  In other words
>> - you can trust us to commercially support that solution and provide
>> SLAs.
>>
>>
> I trust you, you know, but we need to know what will be the road map for
> it.
>
>
>> You are correct that we *could* provide a single configuration OOTB
>> for HA.  The question is: which one.
>>
>> Bear in mind that HA behaviour combines:
>>
>> * Immediate acceptance
>> * Immediate delivery
>> * Eventual acceptance
>> * Eventual delivery
>>
>> Note that:
>>
>> (a) these requirements are in tension with one another; so it is
>> expensive to provide a system that is able to provide all of them at
>> once so that, somehow, the system works out what each message needs
>> and reacts accordingly based on need.
>>
>> (b) more importantly, the requirements are based on a notion of TIME.
>> This is problematic because time based guarantees are
>> non-deterministic.  For some systems an unavailability window of 1ms
>> is fine because the client can wait.  For others it is too big.  Many
>> things can affect this.  For example you could be running a messaging
>> system on a real time Linux kernel, but still have variance in your
>> failover window due to how your network and disk hardware is behaving
>> based on (eg) load and contention.
>>
>> This is all ok, but we need to make our assumptions *explicit* for any
>> one HA OOTB solution.
>>
>>
>>
> You are right. But some situations are the same of ppl, I think that a good
> solution is We have this ways to do HA OOTB it's part of product, but if you
> not attend you we can optimize it soing something like X,Y or
> Z approaches... (your commercial support).
>
>
>>
>> > do you have one HP OOTB solution why
>> > not HA ? May be one HA OOTB is not the best solution, and may be not
>> attend
>> > 100% of users but may be 70% and it's sounds good for me not for you ?.
>>
>> Yes that's fine Gustavo, but as soon as you make a promise about one
>> configuration being "HA" then people immediately expect that you can
>> do things that break the laws of physics.
>
>
>> > We know about one AMQP product that have a HA OOTB solution, and we are
>> > trying this "HA OOTB" and this product will be used in CME. But can't do
>> the
>> > same with RabbitMQ.
>>
>> Why don't you describe the main HA parameters that you need, on this
>> list, and we can talk about them.
>>
>>
> Sure, here is.
>
> We have two scenarios for HA:
> -  Cluster
> - "Federation"
>
> Cluster:
>
> A* Cluster* will be a group of brokers that act as a single broker.
> Changes on any broker are replicated to all other brokers in the same
> Messaging Cluster, so if one broker fails, its clients can fail-over to
> another broker without loss of state. The brokers in a Messaging Cluster may
> run on the same host or on different hosts.
>  Clusters need to provide a reliable multicast protocol, tools, and
> infrastructure for implementing replicated services.
> Clusters need to uses an Active/Active model. In this model, all possible
> brokers are active at all times. Producers and consumers can be connected to
> any broker in the cluster. Additionally, any broker can be killed and
> restarted, and the cluster will retain its operational state to provide
> scalability and enhanced load-balancing.
>
>
> "Federation":
>
> Federation will be used to provide geographical distribution of brokers. A
> number of individual brokers, or clusters of brokers, can be federated
> together. This will allows client machines to see and interact with the
> federation as though it were a single broker. Federation will be used where
> client machines remain on a local network, even though their messages have
> to be routed out.
>
>  Federation will be used for connecting disparate locations across a wide
> area network. Full connectivity across an enterprise can be achieved while
> keeping local message traffic isolated to a single location. Departmental
> brokers can be specified with individual policies that control
> inter-departmental message traffic flow.
> Client produces data that must be delivered reliably but connectivity can
> not be guaranteed. In this case, a co-resident broker provides queueing and
> durability that is not available in the client on its own.
>
> RabbitMQ provide a part of "Federation" but the problem is node dow all
> others don't see more the queue and it broke all the process, and RabbitMQ
> "route" don't guarantee delivery if one node is down out of local network,
> today if one node is down all process fail.
>
>
>> Also: Do you want to see a copy of the HA Guide?
>>
>
> I would like so much to get a copy of this guide.
>
>
>
>> > I understand you, and you know my case, we are working in
>> > microseconds/nanoseconds, may be this HA don't attend our requirements,
>> but
>> > it is one solutions and until now we don't have a good solution for HA
>> using
>> > Rabbit, about HP we don't need nothing to talk about RabbitMQ is very
>> good
>> > Broker routing 2.000.000 messages per second just inboud. But one good
>> HA
>> > solution is getting my sleep.
>>
>> I'm sorry to hear that we are keeping you awake at night.
>>
>
> No worry it's not your fault, it's part of IT skill :)
>
>
>>
>>
>> > Do you know what we need,we talked about that is just one HA solution
>> from
>> > RabbitMQ with a very low latency, SLA and guarantee that no messages
>> will be
>> > loosed. O.S cluster or putting HA logic in our application is not a good
>> > solution in our case.
>>
>> Actually, doing dedup on the consumer could be a very good solution
>> based on what you have told me in the past.  But, let's keep
>> discussing it.
>>
>> alexis
>>
>
> Great. let's discussing about.
>
> Regards.
>
> Gustavo
>
>
>
>>
>>
>>
>> > Regards.
>> >
>> >
>> >
>> > On Wed, Apr 14, 2010 at 4:56 PM, Alexis Richardson
>> > <alexis.richardson at gmail.com> wrote:
>> >>
>> >> Gustavo
>> >>
>> >> We are about to release an HA Guide including recommendations.  It's
>> >> just being reviewed.
>> >>
>> >> It is not possible to provide "HA OOTB" because each system is
>> >> slightly different.  If someone tells you that they can support HA
>> >> OOTB, they are misleading you.  What is possible, and desirable, is a
>> >> much simpler and more "OOTB like" experience for the main four or five
>> >> HA cases.  For example, the Beetle project aims to solve for cases
>> >> where idempotency is desired.  Your case might require a different
>> >> optimisation.  E.g. sometimes latency needs to be below X
>> >> microseconds, and sometimes it does not.
>> >>
>> >> We are in the business of providing long term commercial support for
>> >> RabbitMQ based solutions, including where an SLA is in use.  Just let
>> >> us know what solution or solutions you need.
>> >>
>> >> alexis
>> >>
>> >>
>> >>
>> >> On Wed, Apr 14, 2010 at 8:43 PM, Gustavo Aquino
>> >> <aquino.gustavo at gmail.com> wrote:
>> >> > Alexis,
>> >> > I know that it's not the unique, you know how much I'm looking for a
>> >> > good HA
>> >> > solutions using RabbitMQ. The problem is RabbitMQ doesn't have a
>> default
>> >> > solution out of the box for HA, and do you know companies need it,
>> need
>> >> > a
>> >> > commitment from product owner about HA. Do my own solution means,
>> >> > the responsible of maintenance and evolution of this solutions is my,
>> >> > and it
>> >> > is not my business it will be yours. Understand why I do this
>> question
>> >> > about
>> >> > support ?
>> >> > I'm a community enthusiast, and work hard of lot of open projects and
>> is
>> >> > a
>> >> > pleasure if I can help to Rabbit HA, the point is today all products
>> >> > inclusive cloud products need to have one HA solutions OTB, It's is
>> the
>> >> > best
>> >> > ? may be not but need to have it, and today we have a problem of
>> using
>> >> > RabbitMQ because it don't have it. So how can I put Rabbit on my core
>> if
>> >> > cannot guarantee a single SLA, or without putting responsible in side
>> >> > like
>> >> > O.S cluster solution, OpenaIS, LVS shared storage or whatever.
>> >> > Do you know any about the overhead in throughput using this Xing's
>> >> > Solution
>> >> > ?
>> >> >
>> >> > Best Regards.
>> >> >
>> >> > On Wed, Apr 14, 2010 at 4:08 PM, Alexis Richardson
>> >> > <alexis.richardson at gmail.com> wrote:
>> >> >>
>> >> >> Thanks Salvatore.
>> >> >>
>> >> >> Gustavo,
>> >> >>
>> >> >> We and the community will support good solutions and 'things that
>> >> >> work'.  I like the approach, but as Salvatore suggested - it is not
>> >> >> unique.
>> >> >>
>> >> >> alexis
>> >> >>
>> >> >>
>> >> >>
>> >> >> On Wed, Apr 14, 2010 at 7:37 PM, Salvatore Sanfilippo
>> >> >> <antirez at gmail.com>
>> >> >> wrote:
>> >> >> > On Wed, Apr 14, 2010 at 8:34 PM, Gustavo Aquino
>> >> >> > <aquino.gustavo at gmail.com> wrote:
>> >> >> >
>> >> >> >> So if RabbitMQ is a VMWare product and Redis it too, so do Rabbit
>> >> >> >> support
>> >> >> >> this HA approach ? :-)
>> >> >> >
>> >> >> > I think that surely it helps that this feature depends from
>> another
>> >> >> > VMWare product ;)
>> >> >> > But my guess is that the way to go is to have such a support
>> built-in
>> >> >> > for RabbitMQ.
>> >> >> >
>> >> >> > Ciao,
>> >> >> > Salvatore
>> >> >> >
>> >> >> >> Regars.
>> >> >> >>
>> >> >> >> On Wed, Apr 14, 2010 at 2:09 PM, Jason J. W. Williams
>> >> >> >> <jasonjwwilliams at gmail.com> wrote:
>> >> >> >>>
>> >> >> >>> Been reading the page since y'all started tweeting about it. :)
>> >> >> >>> It's
>> >> >> >>> hard to tell how it works.
>> >> >> >>>
>> >> >> >>> -J
>> >> >> >>>
>> >> >> >>> On Wed, Apr 14, 2010 at 11:02 AM, Alexis Richardson
>> >> >> >>> <alexis.richardson at gmail.com> wrote:
>> >> >> >>> > Hi everyone,
>> >> >> >>> >
>> >> >> >>> > It looks like Xing's Beetle project is up on Github:
>> >> >> >>> > http://github.com/xing/beetle
>> >> >> >>> >
>> >> >> >>> > There is a project page here: http://xing.github.com/beetle/
>> >> >> >>> >
>> >> >> >>> > Please take a look and share any comments you may have here
>> :-)
>> >> >> >>> >
>> >> >> >>> > alexis
>> >> >> >>> >
>> >> >> >>> > _______________________________________________
>> >> >> >>> > rabbitmq-discuss mailing list
>> >> >> >>> > rabbitmq-discuss at lists.rabbitmq.com
>> >> >> >>> >
>> >> >> >>> >
>> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>> >> >> >>> >
>> >> >> >>>
>> >> >> >>> _______________________________________________
>> >> >> >>> rabbitmq-discuss mailing list
>> >> >> >>> rabbitmq-discuss at lists.rabbitmq.com
>> >> >> >>>
>> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>> >> >> >>
>> >> >> >>
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Salvatore 'antirez' Sanfilippo
>> >> >> > http://invece.org
>> >> >> >
>> >> >> > "Once you have something that grows faster than education grows,
>> >> >> > you’re always going to get a pop culture.", Alan Kay
>> >> >> >
>> >> >
>> >> >
>> >
>> >
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100416/5cbb6255/attachment.htm 