[rabbitmq-discuss] Deploying RabitMQ for my Application - Questions and Doubts

Wed Dec 30 11:03:33 GMT 2009

Karthik

Here are some links about XMPP and AMQP:

http://delicious.com/alexisrichardson/rabbitmq+xmpp+amqp

alexis

On Wed, Dec 30, 2009 at 11:02 AM, Alexis Richardson
<alexis.richardson at gmail.com> wrote:
> Karthik
>
> Yes that is feasible.  If you were simply talking about a 'chat'
> system then I would suggest XMPP would be great for your case.  But,
> the mixture of pubsub, fanout and queueing ("pushing .. as and when
> the need arises") suggests that you should consider AMQP.
>
> Re use cases:  I cannot talk about *everyone* doing exactly the same
> as you, since many of our customers don't like to (or cannot) discuss
> their cases in public.   However, here is a reasonably well known
> 'multi-location multi-datacenter multi-cloud' use case that is
> similar:
>
> http://www.oceanobservatories.org/spaces/display/CIDev/Messaging+Service+Specification+V1
> http://io9.com/5410036/fleets-of-autonomous-robots-coming-to-an-ocean-near-you
> http://www.calit2.net/newsroom/article.php?id=1505
>
> Here is a case that has several data centers, similar to what you are
> asking about: http://qconsf.com/sf2009/presentation/Open+Source+at+Unibet.com+-+10x+Scalability+at+Half+the+Cost
>
> Here are some more 'cloud' use cases:
> http://delicious.com/alexisrichardson/rabbitmq+cloud+usecase
>
>  More below...
>
>
> On Wed, Dec 30, 2009 at 10:39 AM, Karthik <coolkarthik88 at gmail.com> wrote:
>> Hi Alex,
>> Thanks for writing in. Here is the scenario.. I need to be running a single
>> cluster of Servers housed in a single Data Center. The problem I am trying
>> to solve involves pushing out information to clients spread across the globe
>> asynchronously aka push the data to them as and when the need arises.
>
> OK.  I assume that 'as and when the need arises' means that
> connections will be somewhat unpredictable, and some data will need to
> be queued for offline consumers?  Will it also need to be stored?
>
>
>> The
>> data to be pushed in will be pumped by a Web Based front end..
>
> Would you want to use one of the RabbitMQ web clients for this?
>
>
>
>> The communication will primarily be Private communication, that is server
>> directly talks to the client.. In my problem none of the clients can talk to
>> each other.. Information can only be exchanged between the client and server
>> in both the directions. PubSub or fanout communications will be involved
>> too, but most communication will be private (connections need to be
>> encrypted).
>
> OK.
>
>
>
>> To scale my system, I was thinking that splitting the system on a country
>> wise basis, will allow me to scale easily.
>
> Would each country server be isolated?  Or could two clients in
> different countries exchange messages via the servers, like this:
>
> Client -> Server -> Server -> Client
>
>
>> I am planning to use Amazon EC2,
>> so during night times or off peak hours, I'd want the power of different
>> instances responsible for different countries to be reduced, to save
>> expenses.
>
> That makes sense.  Of course currently EC2 is in three location (2 in
> US, 1 in EU).
>
>
>
>
>> Can you point me to real implementations of this kind that have used
>> RabbitMQ in the past?
>
> See above.
>
>
>> Also I'm expecting anywhere between  10-5 M0 Million
>> clients, so do you think its feasible?
>
> Yes.
>
>
>> Also what would be latencies involved
>> on an approximate measure...
>
> The main contributor to your latency will be the time taken for data
> to travel across the wide area between servers and clients or vice
> versa.
>
> alexis
>
>
>
>
>> Regards,
>> Karthik Kastury
>>
>> http://dailyapps.net/ | +91 9739864254
>>
>>
>> On Wed, Dec 30, 2009 at 3:52 PM, Alexis Richardson
>> <alexis.richardson at gmail.com> wrote:
>>>
>>> Karthik
>>>
>>> Some questions:
>>>
>>> What is the nature of this communication?  Is it pubsub?  Must queues
>>> be involved?  What kind of recovery and failover do you need?
>>>
>>> Are you saying that the server(s) must be in one physical location?
>>>
>>> Or, are you saying there must be one server "bot" per country?  If so
>>> then can there be communication 'between server bots'?
>>>
>>> alexis
>>>
>>>
>>> On Wed, Dec 30, 2009 at 2:26 AM, Karthik <coolkarthik88 at gmail.com> wrote:
>>> > Hi,
>>> >
>>> > I am looking for a solution for the problem that am currently looking
>>> > to solve... I am trying to build a scalable AMQP Messaging system,
>>> > where multiple bots(in the range of 1 for each country in the world)
>>> > push out messages to clients spread all over the world. The
>>> > communication will be truly one to one, that is one server bot will
>>> > communicate with bots, and each client is assigned one server bot.
>>> > Messages are mostly private, i.e. only from the server to client and
>>> > vice versa.
>>> >
>>> > There can be anywhere between 10-50 Million Concurrent Clients,
>>> > Messages are in the size of 2-3KB(Well formed XML), that can be
>>> > transmitted in the both the directions i.e. Client->Server(bots) and
>>> > vice versa.... My estimates put at the messages to be going outwards
>>> > in either directions somewhere in the range of 50-100 messages/second.
>>> >
>>> > The Clients will need to send/receive all the information as encrypted
>>> > data using SSL+TLS and any other scheme..
>>> >
>>> > Will RabbitMQ be a good choice for the kind of the problem I'm trying
>>> > to solve..
>>> >
>>> > Any help will be appreciated.. Also how much better is using AMQP
>>> > based RabbitMQ in comparison to XMPP based Ejabberd/Openfire etc?
>>> > Regards,
>>> > Karthik Kastury
>>> >
>>> > http://dailyapps.net/ | +91 9739864254
>>> >
>>> > _______________________________________________
>>> > rabbitmq-discuss mailing list
>>> > rabbitmq-discuss at lists.rabbitmq.com
>>> > http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>> >
>>> >
>>
>>
>