[rabbitmq-discuss] building chat

Sun Aug 9 22:25:48 BST 2009

Ben,

On Sat, Aug 8, 2009 at 2:28 PM, Ben Browitt<ben.browitt at gmail.com> wrote:
>>
>> Do you want to do MUC or IM?
>
> I'm not actually building it.
> I keep hearing about AMQP and RabbitMQ so I read the spec and the server
> docs.
> I couldn't find use cases on the web so I thought I'll try to fit the
> protocol to a fun application.

Cool - understood.

>> In fact, you can use a single approach to both.
>>
>> One approach is to have one queue per consumer 'chat window' and one
>> exchange per 'chat room'.  The queue is bound to the exchange when the
>> user is in the chatroom.  Then, IM is a special case of MUC - it is a
>> chatroom with two participants.
>>
>> Another approach would be to have one queue per consumer, and have
>> that queue bound to the exchange(s) N times, where N is the number of
>> chatrooms (or IMs) which the consumer is currently in.  In this
>> approach, you will need to identify which message came from which
>> room, before you show the messages to the user.  That's easy but it's
>> extra work and data to carry around.
>
> When a client sends a message to another client he usually doesn't create a
> chatroom for two,
> he just sends the message and the server routes it. With your suggestion the
> server will create
> the room if it doesn't exists when it needs to route a message.

What I am suggesting is that the solution for IM can use queues.  In
AMQP you can also think of queues as buffers with configurable
properties including routing rules based on AMQP's "bindings" and QoS
eg durability.  So, for an IM example you could have each user
consuming a message stream of 'what the other user said' through that
user's queue.  So, yes, when the queue doesn't exist, and is needed,
it should be created.  It can be deleted later, or kept around -
depending on what you want to achieve.

> When all the users leaves a MUC room if it is not persistent the server
> destroys it.

You could set up an AMQP MUC system to do that if you liked.  Or, you
could set it up to make the queues durable and therefore not destroyed
in this case.

> What will be the case with a chatroom with two participants?

It's up to you.

I would suggest deleting the queues used for the IM once the IM has completed.

Separately, you might want to add the feature "being able to send
messages when the other user is offline.  For this, you would want the
queue lifecycle to be independent of the user's connections.

> Maybe the server will destroy it when both users are offline?

It comes down to how you set up your system.  AMQP is very flexible,
eg allowing both the cases (a) destroy queues, (b) keep queues.

> Is there a way to destroy an exchange if it is not active for several
> minutes to save memory?

You can destroy exchanges whenever you like.  How you monitor activity
is up to you.

There isn't a 'built in' functions for 'delete an unused exchange
after time T'.  Some of the ideas Paul was talking about could be
helpful here - using presence to manage internal broker lifecycle.

Direct and fanout exchanges are 'lightweight' lookup tables so you
might want to manage memory by monitoring overall memory use and user
activity.

> Can you use a standard XMPP client with this setup?

You can use an XMPP client if you run RabbitMQ with the XMPP adaptor.

>> Let me know if you want more detail on either of the above, eg which
>> exchange type to use.
>
> I thought of a non durable exchange. One exchange per chatroom and one queue
> per user.

That seems like a good approach.  In this case you would use a
'fanout' exchange.

Another approach would be to use just one 'direct' exchange and bind
all user queues to it using the routing keys corresponding to the
chatrooms the users are in.

If you haven't yet seen it, the 'rabbits and warrens' intro article
linked to here is good: http://www.rabbitmq.com/how.html

>> > I read the xmpp gateway
>> > docs and have some questions.
>> >
>> > In a chat we want to be able to get presence from users on our
>> > friend-list,
>> > send them messages
>> > and send subscription request and messages to users not yet on our
>> > friend-list.
>>
>> Do you only want to enable chat when both users are online and have a
>> 'friend' relationship?
>>
>>
>> > One possibility is that each user will have an exchange and a queue with
>> > binding to his own exchange.
>> > When user1 wants to send a message to user2 he send it directly to the
>> > exchange of user2.
>> > There is a problem with presence updates. If user1 changes his presence,
>> > he'll have to sends the update
>> > to the exchange of all the users on his friend-list. With this design we
>> > move the logic to the client rather then use the AMQP features.
>>
>> One approach to presence is to broadcast changes using the pubsub
>> system.  Changes could include:
>>
>> * User goes online / offline
>> * User changes their 'status message'
>
>
> All I know is that there are queues exchanges and bindings.
> What do you mean by the pubsub system?

By 'the pubsub system' I mean the RabbitMQ broker, and how you have
configured the queues by binding them to the exchange(s).

For example you can do 'pubsub' by binding queues to a direct
exchange.  Take a look at slides 20-22 from the presentation linked to
from this page:
http://skillsmatter.com/podcast/erlang/alexis-richardson-introduction-to-rabbitmq
 This shows how a queue can use one or more bindings to 'subscribe' to
what producers ('publishers') are saying.

So there is nothing stopping you using the same mechanism to publish
presence information.  Suppose that whenever a user's presence
changes, a message is sent to a direct exchange on the broker with key
'username' and content 'presence_update_info'.  Then, anyone who cares
about that user's presence just has to bind a queue with the key
'username'.  I suggest you try and implement this pattern - it should
be very straightforward.

>> > Another option is that when a user logs in he'll create a binding to the
>> > exchanges of all the users on his friend-list.
>>
>> You can do this but it creates quite a lot of churn on the exchange
>> layer.  Paul's suggestion is one way to prevent that.  Another is to
>> use exchange-exchange bindings, which we are also looking at.
>
> I meant to create bindings to all your online friends.

Yes I understand that.

> Can you explain how Paul's suggestion or exchange-exchange bindings solve or
> improve it?

Mostly they are performance optimisations that I would ignore for now,
not least because they are under investigation and not implemented yet
:-)

When you create a binding, this action adds information to the routing
tables (ie the exchange).  This carries a (very small) computational
cost that you may not want to pay (if under heavy load, say).
Additionally if you are running in a multi-machine cluster, RabbitMQ
will copy the bindings to other machines, so that the routing table
info is shared across the cluster.  This additional copying carries a
(higher) computational cost.  To avoid paying this cost, you might
want to optimise how much binding and unbinding takes place.  Using
exchange-exchange bindings would mean, for example, that you could
store N bindings persistently on one exchange, corresponding to a
user's N friends, and have one binding between exchanges which gets
destroyed when that user is offline, and recreated whenever they come
online.  This would save you from having to make N bindings every time
a user came online.

>> > When user1 want to send a private message or a presence update to user2
>> > he'll send it to his own exchange and it'll be directed to user2. The
>> > problem with this design is that users without mutual presence
>> > subscription
>> > can't send private message to each other and can't send subscription
>> > requests.
>>
>> You can decouple presence, which is about status updates, from
>> friendship.  If you like, you could have a set up where one user can
>> DM another, provided they are mutual friends, regardless of who is
>> online.  Do you want that?
>>
>> Subscription requests could also be implemented using a queue - one
>> per user.  Whenever a new user wishes to request mutual friendship
>> with user A, then user A could be notified of this via a queue.
>
> In that case will the server send the friendship request directly to the
> queue?

Yes.

> Do you need another queue per user just for friendship requests or can you
> use the same queue from above?

It's up to you, but I would have, for each user, a special queue for
messages from people they are not following yet.

Is this helping?

alexis

>> > Can you please share your thoughts on the two options and maybe other
>> > design
>> > options?
>>
>> I've tried to shed some light on some of the issues above.  Feel free
>> to elaborate on your needs and ask more questions.
>>
>> alexis
>>
>>
>>
>>
>>
>> > Thanks
>> > _______________________________________________
>> > rabbitmq-discuss mailing list
>> > rabbitmq-discuss at lists.rabbitmq.com
>> > http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>> >
>> >
>
>