[rabbitmq-discuss] Dropping out messages from a queue

Mon Jul 6 18:36:36 BST 2009

Gagan

One way to simplify the subscription management and get rid of 'heavy'
churn, is to use "exchange to exchange" bindings.  These are something
we have looked at - I am just checking with the guys to find out of
this solution would work well, and if so then when it will appear.

Another option would be to create a custom exchange which used
presence to 'show' and 'hide' queue bindings.  This would be quite
neat, but it would involve some coding.

alexis

On Sat, Jul 4, 2009 at 8:07 PM, GAGAN ARORA<gaganarora.itm at gmail.com> wrote:
> Alexis,
>
> Thanks for your wonderful support and solutions.
>
> Firstly answers to your queries:
>
> 1. Are you using fanout exchanges for this (if you want one exchange
> per user, then the answer should be 'yes').
>
>>> No I am using topic exchanges because there will be three type of
>>> bindings of a queue with an excajnde. One for publishing messages to all
>>> users, another for one to one messaging and third for publishing message to
>>> a set of users.
>
> 2. Where is the RabbitMQ server running?  Is it on Server 1 or 2, or
> is it on a 3rd server?  It may not be that important but it would be
> good to know.
>
>>> RabbitMQ Server is running on 3rd Server.
>
>
> Back to original problem.
>
> Actually I thought of the solution provided by you before posting the query
> to you. But if I follow this solution I have to perform following steps
> whenever a user comes online:
> 1. Create a queue.
> 2. Create a binding of newly created queue with all exchanges of other users
> for retrieving messages published by other users.
> 3. Creating a binding with exchange of all users for one to one messaging.
> 4. Create a binding with exchanges of a set of users.
>
> When user goes offline following operation is to be performed:
> 1. Delete all the existing bindings of a queue.
> 2. Deleting the queue.
>
> Actually I want my online and offline processes light weighted. According to
> me this solution would have a high impact on performance and will increase
> time to come online and offline.
>
> Kindly let me know if this can be achieved in some other manner.
>
> Thanks
> Gagan Arora
>
>
>
>
>
> On Sat, Jul 4, 2009 at 3:23 PM, Alexis Richardson
> <alexis.richardson at gmail.com> wrote:
>>
>> Gagan,
>>
>> Hi - thank-you for sending the diagram and information which was very
>> useful.  There are several ways to implement the 'twitter style'
>> pattern that you have described.  See below for one way to do it that
>> illustrates the AMQP model.  It may not be the most efficient solution
>> in terms of performance as I shall point out.
>>
>> Two quick questions:
>>
>> 1. Are you using fanout exchanges for this (if you want one exchange
>> per user, then the answer should be 'yes').
>>
>> 2. Where is the RabbitMQ server running?  Is it on Server 1 or 2, or
>> is it on a 3rd server?  It may not be that important but it would be
>> good to know.
>>
>> OK - to your cases -
>>
>> Case1: User B sending his message to UserB Exchange which will route
>> message to User A queue but User A is not connected to any of servers.
>> At this point of  all messages sent to Queue A should be dropped.
>>
>> >>> One way to do this is to identify the lifecycle of Queue A with the
>> >>> presence of User A on servers 1 and 2.  In other words, when User A is not
>> >>> connected to server 1, and also not connected to server 2, then Queue A does
>> >>> not exist.  Note - This is not the only way to achieve what you want but it
>> >>> may be the least fiddly :-)
>>
>> Case2: User A connected to server 1 but UserA consumer is not yet
>> subscribed to UserA queue. If UserB sends any message to its exchange
>> which is routed to Queue of UserA. At this moment of time I want all
>> messages to be stored in queue.
>>
>> >>> One way to do this (continuing from the above) is to create Queue A
>> >>> immediately after UserA connects to server 1.  This is an action that server
>> >>> 1 can initiate on behalf of UserA.  After Queue A has been created, then it
>> >>> should be bound to all exchanges from which it will receive messages, such
>> >>> as UserB's exchange.  This will then have the stated effect you seek "If
>> >>> UserB sends any message to its exchange [it] is routed to Queue of UserA.".
>> >>>  Also, because UserA's consumer has not yet started consuming messages from
>> >>> Queue A, the messages will all remain stored in the queue.
>>
>> >>> In addition, if instead of proceeding to Case 3, we revert to Case 1,
>> >>> for example User A disconnects from Server 1, then Queue A can simply be
>> >>> deleted (along with its contents).  (See also Case 5 below)
>>
>> >>> However - while simple, the above is possibly not the most performant
>> >>> solution if (a) you are running RabbitMQ in clustered mode and (b) your
>> >>> Users generally follow a lot of other Users and (c) they connect and
>> >>> disconnect frequently.  This is because the effect of (c) is to frequuently
>> >>> update the routing table, possibly with a lot of binding/routing data (b),
>> >>> and then replicating it across the cluster (a).
>>
>> >>> Let us know if you want more info about other ways to do this.
>>
>>
>> Case3: User A is connected to both servers and UserA consumer is
>> subscribed to UserA queue. If UserB sends any message to its exchange
>> which is routed to Queue of UserA. At this moment of time I want all
>> messages stored in queue to be delievered plus any other incoming
>> message should also be routed.
>>
>> >>> In any case, to achieve this, just start consuming messages from Queue
>> >>> A.
>>
>> >>> I assume by 'delivered', you mean 'delivered to UserA's consumer'.
>> >>>  Where is that running?
>>
>>
>>
>>
>> Case4: User A is connected to both servers and UserA consumer is
>> subscribed to UserA queue. Now User disconnects from Server 2. If
>> UserB sends any message to its exchange which is routed to Queue of
>> UserA. Again I want to store all messages.
>>
>> >>> To achieve this, just stop consuming messages from Queue A.  They will
>> >>> be stored in Queue A.
>>
>>
>>
>> Case5: If user A is disconnected from both servers. Initial state is
>> achieved again.
>>
>> >>> When User A disconnects from Server 1, then Server 1 can tell RabbitMQ
>> >>> to delete Queue A (along with its contents)
>>
>> Cheers,
>>
>> alexis
>>
>>
>>
>>
>>
>>
>> On Fri, Jul 3, 2009 at 5:05 PM, GAGAN ARORA<gaganarora.itm at gmail.com>
>> wrote:
>> > Hi Alexis
>> > Attaching a doc having architecture diagram and cases desired. Hope this
>> > would make things more clear to you.
>> > Thanks
>> > Gagan Arora
>> >
>> > On Fri, Jul 3, 2009 at 8:23 PM, Alexis Richardson
>> > <alexis.richardson at gmail.com> wrote:
>> >>
>> >> Gagan,
>> >>
>> >> I'm a bit confused about what you are trying to do.  I think what you
>> >> describe is possible but I am not sure what role is being played by
>> >> your two servers.  Is RabbitMQ running on both of them, one of them,
>> >> or neither?  You may find it easier to just use one RabbitMQ server.
>> >>
>> >> Can you send a drawing of what your desired configuration is?
>> >>
>> >> alexis
>> >>
>> >>
>> >>
>> >>
>> >> On Fri, Jul 3, 2009 at 3:08 PM, GAGAN ARORA<gaganarora.itm at gmail.com>
>> >> wrote:
>> >> > Hi
>> >> > I am developing a system in which a user will connect to two servers.
>> >> > On
>> >> > connecting to first server he will become online and will send his
>> >> > presence
>> >> > update but he will receive presence updates only once he is connected
>> >> > to
>> >> > other server as his consumer subscribes to his queue after connecting
>> >> > to
>> >> > second server.
>> >> > I am using one exchange and one queue per user. So if one user comes
>> >> > online
>> >> > then he will send his presence update to his exchange and users bound
>> >> > to
>> >> > that will receive updates.
>> >> > I need to configure a queue which would hold messages for user who
>> >> > has
>> >> > just
>> >> > came online until he is connected to other server. After connecting
>> >> > to
>> >> > second server all messages will be consumed. All messages sent to
>> >> > queue
>> >> > will
>> >> > be dropped once user goes offline.
>> >> > Is it possible?
>> >> > Thanks
>> >> > Gagan Arora
>> >> > _______________________________________________
>> >> > rabbitmq-discuss mailing list
>> >> > rabbitmq-discuss at lists.rabbitmq.com
>> >> > http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>> >> >
>> >> >
>> >
>> >
>
>