[rabbitmq-discuss] Advice needed on new 'feature'

Wed Apr 22 16:46:13 BST 2009

Hello,

So first off I want to say how delighted I am with RabbitMQ. Once I kinda
got the hang of Erlang, the code is really nice and easy enough to
understand; though I'm still very much new at this.

First a little background (sorry it's so long): I have a use-case where I'm
using RabbitMQ with the STOMP adapter (which is also wonderful BTW) but ran
into some limitations. We (my team working on a project) plan for every user
of our service to have their own 'inbox' (queue) to receive real-time data
over via a topic exchange. What we want to allow though is for same user (or
a default 'guest' account) to log in from multiple places and receive from
the same queue. Now one way to do this I suppose would be to create an
exchange per user to fan out and have new temporary/auto_delete queue's for
each connection; kind of like what the STOMP adapter docs describe for
broadcast. But creating an exchange per user just seemed too...heavy for me
and overkill, I think it's easier to just give each a queue.

Problem was, I couldn't properly map in my head (or in code) how to get all
consumers of that queue to get the message. The broadcast-like behaviour
using another fanout exchange like described didn't seem like it would work
for me.

So I modified the RabbitMQ-server code to add another boolean attribute to
each queue called 'broadcast'. When you declare a queue, you can turn on
'broadcast' (default is false) and that will make all the current/active
consumers of the queue receive the message at once instead of round-robining
to a single consumer. I thought about maybe setting this flag per message
instead but that thought only came as I was almost finished with the
per-queue one and that seems to work great for me.

I know this probably breaks the AMQP protocol as defined, but I found it to
be really useful and kind of surprised that such a behaviour didn't exist.

Because I've been Erlanging for all of 2 days now, I just wanted to post my
diff here with the changes to do this feature and see if some of you
talented RabbitMQ devs could give it a quick glance and see if I might have
royally screwed up somewhere. I tested it for me and it works fine, but
haven't done much load-testing on it yet. I just really love that I can have
however-many STOMP connections to a single broadcast-queue and they all get
a message going to it, it's really great.

So attached is the diff for the server. I did the testing for this using
Barry's nice py-amqplib and made a change there too to send another Bit for
the boolean broadcast flag when declaring a queue; I attached the new file I
changed for that too.

I would just appreciate any feedback you guys might be able to give for this
and thank you again for such a great peice of software!

-Nemanja

P.S. The changes I made were to the tip of hg as of yesterday morning (well
morning for me in GMT-6).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20090422/f550617e/attachment.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: amqp-0.8.json
Type: application/octet-stream
Size: 25300 bytes
Desc: not available
Url : http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20090422/f550617e/attachment.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: channel.py
Type: application/octet-stream
Size: 84696 bytes
Desc: not available
Url : http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20090422/f550617e/attachment-0001.obj 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: broadcast.diff
Type: application/octet-stream
Size: 6933 bytes
Desc: not available
Url : http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20090422/f550617e/attachment-0002.obj 