[rabbitmq-discuss] Design for Comet

Thu Feb 23 16:54:50 GMT 2012

We have a comet-like server that has four basic tiers of messages:
page-level, user-level, topic, and global.  When a page is rendered
that hooks into the comet system, it initially subscribes to the comet
server using a POST with the tiers that it wants, and then it uses GET
to wait on new messages using long-polling.  To support this, I've
made a somewhat overly-complicated server that does a lot of internal
queueing, but is ultimately backed by Rabbit.

I've been looking at simplifying this, but first I thought I'd run it
past you guys, since I think my idea may have some components that
rabbit would hate.  Basically, what I want to do is make a short-lived
(and non-durable) queue per page (where a single page is dynamic, so
it tends to live for a while), and then have that queue be subscribed
to a "global" exchange, a "user" exchange, and a "topics" exchange.  I
see that Rabbit now has a queue-expiry extension, so unused queues
could be automatically garbage collected (I was thinking of basically
having a 10s idle timeout on my temp queues).  My concern is whether
rabbit likes having thousands or tens of thousands of queues created
and torn down every few minutes.  I seem to recall that it didn't like
that at all in the past, but Rabbit's been progressing like mad
lately, so I figured I'd ask if that's a sane thing to do these days.