[rabbitmq-discuss] flow control and erlang client(s)

Wed Jun 24 04:59:03 BST 2009

It makes sense that it is not reasonable to rate limit a publisher's
connection to a queue, since publishers don't connect to queues, they
connect to exchanges.   However, it shouldn't be too complex to rate
limit any two points that connect directly.  The trouble here is in
rate-limiting through a routed path.  Its like a train where you can
adjust the tension between two connected cars, but not the tension
between the first and last cars.  Thus, if you want to rate-limit
between an exchange and a specific queue, that might be possible, but if
you want to rate limit between a publisher and a queue, that isn't.

Yet, I'm not sure that its *impossible* to rate limit a publisher's
messages destined for a specific queue, even on the current code, but it
would be such a kludge, and so site-specific, that nobody would
reasonably wish to implement it.  Even then, it would be very limited in
its effectiveness.  Of course, Matthias knows better than I do, but I
believe I'm speaking rightly.  With major architecture changes, this
might be easier to implement, and it might be a use-case to suggest
when/if an architectural change is considered.  Again, refer to Matthias
on this.

Now, as for the "more reasonable" rate-limiting features, such as "from
car to car" in my train example, there are still concerns.  Based on my
discussions with Matthias, there is a significant and reasonable worry
that such checks can be performance heavy.  They're arguably vital for
some deployments, but unnecessarily slow for everyone else.  Surely,
like any software, there must be a line drawn between performance and
functionality, and I personally suspect that performance is the current
priority amongst the core team.

Last night, I suggested to Matthias that it might be worthwhile to
investigate ways to tack on "slow" code for those that need it, without
significantly impacting everyone else.  Examples I provided would be
registering handlers via modules, or an ability to subclass and extend
the Rabbit server itself, building independent servers built on a Rabbit
framework.  For this precise reason, as two examples, Apache and Linux
both have pluggable modules, compile-time feature sets, varied
workers/schedulers, etc.

The first step to this would be to decide if such an architecture is the
compromise solution for this problem, if Rabbit wants to go there, and
broadly, how.   Then, address the roadmap (something I feel not
worthwhile discussing at this point).

-- 
Regards,
Eric Windisch