[rabbitmq-discuss] Use of Java 1.4 with the RabbitMQ Java Client library

[Holger Hoffstätte]

Kirk Wylie wrote:
> On Wed, Jan 7, 2009 at 2:15 PM, Holger Hoffstätte <holger at wizards.de> wrote:
>> I've had many ideas flying around in my head (some crazy, some rather
>> mundane) but haven't really found the time to implement any yet. Can you
>> share what you're planning?
> 
> One of the first things I wanted to do was to improve the current test
> suite to add quite a few more "pure" unit tests (right now it's almost

good - though not sure how much they are ultimately worth it, as much of
the functionality is in the autogenerated protocol. Other than that I
found the code to be very readable and understandable, especially
considering how much time could be spent on it.

> Then the first crazy/mundane thing that I was going to attempt was a
> connection pooling system. Because that'll be quite complex, I want to
> tackle the test side first so that I'm confident that I'm not breaking
> things in subtle ways.

Oh dear. :) Got so many connections? I thought they were supposed to be
kept open.

> What ideas did you have?

Crazy German Alert!

- define common org.amqp interfaces (ala JMS) so that client app code
doesn't rely on the client lib implementation. Would require some work
with the WG but as the spec, entities and semantics keep changing it would
be a moving target so I haven't really bothered yet. Not sure if anybody
would even care as soon as the JMS mapping has been wrangled into the spec.

- n:m mapping of channels to socket connections, based on either some
up-front fixed ratio/priority scheme or dynamically adjusted traffic per
channel (maybe grouping slow/mostly-idle periodic polling/ticker channels
together). This would allow to connect to multiple broker instances in
parallel.

- maybe moving to NIO since it might nicely map to AMQP channels. A Netty
AMQP packet codec should be easy, and a consumer could register a payload
decoder on top of a channel for easy payload disassembly - Thrift,
Protobufs, XML (arrgh).

- more concurreny all the way, decoupling fragment assembly/concurrent
consumer delivery and the reader thread, backed by an injectable Executor
(looking at you, WorkManager). Incidentally that's how Netty works too.
Also the channel map is pretty oversynchronized and used from multiple
paths..not so good for multiple connections.

- the above would also accidentally slip in a JTA transaction thread boundary.

- no blocking queues and only concurrent dequeues where they cannot be
avoided. only blocking on condition objects, and always with timeouts.

- find a way towards as-zero-copy-as-possible fragment assembly. Each time
I see dozens of litte byte[]s being copied around 17 times for a single
message because of the !"§$ stream APIs my inner geek dies a little.

- JMX beanz for stats

- write a Spring-Integration transport because it happens to be based on
channel abstractions and will likely map nicely.

like I said, crazy.. and probably totally useless.

More JUnit tests, increased coverage or a proper maven build are probably
more helpful :-)

Holger