[rabbitmq-discuss] GPB and AMQP/RabbitMQ

[Kirk Wylie]

Hi, Chuck,

On Thu, Mar 26, 2009 at 2:59 PM, Chuck Remes <cremes.devlist at mac.com> wrote:
>> This is where some work that I did at my last job may pay off.
>> Essentially, we rolled our own packed, binary, self-describing,
>> hierarchical message format. Extremely compact (minimum 10 bytes
>> overhead), fields were all self-describing with type information,
>> everything was easy to map into direct CPUs (no varints), and fully
>> hierarchical. Unfortunately that implementation is trapped at the
>> former employer, but they've given the go-ahead to support a cleanroom
>> implementation of the encoding and tools in the open source arena (and
>> plan on helping with the effort, since they don't want to own it long
>> term).
>
> You two might be interested in something I ran across a few weeks ago.
> It's a self-describing binary protocol (exprot) up on github. You can
> read about it here [1]. There are already implementations in OCamel
> and Ruby. I'm sure it wouldn't be too tough to do others.

Indeed, I remember the day it came out, and blogged about it:
http://kirkwylie.blogspot.com/2009/01/extprot-very-cool-idea.html .

However, there hasn't benen that much of a takeup on the other
language front, and while OCaml and Ruby are all very well and good,
but without your big 3 languages (C++, C#, Java), it's of limited use.
And it doesn't look like the author's necessarily interested in
pushing it to a much wider audience by making it a general purpose
encoding system with multi-language bindings. Because my background is
in financial services (where there's some F# going on, but very little
Ruby and OCaml), I think we probably have something to add there.

That being said, my plan was ultimately to have a reasonable amount of
interchange with other system (so that you can use .proto files to
define a schema, or an extprot definition, or even an XSD). So I want
to learn from what he's done and see where I can learn from what the
extprot guys have done.

Good find, though, it's right up the alley: self describing, delimited, fast.

Kirk