[rabbitmq-discuss] [Minimum Air Induction] Introducing Shovel: An AMQP Relay
0x6e6562 at gmail.com
Sun Sep 21 17:17:16 BST 2008
On Sat, Sep 20, 2008 at 8:18 PM, Valentino Volonghi <dialtone at gmail.com> wrote:
> There is still the duplicates problem but this is a minor issue right now
> at least it's solvable. (It wouldn't hurt to have 2PC :P).
Sure, but I think these days people are going for 1.5PC instead of 2PC
- it's just a cost-benefit comparison.
>> Rabbit will nuke it. Logging the message to disk is done by the queue
>> process, so if nothing gets routed, nothing gets persisted.
> Ok, so I need durable queues and will wait for the overflow on disk
Yes. AMQP is consumer driven, not producer driven.
>> Yep. Don't know what the exact factor is though.
> Ouch... I'll try later today or tomorrow to see what this value actually is
> shovel in the middle... I also wonder which header I should use to make a
> persistent, I'll look into this I guess.
I shouldn't imagine that this will cost you an order of magnitude -
the persister is write-optimized after all, though it does have to
synchronously write to disk.
> They look OK in the sense that even persistent messages can reach those
I guess the IO bandwidth of your filesystem would be an important
factor in this equation.
>> I think that you need to work out what ingress requirements you have
>> (this will be determined by the capacity of the http server) and what
>> egress you need (so as to avoid stuff queuing up too much).
>> Remember that ATM Rabbit does not implement QoS, so your egress will
>> be bound by the slowest consumer.
> What if I have multiple consumers on a queue? Does speed scales almost
No, because the current queue delivery mechanism takes a
robin-robin-ish approach, so the overall throughput is rate limited by
the slowest consumer subscribed to a queue.
This should be addressed when we implement QoS, which would take a
credit based approach that is determined by consumer acks.
If you are interested in the details of what is planned, there is a
design for it here: https://dev.rabbitmq.com/wiki/BasicQosDesign
As indicated before, there is no ETA on this feature.
More information about the rabbitmq-discuss