[rabbitmq-discuss] Clustering and data partitioning - how best to implement

[Alexis Richardson]

Does this help?

http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2011-September/015263.html




On Mon, Sep 26, 2011 at 9:05 PM, justin.adam.miller at gmail.com
<justin.adam.miller at gmail.com> wrote:
> I'm trying to implement some of the ideas from this article:
>
> http://blog.springsource.com/2011/04/01/routing-topologies-for-performance-and-scalability-with-rabbitmq/
>
> Our situation is that we have two rabbit instances, not clustered, so
> that we can spread out load. In our current setup, it's still possible
> for us to hit one broker harder than the other, so ideally we'd like
> to setup these nodes in a cluster. That way, we can have consumers
> evenly distributed across brokers.
>
> We could create one queue that all consumers consume from, but I'm not
> sure that would solve our problems since that queue (and the broker it
> was created on) would become a hotspot. Right?
>
> So from that article, I like the idea of each node creating a queue,
> e.g. myQueue.1, myQueue.2, etc. and having the producer round robin
> them. What I don't know, is how the producer knows how many queues/
> nodes there are. The producers could send to an exchange and just set
> the routingKey, but still, I think the problem is the same.
>
> I guess I could use the management plugin, but I'm trying to figure
> out how to do that without issuing a request for every message I
> produce.
>
> I could imagine some kind of handshake when a producer comes online,
> he sends a "who's-alive" message to a topic exchange and waits for
> responses from consumers. Also, when a consumer comes online, he
> immediately sends out a "I'm-alive" message. In a direct exchange
> (either the default or one that I create), is a routingKey assumed to
> be a queue name? If I made the queues auto_delete, what would happen
> when the producer tries to send a message to a queue that no longer
> exists? Would I get an exception such that I can catch it and send the
> message to the next guy in the list?
>
> Thanks,
> Justin
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>