[rabbitmq-discuss] Robust zone-aware topology?

[Mordy Ovits]

I'm new to rabbitmq and would appreciate some feedback on a design.

I'm looking to create a robust topology for routing messages in a zone-aware way.  I'd like messages to be sent to a "zone" and be handled by zone-local machines reading from a queue in that zone.  In case all the consumers in a zone are down, consumers in another zone become eligible to consume the messages.  If the consumers in the first zone come back, messages to the zone resume routing to the now-up consumers.  Messages should never go to more than one queue, but a rare lost message isn't the end of the world.  IOW, I'd prefer for the machines local to a zone handle the messages in that zone, but if they can't, then another zone can do it until that zone's consumers recover.

The solution I came up with depends on RabbitMQ's alternate-exchange feature.  I create (say) 3 exchanges, each with an a-e of another. E.g.:
X: new_york (a-e: miami)
X: miami (a-e: new_york)
X: london (a-e: new_york)

There is one queue per exchange, direct binding, with auto_delete true.  Consumers in a zone all read only from that one queue.  Since the queue is auto_delete, if no consumer is bound to the (by-design-only) queue in that zone's exchange, the exchange sends its messages to its a-e.

I've tested this and it works.  I can bring consumers up and down in the zone and the locality of message routing is maintained where possible and returns at recovery. However, I have some questions. 

1) Is there a better way to do this?  It seems odd to need a rabbitmq extension, so I'm suspicious I'm missing something basic.  Maybe with federation or shovel?  Keep in mind that a message should only go to one queue.

2) Do I have to worry about a-e routing loops?  The documentation implies it's ok, but doesn't spell it out.  If I have an alternate-exchange path of "a -> b -> c -> a" could it go haywire when all the queues go down?  The (one-hop) testing I did seemed to handle it, but I'd like a clearer understanding of how rabbitmq prevents a-e routing loops.

3) How can I isolate the concerns of producers from the HA topology?  I'd prefer if the producers didn't have to know about the HA nature of the exchange, so that I can change it independent of them (i.e. change the a-e parameter).  However, the "identity" of an exchange in terms of the test for its preexistence in the declare seems to include every parameter, including the a-e setting.  Therefore, producers get failure on their exchange_declare if their zone's exchange details aren't completely identical, but I might have 5 producers managed by three groups.  I might be able to manually manage the creation of exchanges and have the producers use but not declare them, but perhaps there's a cleaner way that I don't know.

Thanks,
Mordy