[rabbitmq-discuss] best practice for EC2 deployment?

Wed May 20 21:09:41 BST 2009

Hi Geir,

Geir Magnusson Jr. wrote:
> I'm about to create a rabbitMQ cluster on EC2.
> 
> Anyone have any pointers or suggestions for best-practices or things  
> that will make this easier?

Basically, there are several things that I have seen people on this list do when deploying 
on EC2.

1. Consider putting Mnesia dir and persister log on EBS if you need persistence beyond 
reboot.

2. If you go with EBS, before creating mnesia dir and before starting broker off of data 
on EBS, you want to make sure your instance is running with the hostname that you control, 
not the one assigned by amazon (not all AMIs are affected by this). This way you will be 
able to use mnesia on another instance if this one crashes (that new instance will need 
same hostname). [1]

3. All hostnames should be defined in /etc/hosts or in DNS that you control so that nodes 
can talk to each other. Plus, (afaik) erlang can't [easily?] communicate over IPs without 
names.

I personally don't know how well rabbitmq reacts to changes in /etc/hosts - say hostA was 
at 10.1.1.1 and that instance crashed and new instance is now 10.2.2.2. As a potential 
solution, (attention! shameless plug) - you may check out VPN-Cubed for EC2 solution, that 
can help you get rid of this problem. [3] Or you may consider Elastic IPs (but they cost $$$).

4. Your clients may need to set insist=True when opening connections. This may become 
especially relevant if some clients are in EC2 and some are not (or are located in another 
region). [2]

5. I would also consider if you need clustering in the first place. Since queues 
themselves are currently not replicated (afaik), I think one of the main use cases for 
clustering is support for large numbers of clients (to better spread IO and connections). 
If it's not your usecase, I don't know if you can benefit from clustering. I would love to 
hear others' opinions as to reasons why they are using broker clusters in their current form.

I may have missed something, hopefully other folks can add/adjust as needed.

[1] http://www.nabble.com/Mnesia-db-integrity-td21754791.html
[2] http://www.google.com/search?q=site%3Alists.rabbitmq.com+insist+ec2+internal
[3] http://www.cohesiveft.com/vpn4ec2/

Hope this gives you a good start!

Cheers,
Dmitriy