[rabbitmq-discuss] Rabbit crash, new persister

[Matthew Sackman]

John,

On Sun, Jun 20, 2010 at 03:58:55PM +0200, John Apps wrote:
> <Begin of Soapbox>
> I have to ask what the rather strange approach to Windows is with the
> RabbitMQ folks?
> 
> Bashing Windows might be a great hobby, not terribly helpful though.

There is actually very little bashing going on, IMO.

1. No one at RabbitMQ uses Windows as their main development environment
because we tend to find it unpleasant.

2. One of the reasons it's unpleasant is that it seems to be full of
surprises - we're not making it up when we say that we have seen issues
with firewalls stopping AMQP clients from working *only* when XML is
being sent across the network. Not even Ubuntu contains such
"functionality" (see, I can bash Ubuntu too).

3. We *do* support Windows. If we didn't, we wouldn't have bothered
investing so much time in the various bat scripts that there are, and we
do test under Windows. However, it is, I'm sure very obviously the case,
that we do not test as thoroughly under Windows. For example, I suspect
this is the first time that the new persister is being tested in anger
under Windows. If we didn't support Windows, then we also wouldn't have
done the 1.7.2 release, which was almost entirely made up of fixes for
Windows. We do take Windows support seriously and have invested a lot of
time in trying to make it less painful for Windows users. If you have
specific suggestions as to where we could do more, we'd be happy to hear
them.

4. To date, and we recognise this may change, not many people have
complained about the status of our Windows support. I'm yet to hear of
anyone talking about Windows in the cloud (presumably because if you
added a window to a cloud, you'd let the rain out). I'm quite sure
Windows will continue to survive as a desktop OS, but *personally* I
would estimate that under 10% of our users are using Windows on servers.
Now sure, you could argue that's *because* of the status of Rabbit on
Windows - it could well be a chicken and egg situation. On the other
hand, if it really was that bad, I would have thought we'd have received
a lot more complaints.

> I understand that RabbitMQ and others are awaiting Erlang support for X64
> Windows, but to belittle the probably largest market you will find, and one
> that is very stable now with Windows Server 2008 and 7, I do wonder what the
> whole point is.

About 3 years ago, a couple of users were confused because Windows
seemed to be running Rabbit *much* faster than under Linux, particularly
with persistent messages in transactions. I spent about half a day
tracing fsync calls all through the Erlang VM to eventually find that,
under Windows, it was implemented as "return 1;". The Erlang developers
themselves very much treat Windows as a second class citizen. Now the
situation has improved somewhat since then, but there are still cases
where bugs likely exist under Windows in Erlang because Erlang
developers tend to assume Unix or Linux-ish behaviour. For example,
deleting mmap'd files is quite different under Linux and Windows, and it
would appear that Erlang with R14A is only now starting to try to ensure
file operations are the same under Linux and Windows.

The absence of a 64-bit Erlang under Windows is a *massive* issue for
us, but not one that we're in a position to do anything about. To a
certain extent, the new persister will alleviate the problem somewhat,
but really, without a 64-bit Erlang, RabbitMQ is unsuitable for *heavy*
workloads under Windows without some careful planning. That said, we do
have (some very large) customers using Rabbit under Windows without any
problem, in mission critical settings. It is not an issue *provided*
that you use Rabbit for routing rather than store-and-forward - i.e. try
to design your architecture so that the queues stay short. But this
advice applies not just to Windows - you will always get better
performance out of Rabbit if you, by design, avoid your queues becoming
very long.

The issue that Øyvind has identified *probably* exists with all previous
versions of Rabbit as I see no reason why it doesn't affect mnesia.
However, it could have something to do with multiple threads accessing
the same file, one as a reader and one as a writer, and then the writer
closing and deleting the file before the reader's closed it. We do this
in the new persister and it works fine under Linux. We have no idea
whether it could be a problem under Windows.

Currently we are a team of 7 developers. We have a roadmap that
stretches at least 3 years with the current head count. We are not in a
position to start tearing out and replacing large parts of the Erlang VM
or to try and understand every subtlety of developing to a closed
platform. Thus we rely on the API provided by the Erlang libraries and
the environment provided by the Erlang VM. When we come across bugs, we
report them to the Erlang developers - I have reported several bugs,
mainly regarding issues with the SSL modules, but beyond reporting bugs,
and the occasional patch, there's not a great deal more we can do.

> Why not come out with an explicit statement that RabbitMQ is simply NOT
> supported on Windows?

Because that's not the case. Please explain why you think this is the
case, and what we should be doing to convince you this isn't the case.

> Make it clear from the beginning rather than writing
> somewhat devious entries in discussion lists?

Please let us know what these "devious entries" are that we are making.

Best wishes,

Matthew