[rabbitmq-discuss] stability of on-disk formats

[John Beck]

Greetings,

We are trying to determine the optimal paths for various on-disk storage
locations (e.g., message store, databases) of RabbitMQ on Solaris, and need
a key piece of information to make this decision.  That is whether the on-disk
formats are generally compatible across upgrades or not.

Background: Solaris supports Boot Environments (BEs; see beadm(1m) for
details).  These allow for easy fall-back if something goes wrong with an OS
upgrade or the system is otherwise damaged.  As such, one factor that needs
to be considered with most sub-systems that use persistent storage is whether
that storage should be in a path that is BE-dependent or BE-independent.
The key considerations are these:

* The time from when an old BE is cloned until a new BE is booted into
  represents a window of potential loss or duplication of messages if BE-
  dependent paths are used.
* For software with version incompatibilities across upgrades, a potential
  inability to handle older (or occasionally newer) message formats exists
  if BE-independent paths are used.

The former consideration generally presents a much greater risk than the
latter, especially for software that is relatively stable in terms of its
on-disk storage formats.

If, as we hope, and some mailing list search hits from 2011 suggested, the
on-disk formats used by RabbitMQ are generally compatible, then we can store
everything in paths that are BE-independent, as this would have the huge
advantage of not losing or duplicating data across upgrades.  In particular,
a message from Simon MacMullen in March 2011 (thread subject: "Rolling upgrade
of a cluster") suggested that upgrades from 2.1.1 onwards would be nearly
seamless if some simple steps were followed, which sounds hopeful that the
formats in question are relatively compatible, or at least easily manageable,
but it seems prudent to ask rather than risk misunderstanding.  So here I am
asking.  :-)

Thank you for your time...

-- John

http://blogs.oracle.com/jbeck/