[rabbitmq-discuss] Latest revision of NYTimes site is supported by RabbitMQ - launched 8 January 2014

[Laing, Michael]

We are using: a single Amazon Linux AMI, RabbitMQ, Cassandra 2, python 2

We use pika with tornado and libev for the nyt⨍aбrik wholesale and retail
pieces; our internal clients use Java and PHP.

We use shovels - lots of shovels - to interconnect.

In production we have a RabbitMQ client 3-cluster and a core 3-cluster in
each region on c1-xlarges. A proxy cluster of c1-mediums in Virginia connects
clients to the client clusters. All services are parallelized so we can add
more cores and clients.

The retail layer autoscales and use c1-mediums with a single rabbit
shovel-connected to one of the core rabbits. Each python websocket/sockjs
gateway supports up to 100K clients.

We autodeploy into subnets within Virtual Private Clouds in AWS. Clients
are routed via least latency to the fastest healthy region.

Of the technical components, the gateway is the most complex. We will be
moving it into open source in pieces and the first piece is likely to be
the python websocket/sockjs libraries which, frankly, beat the crap out of
most other stuff out there and fully conform with the relevant standards.
It can be loosely thought of as a C co-process managed by python, and as
such, may be possible to reuse in other languages/environments.

We have a 12-node Cassandra cluster across the 2 regions / 6 zones. It is
used for persistence of messages and as cache. We do not use persistence in
RabbitMQ. Our services are idempotent and important messages may be
replicated multiple times creating intentional race conditions in which the
fastest wins.

Although it may seem complex, nyt⨍aбrik has simple components and is mostly
principles and plumbing. The key point to grasp is that there is no head,
no master, no single point of failure. As I write this I can see components
failing (not RabbitMQ), and we are fixing them so they are more reliable.
But the system doesn't fail, users can connect, and messages are delivered,
regardless - all within design parameters.

Michael



On Wed, Jan 8, 2014 at 2:58 PM, Michael Klishin <mklishin at gopivotal.com>wrote:

> On 8 Jan 2014, at 23:26, Laing, Michael <michael.laing at nytimes.com> wrote:
>
> > Our premier online offering www.nytimes.com has a new look and new
> underpinnings, now including a messaging architecture implemented using
> RabbitMQ.
> >
> > This architecture - nyt⨍aбrik - has dozens of RabbitMQ instances spread
> across 6 AWS zones in Oregon and Dublin. The instances are organized into
> "wholesale" and "retail" layers. Connection to clients is via
> websockets/sockjs.
> >
> > Upon launch today, the system autoscaled to ~500,000 users. Connection
> times remained flat at ~200ms.
> >
> > nyt⨍aбrik provides subscription services for breaking news, video feeds,
> etc. and will add more event based services. It also supports individual
> messaging related to subscription status for registered users.
>
> Michael,
>
> Congrats on the launch!
>
> I’m curious what technologies and RabbitMQ clients do you use and plan on
> adding
> in the future?
>
> MK
>
> Software Engineer, Pivotal/RabbitMQ
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140109/9831923b/attachment.html>