<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="generator" content="Osso Notes">
<title></title></head>
<body>
<p>Mj----- Original message -----
<br>> Send rabbitmq-discuss mailing list submissions to
<br>> <a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a>
<br>>
<br>> To subscribe or unsubscribe via the World Wide Web, visit
<br>>
<br>> <a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a>
<br>> or, via email, send a message with subject or body 'help' to
<br>> <a href="mailto:rabbitmq-discuss-request@lists.rabbitmq.com">rabbitmq-discuss-request@lists.rabbitmq.com</a>
<br>>
<br>> You can reach the person managing the list at
<br>> <a href="mailto:rabbitmq-discuss-owner@lists.rabbitmq.com">rabbitmq-discuss-owner@lists.rabbitmq.com</a>
<br>>
<br>> When replying, please edit your Subject line so it is more specific
<br>> than "Re: Contents of rabbitmq-discuss digest..."
<br>>
<br>>
<br>> Today's Topics:
<br>>
<br>> 1. Re: Installing RabbitMQ STOMP for PyGoWave (Simon MacMullen)
<br>> 2. Re: RabbitMQ for Master worker with monitoring? (yoav glazner)
<br>> 3. Re: RabbitMQ for Master worker with monitoring? (Emile Joubert)
<br>> 4. Re: Question about Connections and Channels (Thadeu Russo)
<br>> 5. Re: Installing RabbitMQ STOMP for PyGoWave (Chris)
<br>> 6. Re: RabbitMQ for Master worker with monitoring? (yoav glazner)
<br>>
<br>>
<br>> ----------------------------------------------------------------------
<br>>
<br>> Message: 1
<br>> Date: Thu, 23 Dec 2010 12:52:30 +0000
<br>> From: Simon MacMullen <<a href="mailto:simon@rabbitmq.com">simon@rabbitmq.com</a>>
<br>> Subject: Re: [rabbitmq-discuss] Installing RabbitMQ STOMP for PyGoWave
<br>> To: <a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a>
<br>> Message-ID: <<a href="mailto:4D13460E.30706@rabbitmq.com">4D13460E.30706@rabbitmq.com</a>>
<br>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
<br>>
<br>> On 23/12/2010 6:16AM, Chris wrote:
<br>> > Also, I forgot to mention that I've upgraded to RabbitMQ 2.2.0.
<br>>
<br>> In that case it will almost certainly be simplest to download the
<br>> plugin
<br>> as a binary from <a href="http://www.rabbitmq.com/plugins.html">http://www.rabbitmq.com/plugins.html</a>
<br>>
<br>> (NB: plugins tend to be very version-specific, and do not work at all
<br>> with 1.6.0.)
<br>>
<br>> Cheers, Simon
<br>>
<br>>
<br>> ------------------------------
<br>>
<br>> Message: 2
<br>> Date: Thu, 23 Dec 2010 15:25:42 +0200
<br>> From: yoav glazner <<a href="mailto:yoavglazner@gmail.com">yoavglazner@gmail.com</a>>
<br>> Subject: Re: [rabbitmq-discuss] RabbitMQ for Master worker with
<br>> monitoring?
<br>> To: rabbitmq-discuss <<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a>>
<br>> Message-ID:
<br>> <<a href="mailto:AANLkTimn9Fb2jvKszKORiEL-1j2OebohXgCZoLAhO-QF@mail.gmail.com">AANLkTimn9Fb2jvKszKORiEL-1j2OebohXgCZoLAhO-QF@mail.gmail.com</a>>
<br>> Content-Type: text/plain; charset="iso-8859-1"
<br>>
<br>> On Thu, Dec 23, 2010 at 12:26 PM, Emile Joubert <<a href="mailto:emile@rabbitmq.com">emile@rabbitmq.com</a>>
<br>> wrote:
<br>>
<br>> >
<br>> > Hi Yoav
<br>> >
<br>> >
<br>> > On 22/12/10 20:24, yoav glazner wrote:
<br>> >
<br>> > >
<br>> > > On Wed, Dec 22, 2010 at 2:10 PM, Emile Joubert <<a href="mailto:emile@rabbitmq.com">emile@rabbitmq.com</a>
<br>> > > <<a href="mailto:emile@rabbitmq.com">mailto:emile@rabbitmq.com</a>>> wrote:
<br>> > >
<br>> > > Hi Yoav,
<br>> > >
<br>> > >
<br>> > > On 21/12/10 21:35, yoav glazner wrote:
<br>> > >
<br>> > > This is my first post to this list, so Hi!
<br>> > > I'm trying to design a Master/Worker implement-ion with
<br>> RabbitMQ.
<br>> > > I have a few requierments due to lagacy issues:
<br>> > > * works get very long tasks so they should be able to get
<br>> ABORT
<br>> > > messages
<br>> > > while working,
<br>> > >
<br>> > >
<br>> > > You could send updated status information about work items to a
<br>> > > dedicated status exchange that workers subscribe to in order to
<br>> > > obtain abort messages or changes in priority. This assumes that
<br>> > > tasks are interruptible and that workers have multiple threads of
<br>> > > control.
<br>> > >
<br>> > > that's good
<br>> > >
<br>> >
<br>> > It sounds like you want to be able to communicate two different things
<br>> to
<br>> > workers:
<br>> >
<br>> > 1. tasks
<br>> > 2. task status updates
<br>> >
<br>> > The updates (abort, reprioritise, pause) to tasks are the difficult
<br>> part
<br>> > and the reason why frameworks like Celery won't work. I suggest you
<br>> > communicate task status updates to workers via a different route from
<br>> the
<br>> > way you submit tasks. Translating that to AMQP means using using a
<br>> separate
<br>> > exchange and queues, so workers accept tasks via a relatively
<br>> slow-moving
<br>> > queue and obtain updates about task status via a different queue. This
<br>> > assumes that workers have more than one thread of control for
<br>> listening on
<br>> > multiple queues and know how to interrupt running tasks. There may be
<br>> lots
<br>> > of other ways of accomplishing your goal.
<br>>
<br>>
<br>> But then I'll lose messages when a worker die, no?
<br>> -------------- next part --------------
<br>> An HTML attachment was scrubbed...
<br>> URL:
<br>> <<a href="http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101223/e36507ce/attachment-0001.htm>">http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101223/e36507ce/attachment-0001.htm></a>;
<br>>
<br>> ------------------------------
<br>>
<br>> Message: 3
<br>> Date: Thu, 23 Dec 2010 13:35:09 +0000
<br>> From: Emile Joubert <<a href="mailto:emile@rabbitmq.com">emile@rabbitmq.com</a>>
<br>> Subject: Re: [rabbitmq-discuss] RabbitMQ for Master worker with
<br>> monitoring?
<br>> To: yoav glazner <<a href="mailto:yoavglazner@gmail.com">yoavglazner@gmail.com</a>>
<br>> Cc: rabbitmq-discuss <<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a>>
<br>> Message-ID: <<a href="mailto:4D13500D.5080807@rabbitmq.com">4D13500D.5080807@rabbitmq.com</a>>
<br>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
<br>>
<br>> Hi Yoav
<br>>
<br>> On 23/12/10 13:25, yoav glazner wrote:
<br>> >
<br>> > But then I'll lose messages when a worker die, no?
<br>>
<br>> Acknowledging messages only after successful processing will prevent
<br>> them from being lost in case the consumer dies. If a consumer dies
<br>> unacknowledged messages will be requeued.
<br>>
<br>> Regards
<br>>
<br>> Emile
<br>>
<br>>
<br>> ------------------------------
<br>>
<br>> Message: 4
<br>> Date: Thu, 23 Dec 2010 11:51:29 -0200
<br>> From: Thadeu Russo <<a href="mailto:thadeurc@gmail.com">thadeurc@gmail.com</a>>
<br>> Subject: Re: [rabbitmq-discuss] Question about Connections and
<br>> Channels
<br>> To: Emile Joubert <<a href="mailto:emile@rabbitmq.com">emile@rabbitmq.com</a>>
<br>> Cc: <a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a>
<br>> Message-ID:
<br>> <<a href="mailto:AANLkTikOoXywKt8SGGNkZSDeMPRQucHAinNhr9U1e1m1@mail.gmail.com">AANLkTikOoXywKt8SGGNkZSDeMPRQucHAinNhr9U1e1m1@mail.gmail.com</a>>
<br>> Content-Type: text/plain; charset="iso-8859-1"
<br>>
<br>> Hi Emilie,
<br>>
<br>> thanks a lot for your comments.
<br>>
<br>> One thing you pointed out called my attention:
<br>>
<br>> "I would expect it to be more convenient for you to have an AMQP client
<br>> in
<br>> each JVM and therefore use a connection per JVM. AMQP connections are
<br>> cheap
<br>> resources and you need not try to conserve them. Brokers with tens of
<br>> thousands of connections are not unheard of."
<br>>
<br>> This is the way I started implementing, and in each JVM I can have
<br>> multiple
<br>> consumers and producer, so I was thinking about have 1 connection per
<br>> JVM
<br>> with one channel per producer/consumer. I like the idea of separate
<br>> producers and consumers.
<br>>
<br>> As the project goes, I will update the list with question/comments.
<br>>
<br>> Best Regards,
<br>>
<br>> On Wed, Dec 22, 2010 at 3:41 PM, Emile Joubert <<a href="mailto:emile@rabbitmq.com">emile@rabbitmq.com</a>>
<br>> wrote:
<br>>
<br>> >
<br>> > Hi Thadeu,
<br>> >
<br>> > Your project sounds interesting. If I understand your question
<br>> correctly
<br>> > you want to know if you should be using one connection with many
<br>> channels,
<br>> > or many connections each with a single channel.
<br>> >
<br>> > A possibly relevant consideration is that if the broker needs to apply
<br>> > backpressure because of an overactive producer then all the channels
<br>> sharing
<br>> > a connection with that producer will be blocked. Channels that can't
<br>> > tolerate this risk should have a connection to themselves. For the
<br>> same
<br>> > reason you may want to separate producers and consumers to use
<br>> different
<br>> > connections.
<br>> >
<br>> > I would expect it to be more convenient for you to have an AMQP client
<br>> in
<br>> > each JVM and therefore use a connection per JVM. AMQP connections are
<br>> cheap
<br>> > resources and you need not try to conserve them. Brokers with tens of
<br>> > thousands of connections are not unheard of.
<br>> >
<br>> >
<br>> > On 19/12/10 13:20, Thadeu Russo wrote:
<br>> >
<br>> > > Hi,
<br>> > >
<br>> > > I need to architect a Java application using rabbitmq as its message
<br>> > > broker. It is a distributed application and the message broker will
<br>> play
<br>> > > the role of connect the JVMs. Basically, each JVM has sets of objects
<br>> > > (currently they are grouped by host:port) that can change messages to
<br>> > > other objects in other groups in other virtual machines (there is a
<br>> 1:1
<br>> > > bi-directional relationship between groups of objects in different
<br>> JVMs).
<br>> > > The idea of use the broker is to avoid the number of tcp ports I have
<br>> to
<br>> > > open and also to take advantage of the message delivery guaranties.
<br>> > >
<br>> > > The design is simple: each group of objects will have an exchange and
<br>> 2
<br>> > > queues (incoming and outcoming) to change messages.
<br>> > >
<br>> > > I would like to know if you guys advise the following configuration
<br>> > > regarding rabbitmq:
<br>> > >
<br>> > > 1- Have 1 connection per JVM
<br>> > > 2- Have multiple channels to that connection. Each channel will be
<br>> > > associated with those 2 queues
<br>> > >
<br>> > > What is concerning me is: the number of sends/receives per channel is
<br>> > > high (about 5k/second). Should I have multiple connections (like a
<br>> > > pool), or it is ok to use just 1 connection? If I have a pool of
<br>> > > connections, I understand they will share the same tcp port, and I
<br>> could
<br>> > > not see any difference in using a single Connection.
<br>> > >
<br>> >
<br>> > The broker port will be same for each of multiple connections, but the
<br>> peer
<br>> > port will typically be random. The difference is that connections with
<br>> a
<br>> > single channel can fail and block without affecting other connections.
<br>> >
<br>> > Regards
<br>> >
<br>> > Emile
<br>> >
<br>>
<br>>
<br>>
<br>> --
<br>> Thadeu de Russo e Carmo
<br>> <a href="http://www.thadeu.com.br/">http://www.thadeu.com.br/</a>
<br>> <a href="http://www.caelum.com.br/">http://www.caelum.com.br/</a>
<br>> <a href="http://www.caelumobjects.com/">http://www.caelumobjects.com/</a>
<br>> -------------- next part --------------
<br>> An HTML attachment was scrubbed...
<br>> URL:
<br>> <<a href="http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101223/78c8d840/attachment-0001.htm>">http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101223/78c8d840/attachment-0001.htm></a>;
<br>>
<br>> ------------------------------
<br>>
<br>> Message: 5
<br>> Date: Thu, 23 Dec 2010 09:07:35 -0600
<br>> From: Chris <<a href="mailto:jesdisciple@gmail.com">jesdisciple@gmail.com</a>>
<br>> Subject: Re: [rabbitmq-discuss] Installing RabbitMQ STOMP for PyGoWave
<br>> To: <a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a>
<br>> Message-ID:
<br>> <<a href="mailto:AANLkTimsfciPxxXn8CQjhAJqfqKisMm0cEP7c4Oh80PU@mail.gmail.com">AANLkTimsfciPxxXn8CQjhAJqfqKisMm0cEP7c4Oh80PU@mail.gmail.com</a>>
<br>> Content-Type: text/plain; charset="iso-8859-1"
<br>>
<br>> Disregard my preious post; the plugins were only from a guide and I
<br>> don't
<br>> need the same ones.
<br>>
<br>> In ALL things, strive for ><>,
<br>> Chris
<br>>
<br>>
<br>> On Thu, Dec 23, 2010 at 12:16 AM, Chris <<a href="mailto:jesdisciple@gmail.com">jesdisciple@gmail.com</a>> wrote:
<br>>
<br>> > Also, I forgot to mention that I've upgraded to RabbitMQ 2.2.0.
<br>> >
<br>> -------------- next part --------------
<br>> An HTML attachment was scrubbed...
<br>> URL:
<br>> <<a href="http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101223/90180af6/attachment-0001.htm>">http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101223/90180af6/attachment-0001.htm></a>;
<br>>
<br>> ------------------------------
<br>>
<br>> Message: 6
<br>> Date: Thu, 23 Dec 2010 17:59:15 +0200
<br>> From: yoav glazner <<a href="mailto:yoavglazner@gmail.com">yoavglazner@gmail.com</a>>
<br>> Subject: Re: [rabbitmq-discuss] RabbitMQ for Master worker with
<br>> monitoring?
<br>> To: rabbitmq-discuss <<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a>>
<br>> Message-ID:
<br>> <<a href="mailto:AANLkTikMzXgntxT7+YHqm3rQ9oUGMc5SgvwZFvpS1pbS@mail.gmail.com">AANLkTikMzXgntxT7+YHqm3rQ9oUGMc5SgvwZFvpS1pbS@mail.gmail.com</a>>
<br>> Content-Type: text/plain; charset="iso-8859-1"
<br>>
<br>> On Thu, Dec 23, 2010 at 3:35 PM, Emile Joubert <<a href="mailto:emile@rabbitmq.com">emile@rabbitmq.com</a>>
<br>> wrote:
<br>>
<br>> > Hi Yoav
<br>> >
<br>> >
<br>> > On 23/12/10 13:25, yoav glazner wrote:
<br>> >
<br>> > >
<br>> > > But then I'll lose messages when a worker die, no?
<br>> > >
<br>> >
<br>> > Acknowledging messages only after successful processing will prevent
<br>> them
<br>> > from being lost in case the consumer dies. If a consumer dies
<br>> unacknowledged
<br>> > messages will be requeued.
<br>> >
<br>> > Regards
<br>>
<br>>
<br>> Thanks a lot for the input, I guess I'll have to dive in it now.
<br>> -------------- next part --------------
<br>> An HTML attachment was scrubbed...
<br>> URL:
<br>> <<a href="http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101223/32dd9681/attachment.htm>">http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101223/32dd9681/attachment.htm></a>;
<br>>
<br>> ------------------------------
<br>>
<br>> _______________________________________________
<br>> rabbitmq-discuss mailing list
<br>> <a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a>
<br>> <a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a>
<br>>
<br>>
<br>> End of rabbitmq-discuss Digest, Vol 43, Issue 64
<br>> ************************************************
<br>>
<br><br></p>
</body>
</html>