[rabbitmq-discuss] rabbitmq-discuss Digest, Vol 43, Issue 64

Thu Dec 23 16:17:16 GMT 2010

Mj----- Original message -----
> Send rabbitmq-discuss mailing list submissions to
>                rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>
>
> To subscribe or unsubscribe via the World Wide Web, visit
>
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> or, via email, send a message with subject or body 'help' to
>                rabbitmq-discuss-request at lists.rabbitmq.com<mailto:rabbitmq-discuss-request at lists.rabbitmq.com>
>
> You can reach the person managing the list at
>                rabbitmq-discuss-owner at lists.rabbitmq.com<mailto:rabbitmq-discuss-owner at lists.rabbitmq.com>
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of rabbitmq-discuss digest..."
>
>
> Today's Topics:
>
>      1. Re: Installing RabbitMQ STOMP for PyGoWave (Simon MacMullen)
>      2. Re: RabbitMQ for Master worker with monitoring? (yoav glazner)
>      3. Re: RabbitMQ for Master worker with monitoring? (Emile Joubert)
>      4. Re: Question about Connections and Channels (Thadeu Russo)
>      5. Re: Installing RabbitMQ STOMP for PyGoWave (Chris)
>      6. Re: RabbitMQ for Master worker with monitoring? (yoav glazner)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 23 Dec 2010 12:52:30 +0000
> From: Simon MacMullen <simon at rabbitmq.com<mailto:simon at rabbitmq.com>>
> Subject: Re: [rabbitmq-discuss] Installing RabbitMQ STOMP for PyGoWave
> To: rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>
> Message-ID: <4D13460E.30706 at rabbitmq.com<mailto:4D13460E.30706 at rabbitmq.com>>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> On 23/12/2010 6:16AM, Chris wrote:
> > Also, I forgot to mention that I've upgraded to RabbitMQ 2.2.0.
>
> In that case it will almost certainly be simplest to download the
> plugin
> as a binary from http://www.rabbitmq.com/plugins.html
>
> (NB: plugins tend to be very version-specific, and do not work at all
> with 1.6.0.)
>
> Cheers, Simon
>
>
> ------------------------------
>
> Message: 2
> Date: Thu, 23 Dec 2010 15:25:42 +0200
> From: yoav glazner <yoavglazner at gmail.com<mailto:yoavglazner at gmail.com>>
> Subject: Re: [rabbitmq-discuss] RabbitMQ for Master worker with
>                monitoring?
> To: rabbitmq-discuss <rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>>
> Message-ID:
>                <AANLkTimn9Fb2jvKszKORiEL-1j2OebohXgCZoLAhO-QF at mail.gmail.com<mailto:AANLkTimn9Fb2jvKszKORiEL-1j2OebohXgCZoLAhO-QF at mail.gmail.com>>
> Content-Type: text/plain; charset="iso-8859-1"
>
> On Thu, Dec 23, 2010 at 12:26 PM, Emile Joubert <emile at rabbitmq.com<mailto:emile at rabbitmq.com>>
> wrote:
>
> >
> > Hi Yoav
> >
> >
> > On 22/12/10 20:24, yoav glazner wrote:
> >
> > >
> > > On Wed, Dec 22, 2010 at 2:10 PM, Emile Joubert <emile at rabbitmq.com<mailto:emile at rabbitmq.com>
> > > <mailto:emile at rabbitmq.com>> wrote:
> > >
> > > Hi Yoav,
> > >
> > >
> > > On 21/12/10 21:35, yoav glazner wrote:
> > >
> > > This is my first post to this list, so Hi!
> > > I'm trying to design a Master/Worker implement-ion with
> RabbitMQ.
> > > I have a few requierments due to lagacy issues:
> > > * works get very long tasks so they should be able to get
> ABORT
> > > messages
> > > while working,
> > >
> > >
> > > You could send updated status information about work items to a
> > > dedicated status exchange that workers subscribe to in order to
> > > obtain abort messages or changes in priority. This assumes that
> > > tasks are interruptible and that workers have multiple threads of
> > > control.
> > >
> > > that's good
> > >
> >
> > It sounds like you want to be able to communicate two different things
> to
> > workers:
> >
> > 1. tasks
> > 2. task status updates
> >
> > The updates (abort, reprioritise, pause) to tasks are the difficult
> part
> > and the reason why frameworks like Celery won't work. I suggest you
> > communicate task status updates to workers via a different route from
> the
> > way you submit tasks. Translating that to AMQP means using using a
> separate
> > exchange and queues, so workers accept tasks via a relatively
> slow-moving
> > queue and obtain updates about task status via a different queue. This
> > assumes that workers have more than one thread of control for
> listening on
> > multiple queues and know how to interrupt running tasks. There may be
> lots
> > of other ways of accomplishing your goal.
>
>
> But then I'll lose messages when a worker die, no?
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101223/e36507ce/attachment-0001.htm>;
>
> ------------------------------
>
> Message: 3
> Date: Thu, 23 Dec 2010 13:35:09 +0000
> From: Emile Joubert <emile at rabbitmq.com<mailto:emile at rabbitmq.com>>
> Subject: Re: [rabbitmq-discuss] RabbitMQ for Master worker with
>                monitoring?
> To: yoav glazner <yoavglazner at gmail.com<mailto:yoavglazner at gmail.com>>
> Cc: rabbitmq-discuss <rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>>
> Message-ID: <4D13500D.5080807 at rabbitmq.com<mailto:4D13500D.5080807 at rabbitmq.com>>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Hi Yoav
>
> On 23/12/10 13:25, yoav glazner wrote:
> >
> > But then I'll lose messages when a worker die, no?
>
> Acknowledging messages only after successful processing will prevent
> them from being lost in case the consumer dies. If a consumer dies
> unacknowledged messages will be requeued.
>
> Regards
>
> Emile
>
>
> ------------------------------
>
> Message: 4
> Date: Thu, 23 Dec 2010 11:51:29 -0200
> From: Thadeu Russo <thadeurc at gmail.com<mailto:thadeurc at gmail.com>>
> Subject: Re: [rabbitmq-discuss] Question about Connections and
>                Channels
> To: Emile Joubert <emile at rabbitmq.com<mailto:emile at rabbitmq.com>>
> Cc: rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>
> Message-ID:
>                <AANLkTikOoXywKt8SGGNkZSDeMPRQucHAinNhr9U1e1m1 at mail.gmail.com<mailto:AANLkTikOoXywKt8SGGNkZSDeMPRQucHAinNhr9U1e1m1 at mail.gmail.com>>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi Emilie,
>
> thanks a lot for your comments.
>
> One thing you pointed out called my attention:
>
> "I would expect it to be more convenient for you to have an AMQP client
> in
> each JVM and therefore use a connection per JVM. AMQP connections are
> cheap
> resources and you need not try to conserve them. Brokers with tens of
> thousands of connections are not unheard of."
>
> This is the way I started implementing, and in each JVM I can have
> multiple
> consumers and producer, so I was thinking about have 1 connection per
> JVM
> with one channel per producer/consumer. I like the idea of separate
> producers and consumers.
>
> As the project goes, I will update the list with question/comments.
>
> Best Regards,
>
> On Wed, Dec 22, 2010 at 3:41 PM, Emile Joubert <emile at rabbitmq.com<mailto:emile at rabbitmq.com>>
> wrote:
>
> >
> > Hi Thadeu,
> >
> > Your project sounds interesting. If I understand your question
> correctly
> > you want to know if you should be using one connection with many
> channels,
> > or many connections each with a single channel.
> >
> > A possibly relevant consideration is that if the broker needs to apply
> > backpressure because of an overactive producer then all the channels
> sharing
> > a connection with that producer will be blocked. Channels that can't
> > tolerate this risk should have a connection to themselves. For the
> same
> > reason you may want to separate producers and consumers to use
> different
> > connections.
> >
> > I would expect it to be more convenient for you to have an AMQP client
> in
> > each JVM and therefore use a connection per JVM. AMQP connections are
> cheap
> > resources and you need not try to conserve them. Brokers with tens of
> > thousands of connections are not unheard of.
> >
> >
> > On 19/12/10 13:20, Thadeu Russo wrote:
> >
> > > Hi,
> > >
> > > I need to architect a Java application using rabbitmq as its message
> > > broker. It is a distributed application and the message broker will
> play
> > > the role of connect the JVMs. Basically, each JVM has sets of objects
> > > (currently they are grouped by host:port) that can change messages to
> > > other objects in other groups in other virtual machines (there is a
> 1:1
> > > bi-directional relationship between groups of objects in different
> JVMs).
> > > The idea of use the broker is to avoid the number of tcp ports I have
> to
> > > open and also to take advantage of the message delivery guaranties.
> > >
> > > The design is simple: each group of objects will have an exchange and
> 2
> > > queues (incoming and outcoming) to change messages.
> > >
> > > I would like to know if you guys advise the following configuration
> > > regarding rabbitmq:
> > >
> > > 1- Have 1 connection per JVM
> > > 2- Have multiple channels to that connection. Each channel will be
> > > associated with those 2 queues
> > >
> > > What is concerning me is: the number of sends/receives per channel is
> > > high (about 5k/second). Should I have multiple connections (like a
> > > pool), or it is ok to use just 1 connection? If I have a pool of
> > > connections, I understand they will share the same tcp port, and I
> could
> > > not see any difference in using a single Connection.
> > >
> >
> > The broker port will be same for each of multiple connections, but the
> peer
> > port will typically be random. The difference is that connections with
> a
> > single channel can fail and block without affecting other connections.
> >
> > Regards
> >
> > Emile
> >
>
>
>
> --
> Thadeu de Russo e Carmo
> http://www.thadeu.com.br/
> http://www.caelum.com.br/
> http://www.caelumobjects.com/
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101223/78c8d840/attachment-0001.htm>;
>
> ------------------------------
>
> Message: 5
> Date: Thu, 23 Dec 2010 09:07:35 -0600
> From: Chris <jesdisciple at gmail.com<mailto:jesdisciple at gmail.com>>
> Subject: Re: [rabbitmq-discuss] Installing RabbitMQ STOMP for PyGoWave
> To: rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>
> Message-ID:
>                <AANLkTimsfciPxxXn8CQjhAJqfqKisMm0cEP7c4Oh80PU at mail.gmail.com<mailto:AANLkTimsfciPxxXn8CQjhAJqfqKisMm0cEP7c4Oh80PU at mail.gmail.com>>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Disregard my preious post; the plugins were only from a guide and I
> don't
> need the same ones.
>
> In ALL things, strive for ><>,
> Chris
>
>
> On Thu, Dec 23, 2010 at 12:16 AM, Chris <jesdisciple at gmail.com<mailto:jesdisciple at gmail.com>> wrote:
>
> > Also, I forgot to mention that I've upgraded to RabbitMQ 2.2.0.
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101223/90180af6/attachment-0001.htm>;
>
> ------------------------------
>
> Message: 6
> Date: Thu, 23 Dec 2010 17:59:15 +0200
> From: yoav glazner <yoavglazner at gmail.com<mailto:yoavglazner at gmail.com>>
> Subject: Re: [rabbitmq-discuss] RabbitMQ for Master worker with
>                monitoring?
> To: rabbitmq-discuss <rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>>
> Message-ID:
>                <AANLkTikMzXgntxT7+YHqm3rQ9oUGMc5SgvwZFvpS1pbS at mail.gmail.com<mailto:AANLkTikMzXgntxT7+YHqm3rQ9oUGMc5SgvwZFvpS1pbS at mail.gmail.com>>
> Content-Type: text/plain; charset="iso-8859-1"
>
> On Thu, Dec 23, 2010 at 3:35 PM, Emile Joubert <emile at rabbitmq.com<mailto:emile at rabbitmq.com>>
> wrote:
>
> > Hi Yoav
> >
> >
> > On 23/12/10 13:25, yoav glazner wrote:
> >
> > >
> > > But then I'll lose messages when a worker die, no?
> > >
> >
> > Acknowledging messages only after successful processing will prevent
> them
> > from being lost in case the consumer dies. If a consumer dies
> unacknowledged
> > messages will be requeued.
> >
> > Regards
>
>
> Thanks a lot for the input, I guess I'll have to dive in it now.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101223/32dd9681/attachment.htm>;
>
> ------------------------------
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
> End of rabbitmq-discuss Digest, Vol 43, Issue 64
> ************************************************
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101223/3e626c34/attachment-0001.htm>