[rabbitmq-discuss] Searching for a client library/example: Problems getting a reliable connection in Python and Erlang

[Laing, Michael P.]

Here's a straightforward publisher using pika async library.

 1.  Be sure you are using pika 0.9.6 (released about a week ago).
 2.  Create an exchange 'my_log_exchange' in vhost '/'. This and the step below could be done in code – eliminated for simplicity.
 3.  Create a queue '<whatever>' in vhost '/' and bind it to the above exchange with key '#' (to get everything)
 4.  Create a file named 'my_id' in the same directory (or findable) by the code below. Put some text in it.
 5.  Run the program. Look in the queue. Get your message as JSON with the body attribute set to the text.

I am using python 2.7 but this should work with 2.6. http://pika.readthedocs.org/ has good examples too.

Michael

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import sys
import json

import pika

class BasicPublish(object):
    def __init__(self):
        self._properties = pika.BasicProperties(content_type="application/json", user_id='guest')
        self._credentials = pika.PlainCredentials('guest', 'guest')
        self._connection = None
        self._channel = None

    def put_log(self):
        print 'put_message'
        key = 'my_id'
        body = self.get_body(key)
        message = dict(body=body)
        self.put_message(key, message)

    def get_body(self, key):
        print 'get_body'
        body = self.get_file_contents(key)
        return body

    def get_file_contents(self, fname):
        print 'get_file_contents'

        with open(fname) as f:
            return f.read()

    def put_message(self, key, message):
        print 'put_message'

        self._channel.basic_publish(
            exchange='my_log_exchange',
            routing_key=key,
            body=json.dumps(message),
            properties=self._properties
        )

    def channel_on_open_callback(self, channel):
        print 'channel_on_open_callback: channel:{0}'.format(channel)
        self._channel = channel
        self.put_log()

    def connection_callback(self, connection):
        print 'connection_callback'
        self._connection.channel(on_open_callback=self.channel_on_open_callback)

    def connection(self):
        print 'connection'
        parameters = pika.ConnectionParameters(
            virtual_host='/',
            credentials=self._credentials
        )

        self._connection = pika.SelectConnection(parameters, self.connection_callback)

    def connection_close(self):
        self._connection.close()

    def stop(self): # called on interrupt
        self.connection_close()
        self._connection.ioloop.start() # stopped by interrupt - must be restarted to process commands

    def run(self):
        print 'run'
        self.connection()
        self._connection.ioloop.start()

def main(argv=None):
    if argv is None: argv = sys.argv
    basic_publish = BasicPublish()

    try:
        basic_publish.run()
    except KeyboardInterrupt:
        print >>sys.stderr, "Interrupt!"
        basic_publish.stop()

if __name__ == "__main__":
    sys.exit(main())


From: Jerry Kuch <jerryk at rbcon.com<mailto:jerryk at rbcon.com>>
Reply-To: rabbitmq <rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>>
Date: Sun, 4 Nov 2012 13:24:38 -0500
To: rabbitmq <rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>>
Subject: Re: [rabbitmq-discuss] Searching for a client library/example: Problems getting a reliable connection in Python and Erlang

Also, further to what Alvaro said:  The Erlang client is quite reliable and, indeed, is used by us internally for a range of things.  Its main downside is that the documentation can be a bit sparse, especially if you're new to Erlang, and it has had some API changes over the years that have rendered some of what you might read about it on the internet out of date (again, in ways that are probably harder to sort out if you're new to the whole mess)...

+1 to Alvaro's request for some more specifics on what you're up to helping us provide guidance... :-)

Best regards,
Jerry

On Sun, Nov 4, 2012 at 10:20 AM, Alvaro Videla <videlalvaro at gmail.com<mailto:videlalvaro at gmail.com>> wrote:
Would you mind sharing some code? Is hard for us to help you without code.

Regards,

Alvaro

Sent from my iPhone

On Nov 4, 2012, at 7:06 PM, Daniel Neugebauer <mailinglists at energiequant.de<mailto:mailinglists at energiequant.de>> wrote:

> Hi!
>
> I wanted to use RabbitMQ for simple log collection but for some reason I
> can't find any useful AMQP client library (and what good is a server
> without a working client?). I'd like to implement in either Python or
> Erlang but I'm unable to get a reliable connection - either connection
> setup fails, strange error messages appear after a short uptime or the
> client just cannot handle connection loss.
>
> What I've written so far is a simple Python script that takes lines from
> stdin, builds a dictionary from it and tries to send it via amqplib,
> Pika or Kombu (which seems to decide on using amqplib internally). It's
> nothing special, I already have a serialized string, that's all I need
> to send.
>
> However, Pika or amqplib (it's been a while since I tried them, so I
> don't remember which one it was) appeared to have some buffer issue -
> although the connection (to localhost) was still alive, I would get
> error messages about messages queueing up in the client after a few
> minutes. The other library would loose connection a few seconds after
> connecting but doesn't notice. Kombu terminates when the connection to
> RabbitMQ is lost, although I would expect Kombu to retry since I set
> retry = True when calling publish on the Producer object. I'm already
> running a reconnect in a loop with a try/except inside but my script
> just terminates. Strange...
>
> I also tried to use amqp_client in Erlang but failed to get any
> connection (I always get unknown_host unless I enter nothing as host
> name). I should add that I only read into Erlang theoretically so far
> and didn't get to write much but example programs yet, so it's likely I
> do something wrong here (yes, I already tried both binary lists and
> regular lists, no difference; I even tried supplying IP addresses in
> various forms - without success). Documentation is scarce and what I
> found suggests that I'm doing nothing wrong. I would prefer staying with
> Python for this implementatin anyway, just thought the client that comes
> with RabbitMQ should be able to work fine - unfortunately I seem to be
> wrong here. :(
>
> My question is: Does anyone have a complete working example of a
> *reliable* client in Python (or Erlang)? Maybe I'm too stupid to read
> the (very brief?) documentation correctly or the libraries I tried so
> far are just too buggy... What I started with was an example from
> "RabbitMQ in Action" but even that didn't work as expected.
>
> If it wasn't for getting logs piped in on the console I would try Java
> clients instead as I expect them to be more mature - to me, the Python
> libraries appear to be a bit messy, pardon my (maybe wrong) impression
> but I'm starting to get very frustrated with this: I've never had more
> problems to just get a simple plain string handed over to some other
> system. All in all I've spent too many hours in trying to get anything
> working and nothing has worked so far.
>
> Thanks,
> Daniel
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20121104/6f67d0c1/attachment.htm>