[rabbitmq-discuss] Newton + Mule + RabbitMQ --- (Was: NPE in RpcClient)

Sat Apr 5 05:39:14 BST 2008



On 4 Apr 2008, at 04:50, Neil Ellis wrote:

> I'll put out some notes later today about issues and the fixes I  
> used.  I noticed we're still getting lifecycle issues in Mule 1.4.3  
> - I notice this in the ActiveMQ version also. It seems that  
> listeners are becoming active before the components are ready to  
> receive messages. I'll try and get round to posting the details onto  
> the Mule lists.
Nicely done Neil, can you raise an issue about the lifecycle. I doubt  
its an issue in 2.0
>
>
> We need to have a patched Mule connector and (as long as the NPE is  
> fixed) RabbitMQ 1.3, I'll then create a page in the http://mule4newton.codecauldron.org 
>  microsite with all the instructions and downloadables as well as a  
> link to ESOD if we can get round the issues with multicast (which I  
> would have thought we can?).
>
no need to patch the Rabbit/Mule connector, you ahve committer rights  
on the MuleForge project.

> I need to clarify that the demo for RabbitMQ is only text based at  
> the moment, I will switch back to the GUI when I have space to  
> breathe !!!
>
> Kind regards
> Neil
>
>
> On 4 Apr 2008, at 11:56, Alexis Richardson wrote:
>
>> Neil,
>>
>> Please could I post the link for your Newton-Mule-RabbitMQ demo to  
>> the
>> list, plus a description of how to use it?
>>
>> Everyone,
>>
>> We were able to fix a lot of the issues in the RpcClient discussion
>> below.  For Mule users, please see the Mule Forge for any patches.
>> Neil has done a really cool demo that uses Newton to set up a bunch  
>> of
>> Mules and Rabbits performing a distributed workflow, which in the  
>> demo
>> is a news feed and image processor for a notional news aggregator.
>>
>> alexis
>>
>>
>>
>>
>>
>> On Sun, Mar 16, 2008 at 1:43 AM, Neil Ellis  
>> <neil.ellis at mangala.co.uk> wrote:
>>>
>>>
>>> The mule configs are:
>>>
>>>
>>> ------
>>>
>>> <mule-descriptor name="Poller"
>>> implementation="org.mule.components.simple.BridgeComponent">
>>>
>>>           <inbound-router>
>>>               <endpoint
>>> address="http://newsrss.bbc.co.uk/rss/newsonline_uk_edition/front_page/rss.xml 
>>> "
>>>                         connector="pollingHttp">
>>>                   <properties>
>>>                       <property name="pollingFrequency"  
>>> value="10000"/>
>>>                   </properties>
>>>               </endpoint>
>>>           </inbound-router>
>>>
>>>           <outbound-router>
>>>               <router
>>> className="org.mule.routing.outbound.FilteringXmlMessageSplitter">
>>>                   <endpoint address="amqp://news:fanout/?routing_key=news 
>>> "
>>> synchronous="false" transformers="DOM2String">
>>>                       <filter expectedType="org.dom4j.Document"
>>>
>>> className="org.mule.routing.filters.PayloadTypeFilter"/>
>>>                   </endpoint>
>>>                   <reply-to
>>> address="amqp://result:fanout/?routing_key=result"/>
>>>                   <properties>
>>>                       <property name="splitExpression"
>>> value="/rss/channel/item"/>
>>>                       <property name="validateSchema"  
>>> value="false"/>
>>>                   </properties>
>>>               </router>
>>>           </outbound-router>
>>>
>>>       </mule-descriptor>
>>>
>>> -----
>>>
>>>       <mule-descriptor name="Worker"
>>> implementation="org.mule.components.simple.PassThroughComponent">
>>>           <inbound-router>
>>>               <endpoint address="amqp://news:fanout/?queue=news"
>>> transformers="RSSToHeadline" synchronous="false" />
>>>           </inbound-router>
>>>       </mule-descriptor>
>>>
>>> -----
>>>
>>>       <mule-descriptor name="SimpleReceiver"
>>> implementation="org.mule.components.simple.EchoComponent">
>>>           <inbound-router>
>>>               <endpoint address="amqp://result:fanout/? 
>>> queue=result"/>
>>>           </inbound-router>
>>>           <outbound-router>
>>>               <router
>>> className="org.mule.routing.outbound.OutboundPassThroughRouter">
>>>                   <endpoint address="stream://System.out"/>
>>>               </router>
>>>           </outbound-router>
>>>       </mule-descriptor>
>>>
>>> ----
>>>
>>>
>>> The above are on 3 different instances of Mule on the same physical
>>> machine, the RabbitMQ server is running on a seperate VM. The  
>>> exception is
>>> below:
>>>
>>>> [03-16 00:37:00] ERROR DefaultExceptionStrategy  
>>>> [RabbitMQ.dispatcher.15]:
>>> Caught exception in Exception Strategy: Socket closed
>>> java.net.SocketException: Socket closed
>>>       at
>>> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:99)
>>>       at java.net.SocketOutputStream.write(SocketOutputStream.java: 
>>> 115)
>>>       at java.io.DataOutputStream.writeByte(DataOutputStream.java: 
>>> 136)
>>>       at com.rabbitmq.client.impl.Frame.writeTo(Frame.java:189)
>>>       at
>>> com 
>>> .rabbitmq 
>>> .client.impl.SocketFrameHandler.writeFrame(SocketFrameHandler.java: 
>>> 146)
>>>       at
>>> com 
>>> .rabbitmq.client.impl.AMQConnection.writeFrame(AMQConnection.java: 
>>> 295)
>>>       at  
>>> com.rabbitmq.client.impl.AMQCommand.transmit(AMQCommand.java:175)
>>>       at  
>>> com.rabbitmq.client.impl.ChannelN.basicPublish(ChannelN.java:370)
>>>       at  
>>> com.rabbitmq.client.impl.ChannelN.basicPublish(ChannelN.java:351)
>>>       at com.rabbitmq.client.RpcClient.publish(RpcClient.java:163)
>>>       at
>>> com 
>>> .rabbitmq 
>>> .mule.transport.RabbitDispatcher.doDispatch(RabbitDispatcher.java: 
>>> 78)
>>>
>>> I'm taking another look at the code right now to see if I can  
>>> figure out
>>> what's going wrong.
>>>
>>> Kind regards
>>> Neil
>>>
>>>
>>> On 15 Mar 2008, at 18:42, Ross Mason wrote:
>>>
>>>
>>>> Hi Neil,
>>>>
>>>> This does sound interesting.  There is still a fair way to go on  
>>>> the
>>>> Mule Rabbit Transport, but there are some simple use cases that  
>>>> work.
>>>> What is the problem you have with A -> B -> C? is it that the  
>>>> message
>>>> doesn't get to C? If so, Are you trying to perform an RPC call  
>>>> from A?
>>>> (i.e.  RPC from A to get a response from C via B).
>>>>
>>>> BTW Guys, I would like to do a Beta release of the Rabbit Connector
>>>> before MuleCon, but right now I snowed under this weekend.
>>>>
>>>> Cheers,
>>>>
>>>> Ross Mason
>>>> CTO, Co-Founder
>>>> MuleSource Inc.
>>>>
>>>> http://mulesource.com | http://blog.rossmason.com
>>>>
>>>>
>>>> On 15 Mar 2008, at 17:22, Neil Ellis wrote:
>>>>
>>>>
>>>>> Hi Alexis
>>>>>
>>>>> No probs.
>>>>>
>>>>> Alas I don't have the budget for jet-setting :-(
>>>>>
>>>>> I've been having problems with getting the Mule connector to  
>>>>> handle
>>>>> A->B->C messages although A->B seems okay. If anyone from the
>>>>> RabbitMQ side could give me a little hand on getting it up and
>>>>> running that would be fabulous - I'm pretty busy at the moment
>>>>> between this work and project:Einstein/deesel (
>>> http://einstein.codecauldron.org
>>>>> / http://deesel.codecauldron.org ) so any help would increase the
>>>>> chances of having a demo ready for you.
>>>>>
>>>>> I've restructured the current demo to show
>>>>>
>>>>> 1 Polller Mule instance reading from an RSS feed.
>>>>> 3-5 Worker Mule instances transforming the feed (at least a  
>>>>> notional
>>>>> idea of work).
>>>>> 1 Displayer Mule instance displaying the transformed results to  
>>>>> the
>>>>> screen.
>>>>>
>>>>> It's a bit noddy but it shows how Newton can distribute a set of
>>>>> Mule nodes based upon simple criteria, so on a single Newton
>>>>> instance you would have all the Mule instances at the same time on
>>>>> one instance, on 7 instances you would have 1 poller, 3-5 workers
>>>>> and 1 displayer - and in between it scales appropriately. And then
>>>>> shows RabbitMQ as the internode protocol. We could easily turn  
>>>>> this
>>>>> into a demo of running RabbitMQ on a large number of nodes with
>>>>> super-easy deployment and failover in the future - that's  
>>>>> something
>>>>> to think about.
>>>>>
>>>>> We could certainly help with providing materials (i.e. slides or
>>>>> user guide) and help getting you running the demo before hand - in
>>>>> return would you be able to ensure that Paremus/myself get
>>>>> appropriate credit at the demo for our part (i.e. powered by  
>>>>> Newton
>>>>> and the appropriate links)?
>>>>>
>>>>> Anyway let us know - I'm on Skype most of the time - I think you
>>>>> have me as a contact (neilellis).
>>>>>
>>>>> All the best
>>>>> Neil
>>>>>
>>>>>
>>>>> On 15 Mar 2008, at 12:16, Alexis Richardson wrote:
>>>>>
>>>>>
>>>>>> Neil
>>>>>>
>>>>>> Thank-you very much.  That sounds like a really interesting demo.
>>>>>> Please let us know how you get on.  Also - will you be at  
>>>>>> MuleCon?  I
>>>>>> am going and would love to show anything you have to folks at the
>>>>>> conference.
>>>>>>
>>>>>> alexis
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 13, 2008 at 5:48 PM, Neil Ellis
>>>>>> <neil.ellis at mangala.co.uk> wrote:
>>>>>>
>>>>>>> Hi Folks
>>>>>>>
>>>>>>> Not sure if this has already been reported; in version 1.2.0  
>>>>>>> of lib-
>>>>>>> rabbitmq there is a NPE if the publish() method has been  
>>>>>>> called on
>>>>>>> RpcClient because the temp queue picks it up and attempts to  
>>>>>>> call
>>>>>>> setBody() on the blocker. Trivially chaning blocker.set(body) to
>>>>>>> have
>>>>>>> a NP check fixes this:
>>>>>>>
>>>>>>>               if(blocker != null) {
>>>>>>>                       blocker.set(body);
>>>>>>>                }
>>>>>>>
>>>>>>> This was line 150 of RpcClient. I've cc-ed the list for the mule
>>>>>>> transport as it is the mule transport using publish() that  
>>>>>>> caused my
>>>>>>> problem.
>>>>>>>
>>>>>>> Btw. The website for the work I'm doing is
>>> http://mule4newton.codecauldron.org/mule4newton-examples/mule4newton-rabbitmq/index.html
>>>>>>> and the src is at
>>> http://svn.codecauldron.org/mule4newton/trunk/examples/rabbitmq/
>>>>>>> - basically we're combining Newton, Mule and RabbitMQ as an
>>>>>>> illustration of a flexible open-source based messaging system.
>>>>>>>
>>>>>>> Kind regards
>>>>>>> Neil
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> rabbitmq-discuss mailing list
>>>>>>> rabbitmq-discuss at lists.rabbitmq.com
>>>>>>> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Alexis Richardson
>>>>>> +44 20 7617 7339 (UK)
>>>>>> +44 77 9865 2911 (cell)
>>>>>> +1 650 206 2517 (US)
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> rabbitmq-discuss mailing list
>>>> rabbitmq-discuss at lists.rabbitmq.com
>>>> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>>>
>>>
>>>
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-discuss at lists.rabbitmq.com
>>> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>>
>>>
>>
>>
>>
>> -- 
>> Alexis Richardson
>> +44 20 7617 7339 (UK)
>> +44 77 9865 2911 (cell)
>> +1 650 206 2517 (US)
>