[rabbitmq-discuss] Use a mysql databse table as the provider for rabbitmq queue

[Tim Watson]

On 10/17/2012 11:40 AM, Ryan R. wrote:
> I think I understand what you mean with the shared library.
> However, in my case, RabbitMQ would only be installed if need be 
> (meaning more than one of the apps are present, and two of those need 
> to be synchronised for part of their data).
>

That actually complicates the picture somewhat - is there a reason why 
this is the case? In a typical integration architecture, the messaging 
broker is deployed centrally (on the LAN somewhere) and clients choose 
whether or not they want to connect to it from whatever machine they're 
running on. To me, it is sounding like you're describing an architecture 
where both applications reside on the same machine and assuming that the 
broker will also need to be co-resident with them, which is not really 
the case, though there's nothing to prohibit that either.

> That said, using a shared library would require me to "include/import" 
> said library when I need to, therefore making me change my app code 
> depending of the situation I'm in.
>

Well yes, if you're going to add messaging capabilities to your 
applications that don't currently support it, then you are going to have 
to write *some* code and integrate it into them! :)

> And said library would only be required when there's a RabbitMQ 
> available anyway.
>

I think you're making your life more complicated than it needs to be by 
thinking about whether the messaging broker is available vs. not. The 
broker should *always* be available when applications residing on 
different machines need to communicate with one another, regardless of 
whether those applications are running or not. Again, it feels like 
you're trying to deal with applications running on the same machine - 
have I picked that up correctly? It might help if you explained your 
architecture in a bit more detail, so I can understand exactly what 
you're trying to achieve.

> Now a bit further in your message you talk about a listener library.
> I'd like to know a bit more about this.
> How would an external library be able to listen to anything happening 
> within my app ?
> Would it be listening on the DB queries ?
>

No, not at all. Let's say you've got two applications, App1 and App2. 
You'll write some library code that both applications share, that 
probably looks something like this (with *wide* variations depending on 
language/platform - I've just written pseudo code to keep things simple):

-------------------------------------

function init = do
     read_config_file_for_this_app
     open_connection_to_broker
     store_connection_somewhere_in_memory
end

function listen = do
     get_connection_from_memory
     read_message_from_broker
     pass_message_to_application_thread_somehow
     listen
end

function publish = do
     get_connection_from_memory
     send_message_to_broker
end

-------------------------------------

Now in your applications, you'll call the shared 'init' library function 
when you're starting up to bootstrap the connection to the broker. When 
your application is publishing data, it calls publish and if/when you 
need to subscribe to data then you'll call 'listen'. The fact is that 
'how to listen' for incoming messages really depends on how you're going 
to use them. But the point is that the applications read from and write 
to the messaging broker, and do so independently of database tables. You 
*may* decide to do something like write a middle-man application that 
periodically reads a database table and publishes each row to the 
messaging broker so it can be read from a queue, or do that with a 
worker thread instead of a separate application. I would *not* do 
anything here with the database though. If applications need to share 
data, then **they should send it to one another via message queues.** If 
they need to persist data, they should persist their own data in their 
own tables in the database, but they should **not use the database to 
communicate with one another.** That is the key thing with using 
messaging instead shared data(bases).

There is an overhead in sending (and in some cases, duplicating) data 
between applications of course. This is *more* than compensated for by 
the reduced coupling that comes from integrating using messaging 
technology. This approach may not be suited to integrating applications 
that are running on the same physical machine and are tightly and 
deliberately coupled however. I can't really elaborate on the 
suitability of messaging for your project without understanding a good 
deal more about it I'm afraid.

I hope that clears a few things up at least! :)

Cheers,
Tim