[rabbitmq-discuss] Using Message Broker for database replications (currently RabbitMQ )

Mon Feb 10 17:59:55 GMT 2014

Hi,

1. One of the main uses of RabbitMQ is to spread events in a system,
so this seems to fit your use case.

2. When you consume messages from a RabbitMQ queue you can tell the
broker that you want to "acknowledge" each message once you are done
with it. So RabbitMQ will delete the message once the consumer sends
the "ack". So if your consumer crashes before sending the ack, then
the messages will be put back in the queue.

Some notes:

- You can actually send more than one ack at the same time, i.e: ack a
set of messages instead of one by one, so you reduce roundtrips.
- Messages are put back in the queue in order, so you don't lose
ordering here. That is, messages are back in the queue at the same
position where it was before being delivered to the consumer.

3. I'm not sure what you mean here, but RabbitMQ is definitely used in
the way you ask.

Regards,

Alvaro

On Mon, Feb 10, 2014 at 2:19 PM, rails <stammailbox at gmail.com> wrote:
> When my system's data changes I publish every single change to at least 4
> different consumers (around 3000 messages a second) so I want to use a
> message broker.
> Most of the consumers are responsible to update their database tables with
> the change.
>
> (The DBs are different - couch, mysql, etc therefor solutions such as using
> their own replication mechanism or using db triggers is not possible)
> questions
>
> Does anyone have an experience with data replication between DBs using a
> message broker?
> is it a good practice?
>
> What do I do in case of failures?
> Let's say, using RabbitMQ, the client removed 10,000 messages from the
> queue, acked, and threw an exception each time before handling them. Now
> they are lost. Is there a way to go back in the queue?
>
> (re-queueing them will mess their order ).
>
> Is using rabbitMQ a good practice? Isn't the ability to go back in the queue
> as in Kafka important to fail scenarios?
>
> Thanks.
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>