[rabbitmq-discuss] Persisting Data

[Varun Samuel]

Tony,

I posted a little while back about persisting data with durable  
queues, which was giving me some problems.
I've got it working now, and I think the basic problem lay in my  
approach.

I wrote a small program to create and send persistent messages to a  
durable queue. Once it had sent the
messages it would try to read them immediately using basicGet. This  
seems to be the problem. Once I had a separate
client try to read the messages, everything was fine. But one program  
sending messages to the queue then
immediately trying to read them doesn't work. Just thought I'd post  
in case anyone else was trying the same thing.
Thanks very much for your offer to read over my code though. If you'd  
still like to see it, I can send it. Sorry it took a while
to post the resolution of my problem- me and my partner had to change  
direction a bit for our project.

With regards to the other issue- reading the last n messages in a  
queue, you wrote:

---

I see - it looks like you're interested in something more like a "last
value cache" than a queue.

The way durable queues are designed in AMQP is simply that the messages
in the queues are stored to disk, and so survive server restarts. They
are still consumed in the same way as non-persistent/non-durable
messages: that is, for each *queue*, a single message is delivered to a
single consumer, and if that consumer acknowledges the message, it will
not be delivered to other consumers.

To simulate a last-value cache, you could write a simple "replay"
service that keeps a log of recent chat messages, and sends them out
when asked by a new chat participant.

---

I wasn't sure if you meant this was done inside rabbitmq. I just  
settled on sticking every sent message in a LIFO data
structure, outside of rabbitmq. Any client connecting to rabbitmq  
would first get the messages from that

This is a very basic way to do it. I read a bit of the AMQP spec and  
didn't see anything about doing it in there. Does the
method you describe make use of rabbitmq to log recent chat messages?

Thanks, and Happy Holidays!
Varun





> Hi Varun,
>
> Varun Samuel wrote:
>> Sorry, I'm a little new to this mailing list thing. I posted
>> previously about durable queues, and I will
>> try to make my problem more precise.
>
> Sorry for the delay in responding. I got half-way through replying to
> your other mail on Friday and arrived this morning to find an open,
> half-full mail composition window :-)
>
>> I create a durable exchange/queue and send the queue persistent
>> messages. Then I restart my broker. How would I retrieve
>> those messages? I am trying to read the messages with
>> channel.basicGet, but they aren't in the queue.
>
> This is very strange - strange enough for me to double-check that the
> scenario works for me.
>
>  1. create "durable" queue
>  2. post messages to it with deliveryMode = 2
>  3. restart broker
>  4. consume or basicGet from queue
>  5. messages (should, and do for me) arrive
>
> If that basic scenario isn't working for you, there's something  
> wrong -
> perhaps you could email your code to me privately? I'd be happy to  
> cast
> an eye over it.
>
> I'll reply to your other email separately - please do feel free to
> forward my reply to the mailing list if you feel that's appropriate.
>
> Regards,
>   Tony
> -- 
>  [][][] Tony Garnock-Jones     | Mob: +44 (0)7905 974 211
>    [][] LShift Ltd             | Tel: +44 (0)20 7729 7060
>  []  [] http://www.lshift.net/ | Email: tonyg at lshift.net