[rabbitmq-discuss] dropped messages

[Mark Smith]

Matthias,Attached is a zipped Visual Studio 2008 solution. This exhibits the
problems I am seeing. The code contains a producer.exe and consumer.exe,
both projects share a sharedTypes.dll. This all builds from one solution
once you update the reference to a RabbitMQ .NET Client. The code assumes
that the broker is installed locally. Comments are in program.cs of the
producer.exe project.

Thanks for taking a look.

Mark

On Mon, May 25, 2009 at 4:07 PM, Matthias Radestock <matthias at lshift.net>wrote:

> Mark,
>
> smittycb10 wrote:
>
>> however I still have the same problem 100,000's messages in and only
>> 1,000's of messages out. Am I publishing and subscribing in the most
>> efficent manner?
>>
>
> Can you send through the *complete* code? It's impossible to tell
> otherwise why exactly you are getting the results you are seeing.
>
> How are you measuring the rates? If on the sending side you are just
> timing a loop of publishes then all you are measuring is the time it
> takes to hand over the messages to the kernel. They can get buffered
> there, in the network layer, in multiple layers at the broker, etc, etc.
>
> Also, since AMQP's basic.publish is asynchronous, if the publishing
> connection is not terminated cleanly then some messages will inevitably get
> lost.
>
> It's also worth remembering that RabbitMQ is a message *queuing* system, so
> one shouldn't be surprised if occasionally it queues some messages ;) What's
> more, because RabbitMQ's queuing is quite efficient, when you push the
> system hard, i.e. by feeding it messages as fast as you can, it will handle
> the incoming messages at a faster rate than it sends them to consumers.
>
> For meaningful benchmark results it is therefore important to throttle
> the producer, e.g. send messages at a fixed rate, monitor the rate at
> which they are received by consumers, and slowly increase the producing
> rate until the consumption rate starts falling behind.
>
>  Do I need to anything to broker?
>>>
>>
> Unlikely.
>
>  Is a message rate of 100,000 per second unreasonable?
>>>
>>
> For a single producer->broker->consumer chain that would be quite a high
> rate - 10s of thousands of msgs/s is more realistic on stock hardware.
>
> If you have several independent message streams then higher total
> throughputs can be achieved when running on lots of cores.
>
>  is there a max number of messages any one channel can have queued in
>> it at one time?
>>
>
> No.
>
>  I am currently trying to squeeze all the messages into one channel,
>> should I be splitting work across channels and threads on the
>> receiving side??
>>
>
> Having multiple channels and, even better, multiple connections, provides
> more opportunities for parellelism at the server. Whether or not that
> actually makes a positive difference depends on lots of factors.
>
>
> Regards,
>
> Matthias.
>



-- 
Mark Smith

please send all correspondences to this
new email address
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20090527/fb755cf7/attachment.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rabbitMQ_Producer.zip
Type: application/zip
Size: 21116 bytes
Desc: not available
Url : http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20090527/fb755cf7/attachment.zip