[rabbitmq-discuss] Rabbitmq File descriptor , socket descritor and memory

[sagu prf]

Hi Team ,


We are  publishing non-persistent messages to rabbitmq whether these
paged messages are written to disk (mnesia db ) or it should use swap
space .


it seems rabbitmq not using Swap space ?

rabbitmq at rabbitmq1$ free -m
             total       used       free     shared    buffers     cached
Mem:         24576      12175      12400          0        735       1252
-/+ buffers/cache:      10187      14388
Swap:        12255          7      12248
rabbitmq at rabbitmq1$


Memory 7.8GB  - backlog messages

Paging(?) .

 4,537,909 / 5,056,437 msg (in RAM / total)
RAM target: 9,158,043 msg


It seems the rabbitmq paged out  (non-persistent messages ) to  mnesia database)

mnesia/rabbit at rabbitmq1$ du -sh *

4.0K cluster_nodes.config

4.0K DECISION_TAB.LOG

4.0K LATEST.LOG

38M msg_store_persistent

428M msg_store_transient

4.0K nodes_running_at_shutdown

8.1M queues

68K rabbit_durable_exchange.DCD

12K rabbit_durable_exchange.DCL

252K rabbit_durable_queue.DCD

72K rabbit_durable_queue.DCL

352K rabbit_durable_route.DCD

20K rabbit_durable_route.DCL

4.0K rabbit_runtime_parameters.DCD

4.0K rabbit_serial

4.0K rabbit_user.DCD

8.0K rabbit_user_permission.DCD

4.0K rabbit_vhost.DCD

28K schema.DAT

4.0K schema_version

Regards
Sagu

On Wed, Jul 2, 2014 at 8:59 AM, Michael Klishin <mklishin at gopivotal.com> wrote:
> On 2 July 2014 at 19:53:52, sagu prf (sagu.prf1 at gmail.com) wrote:
>> > I have increased the file descriptor value for the rabbitmq
>> users to
>> 100K and now it showing socket decriptor value to 89908 .
>>
>>
>> So now we can connect maximum 89908 TCP connection (amqp) rights
>> ?
>
> File descriptors cover both file handles and sockets. RabbitMQ tries
> to reuse file handles where possible to conserve them. With the file descriptor
> limit of 100K you won't be able to have 100K open connections because RabbitMQ
> will use at least some of them for open files (the more queues you have, the greater
> the number).
>
>
>> 2. Rabbitmq paging ..
>>
>>
>> {vm_memory_high_watermark,0.4}, - 100GB RAM - high water mark
>> 40GB
>>
>> {vm_memory_high_watermark_paging_ratio,0.5}] rabbitmq
>> pagingout the
>> messages will start at 20GB Physical RAM utilization right ?
>
> From the example config [1]:
>
>    %% Fraction of the high watermark limit at which queues start to
>    %% page message out to disc in order to free up memory.
>    %%
>    %% {vm_memory_high_watermark_paging_ratio, 0.5},
>
> 1. http://hg.rabbitmq.com/rabbitmq-server/file/27d5539e2771/docs/rabbitmq.config.example
> --
> MK
>
> Staff Software Engineer, Pivotal/RabbitMQ