When Linux Runs Out of Memory
Subject:   Parsing /proc/meminfo before 'malloc'ing
Date:   2007-03-22 21:20:36
From:   Unna.KB
Sorry. I came across this article after nearly 4 months since it is written. I found this article very useful. I need a help to improve the memory performance of my embedded system.
I have a memory performance testcase scenario wherein I allocate two chunks of 32 MB , memset chunk1 to all 1s. Then, memcpy chunk1 to chunk2. In this process, i get my running process killed when it is performing memcpy at 27MB. So, as suggested in the article, i tried to parse the /proc/meminfo file for free usable memory before a malloc operation. Usable Memory = MemFree + Cached, as Buffers and SwapFree are not applicable in my case. MemTotal = 71MB.
Before allocation of chunk 1, it showed 60MB of Usable Memory. I allocated chunk 1 using malloc. When i again checked for usable memory for allocating chunk 2 it again showed 60 MB. Should not it alerted me by showing 28 MB, so that i would have averted from allocating the next chunk of 32 MB. Please advice me on how to avoid such memory allocations. Which other field i need to check in the /proc/meminfo.?

I cannot use strict overcommit in my system, as application has been on its final stage of development, tuning to strict overcommit leads to lot of NO MEMORY errors.

Full Threads Newest First

Showing messages 1 through 3 of 3.

  • Re: Parsing /proc/meminfo before 'malloc'ing
    2007-03-24 10:36:40  mulyadi_santosa [View]

    Hi Unna..

    Sorry for this late reply. I am also confused why it can actually report there was still 60 MB of free memory after you do malloc()+memset() the first 32 MB block. The things I can suggest are:
    1. Please make sure you get a valid memory report. There is a chance you are reading not-up-to-date information (kinda delayed).

    2. Find out more about your OS. Is it Linux? BSD? else? Pay attention for things like how they actually do memory allocation. I also forgot to tell the reader one thing (more because I haven't done closer research about it), kernel actually reserves some amount of RAM for special purpose. I don't know the exact amount, so you probably hit this "unseen" area.

    Feel free to reply on this thread... and anyone may CMIIW.


    • Re: Parsing /proc/meminfo before 'malloc'ing
      2008-11-17 22:35:39  dee_ [View]

      Hi Mulyadi,

      Thanks for this great article. It is really helpful.

      I am facing one problem. I want that my application should be able to find two things:
      1) available heap memory
      2) largest block of heap memory that it can allocate.

      Calculating available free pages and available user space will help me how?
      Can you help me in this?


      • Re: Parsing /proc/meminfo before 'malloc'ing
        2008-12-05 20:12:08  santosam [View]

        Hi Deepak

        Sorry, I know no built-in glibc function that is able to do what you ask. Seems like you have to parse /proc/the-pid/maps by yourself and from there determine the largest VMA block you can allocate.

        Pay attention that you could split the size you ask into smaller parts, since in real situation it's a bit hard to find big continous virtual memory area.

        Sorry if it doesn't help you a lot and thanks for reading my article. Glad you find it useful.