How do I limit the amount of memory that my program uses?

There are several ways to limit the amount of memory that a program uses. It is a good idea to do so if you know that it should never go over a certain value.

Different syntax but same underlying behavior

All of the approaches below use the same underlying kernel functionality.

The different commands and functions on this page all set the maximum size of a process's virtual memory to some size (see RLIMIT_AS in man setrlimit). Note that this is per process. This means that for every process that gets forked, they have the same limit but each process can independently use up to that limit (e.g. 1 process with a 1 GB limit that forks 3 times can use 4 GB between the parent and three children).

bash and tcsh (useful for job submission scripts)

Both bash and tcsh can set the virtual memory usage of themselves and their child processes. The number is specified in kilobytes. This can be put in your batch job submission scripts in a line above where your program gets called.

bash (1 GB in kilobytes):

ulimit -v 1048510

tcsh (1 GB in kilobytes):

limit vmemoryuse 1048510

Example job script:

#!/bin/bash
#PBS -l nodes=1:ppn=1,pmem=2GB,walltime=01:00:00
#PBS -m n

MAXMEM_MB=2048

# The following line changes to the directory that you submit your job from
cd "$PBS_O_WORKDIR"

#set to MAXMEM_MB per process in KB. won't affect processes launched on other nodes (e.g. MPI)
ulimit -v $(($MAXMEM_MB*1024))
/fslhome/myusername/compute/myprogram

C/C++

If you are using C or C++, you can use setrlimit to set a maximum memory size in bytes. The example code below uses an environment variable, MAXMEM_MB to accomplish this. You can then set the maximum amount of memory at runtime using:

env MAXMEM_MB=1024 ./myprogram   # must include code from below

The code to use the MAXMEM_MB environment variable:

#include <sys/time.h>
#include <sys/resource.h>
#include <stdlib.h>

#define MEM_ENV_VAR "MAXMEM_MB"

void setmemlimit();

int main(int argc, char **argv)
{
        // launch MPI here first if you are doing so

        setmemlimit();

        // the rest of the program goes here
        // ...
        // ...

        return 0;
}

void setmemlimit()
{
        struct rlimit memlimit;
        long bytes;

        if(getenv(MEM_ENV_VAR)!=NULL)
        {
                bytes = atol(getenv(MEM_ENV_VAR))*(1024*1024);
                memlimit.rlim_cur = bytes;
                memlimit.rlim_max = bytes;
                setrlimit(RLIMIT_AS, &memlimit);
        }
}