Linux Memory Overcommit: Why malloc Never Fails and the OOM Killer Does

On Linux, malloc() almost never returns NULL. You can allocate 50 GB on a 4 GB machine and get a valid pointer. The failure comes later, when you actually touch the memory and there's nothing to back it — and then the OOM killer picks a process to kill. That's memory overcommit. (The Linux OOM Killer)

Why Linux overcommits

Allocating memory and using it are different things:

• Allocation (malloc, mmap) reserves virtual address space. No physical page is assigned yet.
• Use happens on first write: a page fault, and the kernel assigns a physical page then.

Lots of software allocates far more than it touches: sparse arrays, large thread stacks, runtimes reserving heap up front, and — most importantly — fork(). A fork duplicates the parent's whole address space, but with copy-on-write: pages are shared until one side writes. A 10 GB process forking to run a tiny shell command "needs" 20 GB of virtual memory but only a few extra pages in practice. Without overcommit, that fork would fail.

The three modes

Controlled by vm.overcommit_memory:

Value	Name	Behaviour
0	Heuristic (default)	Refuses only allocations that are obviously impossible; otherwise allows overcommit
1	Always	Skips the overcommit check; allocations can still fail for other resource limits
2	Strict / never	Refuses allocations beyond a fixed commit limit

cat /proc/sys/vm/overcommit_memory
sudo sysctl vm.overcommit_memory=2

CommitLimit and Committed_AS

In mode 2, the kernel tracks committed memory — what processes could touch:

grep -i commit /proc/meminfo
# CommitLimit:     6123456 kB
# Committed_AS:    4321098 kB

• Committed_AS: total memory currently promised.
• CommitLimit = swap + RAM × vm.overcommit_ratio/100 (default ratio 50), or set absolutely with vm.overcommit_kbytes.

In mode 2, an allocation that would push Committed_AS past CommitLimit fails immediately with ENOMEM, which the program can handle — instead of something being killed later. Committed_AS is still worth watching in other modes as a measure of how big the bet is.

Note the default ratio of 50% means only half of RAM is committable without swap — set the ratio (or kbytes) deliberately if you use mode 2.

Why Redis wants mode 1

Redis persists data by fork()ing a child that writes a snapshot while the parent keeps serving. With mode 0 on a busy box, the kernel's heuristic can refuse that fork of a large process, and the background save fails. Redis's documentation therefore recommends vm.overcommit_memory = 1, and warns at startup if it isn't set. (Redis: When You Need It)

Why Postgres prefers mode 2 (on dedicated servers)

Postgres uses a process per connection plus shared memory. If the OOM killer picks the postmaster or a backend holding shared state, every connection is dropped and the server goes through crash recovery. The Postgres documentation suggests strict overcommit (vm.overcommit_memory = 2, with a carefully chosen ratio) on dedicated database servers, so a big query gets an allocation error instead of the whole instance being killed. It also documents how to make the postmaster less likely to be chosen by the OOM killer. (Postgres Memory Tuning)

So Redis and Postgres on the same host may want different policies. Separate VMs can choose their own settings; cgroups limit each service's memory but don't give containers independent overcommit policies. (Linux overcommit documentation)

Containers and cgroups

Overcommit settings are host-wide; containers can't set their own. What containers get instead are cgroup memory limits (memory.max). When a cgroup exceeds its limit, the kernel reclaims memory and, failing that, OOM-kills a process inside that cgroup — the familiar exit code 137. The allocation itself still succeeds under overcommit; the kill comes at use time. memory.high adds throttling before the hard limit. (Container CPU and Memory Limits, Linux Pressure Stall Information)

Practical guidance

• General-purpose app servers: leave mode 0, set cgroup/systemd MemoryMax per service, and monitor memory pressure.
• Redis hosts: mode 1, with enough RAM headroom for copy-on-write during saves.
• Dedicated Postgres servers: consider mode 2 with a ratio sized to RAM + swap, after testing.
• Everywhere: a small amount of swap gives the kernel room to move cold pages instead of killing things, and alerting on available memory beats discovering the OOM killer in dmesg. (Check Disk Space and Memory on Linux)

The summary

• Overcommit lets allocations exceed physical memory; failures move from malloc to the OOM killer.
• Mode 0 = heuristic, 1 = always, 2 = strict with CommitLimit.
• Redis wants 1 (for fork), dedicated Postgres prefers 2 (to avoid OOM kills); containers rely on cgroup limits.

---

EasySpawn gives each server its own VM with memory sized for your app and database, so your workloads don't compete with anyone else's — and Claude Code can investigate when memory runs tight. See how it works or join the waitlist.

Related: Linux Namespaces Explained · Node.js Memory Leaks · JavaScript Heap Out of Memory · Containers vs Virtual Machines