fsync Explained: When Is Data Actually on Disk?

When write() returns success, your data is almost certainly not on disk. If the machine loses power a moment later, it may be gone. The system call that closes that gap is fsync() — and getting it right is the foundation of every database's durability guarantee.

The layers between write() and the platter

1. Your process's buffers (e.g. a userspace I/O library) — flush() moves data to the kernel.
2. The kernel page cache — write() copies data here and returns. The kernel writes it out later (typically within ~30 seconds) at its own pace.
3. The device's volatile write cache — the SSD or controller may acknowledge writes held in RAM.
4. Stable storage — NAND or platters.

A crash loses whatever hasn't reached layer 4.

What fsync does

fsync(fd) asks the kernel to write all dirty data and metadata for that file to the device and to issue a cache flush so the device commits it to stable storage. It returns only when that's done (or failed).

• fdatasync(fd) skips metadata that isn't needed to read the data back (like modification time) — often noticeably cheaper for append-heavy logs.
• O_SYNC / O_DSYNC open flags make every write() behave as if followed by fsync/fdatasync.
• O_DIRECT bypasses the page cache, but doesn't by itself guarantee durability — you still need a flush for the device cache.
• sync() flushes everything, system-wide; too blunt for applications.

The step everyone forgets: fsync the directory

Creating, renaming or deleting a file changes the directory, which is its own piece of metadata. After creating a new file and fsyncing it, a crash can still lose the directory entry — the data exists but no name points to it. To make a new file durable, fsync the file and its parent directory.

The atomic write pattern

To replace a config or state file so readers see either the old version or the new one — never a half-written mix:

import os

def atomic_write(path, data: bytes):
    tmp = path + ".tmp"
    with open(tmp, "wb") as f:
        f.write(data)
        f.flush()
        os.fsync(f.fileno())          # 1. data durable in the temp file
    os.replace(tmp, path)             # 2. atomic rename over the old file
    dir_fd = os.open(os.path.dirname(path) or ".", os.O_RDONLY)
    try:
        os.fsync(dir_fd)              # 3. the rename itself durable
    finally:
        os.close(dir_fd)

Skip step 1 and some filesystems can leave you with an empty or zero-filled file after a crash. Skip step 3 and the rename may be undone.

This Linux example assumes one writer and a trusted directory. For concurrent writers, create a unique temporary file in the same directory with tempfile.mkstemp, rather than sharing path + ".tmp"; otherwise writers can overwrite each other's temporary data. Clean up that temporary file on failure, and preserve the destination's intended permissions when replacing it.

fsyncgate: what happens when fsync fails

In 2018 the PostgreSQL community discovered that on Linux, when writeback of a dirty page fails (say, an I/O error), the kernel reports the error to the next fsync — and then clears it, and may mark the pages clean. Postgres would retry fsync, get success, and believe the data was safe when it had been lost.

The fix on the Postgres side: an fsync failure is now treated as unrecoverable — Postgres PANICs and recovers from the WAL, rather than retrying (data_sync_retry = off by default). The broader lesson: a failed fsync means data may be gone; you can't retry your way to safety. (The Postgres WAL Explained)

How databases trade durability for speed

fsync is slow — often milliseconds per call, the dominant cost of a small transaction. Databases offer knobs:

• Postgres synchronous_commit = off: commits return before the WAL is flushed. A crash can lose the last fraction of a second of transactions, but never corrupts the database. Reasonable for low-value writes; settable per transaction. Group commit (several transactions share one flush) helps throughput by default.
• Postgres fsync = off: never do this outside throwaway environments — a crash can corrupt the cluster.
• SQLite synchronous = NORMAL in WAL mode: durable on application crash, may lose recent commits on power loss, no corruption. (SQLite WAL Mode)
• Redis appendfsync everysec: up to a second of writes at risk.

Hardware and virtualisation

• Enterprise SSDs with power-loss protection can acknowledge flushes quickly and safely; consumer drives may be slow or, rarely, lie about flushes.
• Virtual disks: a VM's fsync is only as good as the hypervisor's and storage backend's handling of flushes. Cloud block storage generally honours them; network-backed disks add latency to every fsync. (How KVM Virtualization Works)
• Measure: pg_test_fsync (ships with Postgres) shows how many syncs per second your disk sustains.

The summary

• write() puts data in the page cache; only fsync/fdatasync (plus a device flush) makes it durable.
• New files and renames need the directory fsynced too; use write–fsync–rename–fsync-dir for atomic replacement.
• A failed fsync means possible data loss — don't just retry; databases expose explicit durability/speed trade-offs.

---

EasySpawn runs your database on a server with local NVMe storage and daily backups, so durability doesn't depend on a single disk flush — and Claude Code can benchmark your storage with pg_test_fsync. See how it works or join the waitlist.

Related: Database Transactions Explained · Postgres Point-in-Time Recovery · io_uring and Security · Encryption at Rest vs in Transit