← All essays

· 5 min read

Refs should stop being files

Git 3.0 plans to make reftable the default reference store, demoting the filesystem layout that tools have spent years mistaking for Git's branch API.

An open Library of Congress card-catalog drawer sits on a wooden table, packed with rows of index cards.
Michael Holley (Swtpc6800), Public domain

Git taught everyone a dangerous little trick: a branch looks like a text file. Open .git/refs/heads/main and, in an ordinary repository, you can often read the object ID with cat. A joy to inspect, and a trap: a generation of scripts learned to treat the storage layout as the interface.

The familiar files backend
.git/├── refs/heads/main├── refs/tags/v1.0├── packed-refs└── logs/refs/heads/main

Then git pack-refs moves names out of that directory and into packed-refs, while reflogs live under a parallel tree. The branch stayed the same even as Git moved the drawer. If your tool discovers branches by walking .git/refs, it has already confused a convenient implementation with Git.

Git's current Git 3.0 breaking-change plan says new repositories will eventually use reftable by default. Good. A reference is a named pointer with update rules, and the filesystem has been leaking its own rules into that namespace for long enough.

The namespace already outgrew the directory#

The files backend has to reconcile two representations before it can answer a simple question. Loose refs are individual paths, packed refs are records in one flat file, and deleting a packed ref can require rewriting that file. The Git project's reftable design document uses Android as the uncomfortable end of this curve: one repository with 866,000 references. At that size, a branch list has stopped resembling a cute directory of forty-byte files.

Filesystem semantics leak into names too. Git's breaking-change notes call out case-insensitive filesystems, where two refs that differ only by case cannot coexist with the files backend, and macOS filename normalization creates a similar problem for some Unicode spellings. Reftable stores reference names as records instead of paths, so the host filesystem does not get to reinterpret the namespace on the way to disk.

The tree is lovely to poke at and a bad thing to build on. Git has commands for reading and updating refs precisely because callers should ask Git what refs/heads/main means instead of guessing which file currently happens to contain it.

One transaction should have one visible state#

The nastier limitation is atomicity. The files backend creates separate files when a transaction updates several refs, so a reader can observe an in-between state while those writes land. Git's own 3.0 notes list this as a reason for changing the default. A ref transaction deserves the same boring property you expect from any other database update: readers get the old set or the new set.

Reftable gets there with immutable files and one tiny stack manifest. The official format documentation lays out the write path explicitly:

  1. Lock

    Take tables.list.lock

    One writer chooses the next update index while readers keep their current snapshot.

  2. Write

    Create a new immutable table

    The transaction's ref changes and reflog entries are written together into a temporary reftable.

  3. Publish

    Rename the table into place

    The finished file receives its permanent name before the stack starts referring to it.

  4. Commit

    Replace tables.list

    One rename publishes the new stack, so later readers see the whole transaction at once.

A reftable transaction publishes a new immutable table by replacing the stack manifest only after the table is complete.

Readers open tables.list, open the tables it names, and retry only if one disappeared before it could be opened. Once those files are open, that reader owns a consistent snapshot for as long as it needs it. Deletes become tombstones in a newer table, while geometric compaction folds old tables together later instead of forcing every small update through one giant rewrite.

Prefix-compressed binary records are less charming than a directory you can inspect with ls. Fine. Git can give humans git show-ref, git for-each-ref, git update-ref, and git reflog while giving its storage engine the representation that fits the job. The object database crossed that line long ago. Nobody expects to edit a packfile with a text editor because loose objects were once easy to open.

The compatibility detail I like most is almost comic. Reftable repositories still keep dummy filesystem artifacts so older clients recognize the directory as Git. The format documentation says .git/HEAD should contain ref: refs/heads/.invalid, beside a placeholder .git/refs tree. The files are still there, in other words. They just stopped meaning anything, which is roughly where this was always headed.