Git multi-pack index and reachability bitmaps for repository performance¶

Large Git repositories on CI runners, central mirrors, and developer workstations accumulate dozens or hundreds of packfiles over time. Every push, pull, and CI fetch creates new packfiles. Searching through hundreds of individual packfile indexes slows down commit lookups, tree walks, and object transfers.

The traditional fix is git repack -a -d, which repacks every object into a single monolithic packfile. On repositories with tens of gigabytes of history, a full repack consumes excessive memory, saturates disk I/O, and risks out-of-memory crashes on CI runners.

The multi-pack index (MIDX) feature solves this problem without rewriting existing packfiles. It provides a single consolidated index across all packfiles and enables multi-pack reachability bitmaps for rapid object negotiation.

How the multi-pack index works¶

In a standard repository without a multi-pack index, each packfile in .git/objects/pack/ has a companion .idx file:

.git/objects/pack/
├── pack-1111.pack  <-- pack-1111.idx
├── pack-2222.pack  <-- pack-2222.idx
├── pack-3333.pack  <-- pack-3333.idx
└── pack-4444.pack  <-- pack-4444.idx

When Git needs to find an object by its SHA, it scans each .idx file in order. If fifty packfiles exist, Git performs up to fifty binary searches.

When you create a multi-pack index, Git scans all .idx files and generates a single binary file named multi-pack-index:

.git/objects/pack/
├── multi-pack-index  <-- Unified lookup table
├── pack-1111.pack
├── pack-2222.pack
├── pack-3333.pack
└── pack-4444.pack

The multi-pack-index file contains:

A list of all packfile names included in the index.
An OID fanout table (256 entries) for fast prefix lookup.
A sorted list of all unique object IDs across all indexed packs.
Pointers from each object ID to its specific packfile and byte offset.
Large offset tables for packfiles exceeding 2 GB.

With a MIDX in place, Git finds any object in a single binary search, regardless of how many packfiles exist in the directory.

Core commands¶

Writing a multi-pack index¶

Run the write subcommand to index all current packfiles:

git multi-pack-index write

Git creates or updates .git/objects/pack/multi-pack-index. Existing .idx files remain on disk for compatibility with older Git clients or alternative tools.

Verifying index integrity¶

Run verify to ensure the MIDX matches the underlying packfiles and has no corrupted offsets:

git multi-pack-index verify

Git reads every object in the index, verifies its offset in the referenced packfile, checks data checksums, and ensures object ordering is intact. The command exits with code 0 on success.

Multi-pack reachability bitmaps¶

Reachability bitmaps drastically accelerate Git operations that traverse commit history, including git rev-list, git log, and the object negotiation phase of git push and git fetch.

Historically, Git bitmaps required a single packfile created via git repack -a -b. If you added new commits, you either lived without bitmaps for the new objects or ran another full repack.

The multi-pack index supports reachability bitmaps across multiple packfiles:

git multi-pack-index write --bitmap

This writes two files:

.git/objects/pack/multi-pack-index
.git/objects/pack/multi-pack-index-<checksum>.bitmap

Selecting a preferred pack¶

Bitmaps require a "preferred pack" to establish a stable base order for objects. By default, Git selects the largest packfile. You can specify a preferred pack explicitly:

git multi-pack-index write --bitmap --preferred-pack=pack-1111.pack

Setting your main historical archive pack as the preferred pack yields the best bitmap compression and traversal speeds.

Incremental batch repacking¶

Rather than repacking an entire multi-gigabyte repository in one heavy operation, Git can repack small packs incrementally while leaving large historical packs untouched.

The repack subcommand¶

The repack subcommand identifies packfiles whose total size is bounded by --batch-size:

git multi-pack-index repack --batch-size=64M

Git executes the following steps:

Evaluates all packfiles tracked by the MIDX.
Selects packs whose sizes are smaller than or equal to 64 megabytes.
Repacks the objects from those selected packs into one new packfile.
Updates the MIDX to reference both the old packs and the new packfile.

Expiring redundant packfiles¶

After git multi-pack-index repack completes, the objects exist in both the old small packs and the new combined pack. The old packs are now redundant.

Remove them safely with expire:

git multi-pack-index expire

Git examines each packfile. If every object inside a packfile is now indexed from a newer packfile, Git deletes the redundant .pack, .idx, and .rev files, then rewrites multi-pack-index to drop references to the deleted packs.

Geometric repacking¶

Git 2.31 introduced geometric repacking via git repack --geometric. This algorithm maintains a geometric progression of packfile sizes (such as each pack being at least twice the size of the next smaller pack).

Combine geometric repacking with MIDX generation in one command:

git repack --geometric=2 -d --write-midx

This merges smaller packfiles until the geometric sequence is restored and immediately updates the multi-pack index. It runs quickly on CI runners because it touches only the newest, smallest packs.

Configuration options¶

Configure Git to use and generate multi-pack indexes automatically:

[core]
    # Enabled by default in Git 2.22 and newer
    multiPackIndex = true

[pack]
    # Generate reverse indexes (.rev) for faster pack parsing
    writeReverseIndex = true
    # Include commit hashes in bitmap caches for faster lookups
    writeBitmapHashCache = true

Set these globally or in the repository configuration:

git config core.multiPackIndex true
git config pack.writeReverseIndex true
git config pack.writeBitmapHashCache true

CI runner and server maintenance routine¶

On busy CI runners, reference caches, or mirror repositories, schedule this maintenance sequence to run after major test suites or on an hourly cron job:

#!/usr/bin/env bash
set -euo pipefail

REPO_DIR="/var/cache/git/app.git"
cd "$REPO_DIR"

# 1. Repack small packfiles into a consolidated batch
git multi-pack-index repack --batch-size=128M

# 2. Safely remove old packs whose objects are now consolidated
git multi-pack-index expire

# 3. Refresh the multi-pack index and write reachability bitmaps
git multi-pack-index write --bitmap

# 4. Verify integrity
git multi-pack-index verify

This sequence keeps pack count low, maintains fast object queries, avoids high memory spikes, and never blocks concurrent read operations.