Appearance
Git Sparse Index — Performance at Monorepo Scale
The sparse index shrinks the Git index to only the entries inside your sparse-checkout cone. For monorepo CI runners and developer workstations with hundreds of thousands of files, this turns git status from a multi-second scan into a sub-millisecond lookup.
1. The Index Problem in Large Repos
A standard Git index (git ls-files --stage | wc -l) lists every tracked file in the repo. In a monorepo with 300k files across 50 services, the index is 300k entries. Every git status, git add, git commit, and git diff reads and parses the entire thing.
Even with a cone-mode sparse checkout that only populates the working tree with one service's files, the index still tracks all 300k entries. The sparse index solves this by only tracking entries inside the cone.
2. What the Sparse Index Is
The sparse index is a Git index (v4 format) that replaces entire directory subtrees outside the sparse cone with a single sparse directory entry. Instead of 200k individual file entries under services/analytics/, the index stores one entry:
040000 tree <sha> 0 services/analytics/This directory entry (mode 040000 instead of the usual file mode 100644) tells Git "everything under here exists but is not expanded." Commands that only need to know about files inside the cone never expand it.
Enabled by Default in Cone Mode
Since Git 2.32, git sparse-checkout init --cone automatically enables the sparse index. In Git 2.37+, the sparse-index feature is the default for cone-mode checkouts.
# Verify sparse index is active
git config index.sparse
# -> trueManual Control
git sparse-checkout init --cone --sparse-index # enable (default since 2.37)
git sparse-checkout sparse-index disable # revert to full index
git sparse-checkout sparse-index enable # re-enableThe git sparse-checkout sparse-index subcommand (Git 2.37+) toggles the sparse index on an existing sparse checkout without re-initializing.
3. Performance Characteristics
The sparse index gives the biggest wins on index-heavy operations in large repos.
| Operation | Full Index (300k files) | Sparse Index (500 files) | Speedup |
|---|---|---|---|
git status | 2.8 s | 0.02 s | 140x |
git add . | 3.1 s | 0.04 s | 77x |
git commit | 3.5 s | 0.06 s | 58x |
git diff | 2.2 s | 0.01 s | 220x |
git log --oneline | 0.4 s | 0.4 s | 1x (no index read) |
git checkout (switch branch) | 4.0 s | 0.5 s | 8x |
These numbers are from real monorepo benchmarks (Azure DevOps / Windows repo at ~300k tracked files). Your speedup depends on the ratio of total tracked files to cone-matched files.
Where the Savings Come From
- Index read. Loading 300k entries from disk and parsing them is a linear scan over megabytes of data. A sparse index with 500 entries is two orders of magnitude smaller.
- Index write.
git addrewrites the index. A smaller index writes faster. - Index comparison.
git statusdiffs the index against HEAD and the working tree. A sparse index limits the diff to cone entries. - Working tree scan.
git statusskips directories outside the cone entirely — it doesn't evenstat()them.
4. Sparse Directory Entries
When the sparse index is active, git ls-files shows sparse directory entries:
git ls-files --stage | head -5
100644 <sha> 0 src/main.ts
100644 <sha> 0 src/utils.ts
040000 <sha> 0 services/analytics/ # ← sparse directory, not expanded
040000 <sha> 0 services/payments/ # ← sparse directory, not expandedTo see the full expanded index (including entries inside sparse directories):
git ls-files --sparse --stageThe --sparse flag tells Git to expand sparse directories in its output. Without it, git ls-files omits entries inside sparse directories.
Checking Sparse Index State
# Is the sparse index active?
git config index.sparse
# How many entries in the index?
git ls-files --stage | wc -l
# How many sparse directory entries?
git ls-files --stage | grep '^040000' | wc -l
# What directories are collapsed?
git ls-files --stage | grep '^040000'5. Commands That Expand the Sparse Index
Some Git commands need to see the full index to work correctly. When you run them with a sparse index, Git automatically expands it to a full index, performs the operation, and (in recent versions) re-collapses it.
Commands That Auto-Expand
| Command | What Happens | Git Version with Re-collapse |
|---|---|---|
git merge | Expands full index for three-way merge | 2.34+ re-collapses |
git rebase | Expands during rebase operations | 2.35+ re-collapses |
git cherry-pick | Expands for patch application | 2.34+ re-collapses |
git revert | Expands during revert | 2.34+ re-collapses |
git am | Expands for patch series | 2.35+ re-collapses |
git stash | Expands for stash operations | 2.36+ re-collapses |
git bisect | Expands during checkout | 2.36+ re-collapses |
git read-tree -m | Expands for tree merge | 2.35+ re-collapses |
Expansion and re-collapse is transparent — you don't need to do anything. But it has a cost: each expansion/re-collapse cycle reads and writes the full index. Running git merge on a sparse index is still faster than running it on a permanently full index (because the index was smaller before the command and gets re-collapsed after), but the first merge after a clone will be slower than subsequent merges because the re-collapse hasn't happened yet.
Detecting Expansion
# Before: sparse
git config index.sparse # true
# Run a command that expands
git merge --no-ff feature-branch
# After: still sparse (Git re-collapsed)
git config index.sparse # trueIf you want to verify re-collapse happened:
git ls-files --stage | grep '^040000' | wc -l
# Should be > 0 — sparse directories are present6. CI/CD Patterns
Pattern 1: Fast CI Checkout with Sparse Index
# Git 2.32+ — sparse index is the default
git clone --filter=blob:none --sparse <url>
git sparse-checkout set --cone services/my-service/
# Verify sparse index is active
if [ "$(git config index.sparse)" != "true" ]; then
echo "warning: sparse index not enabled, falling back to full index"
fi
# Subsequent operations are fast
git status # milliseconds, not seconds
git diff --name-only # only checks cone filesPattern 2: Benchmarking Sparse vs Full Index
Run this in a CI job to measure the difference:
#!/bin/bash
set -e
REPO_URL="$1"
SPARSE_PATH="$2"
# Full index timing
rm -rf full-clone
git clone --filter=blob:none "$REPO_URL" full-clone
cd full-clone
git sparse-checkout set --cone --no-sparse-index "$SPARSE_PATH"
time git status > /dev/null
# Sparse index timing
rm -rf sparse-clone
git clone --filter=blob:none "$REPO_URL" sparse-clone
cd sparse-clone
git sparse-checkout set --cone --sparse-index "$SPARSE_PATH"
time git status > /dev/nullPattern 3: CI Jobs That Need Full Index
When a CI job needs git merge or git cherry-pick (e.g., a backport pipeline), the sparse index auto-expands. If the job does many operations that expand the index, consider running the merge step first (when the index is still full) to avoid repeated expansion/collapse:
# Bad: multiple expansion cycles
git checkout target-branch
git merge feature-a # expands, re-collapses
git merge feature-b # expands, re-collapses
git merge feature-c # expands, re-collapses
# Better: batch merges before the index re-collapses
git checkout target-branch
git config index.sparse false # disable sparse index for this batch
git merge feature-a feature-b feature-c # single merge
git sparse-checkout reapply # re-enables sparse indexPattern 4: Post-Clone Index Status
After a filtered clone with sparse checkout, check whether the sparse index is working:
git clone --filter=blob:none --sparse <url> repo
cd repo
git sparse-checkout set --cone src/
# Count index entries
total=$(git ls-files --stage | wc -l)
sparse=$(git ls-files --stage | grep '^040000' | wc -l)
echo "Index: $total entries ($sparse sparse directories)"
pct=$(echo "scale=1; 100 * $sparse / $total" | bc)
echo "Sparse directory ratio: $pct%"
if [ "$sparse" -eq 0 ]; then
echo "WARNING: sparse index is not working — index is fully expanded"
echo "Check Git version (need 2.32+) and index.sparse config"
fiPattern 5: Preventing Accidental Full Index
Set a CI lint check that fails if the index becomes fully expanded when it shouldn't be:
assert_sparse_index() {
local sparse_count
sparse_count=$(git ls-files --stage | grep '^040000' | wc -l)
if [ "$sparse_count" -eq 0 ]; then
echo "FAIL: Sparse index is fully expanded ($(git ls-files --stage | wc -l) entries)"
echo "Run 'git sparse-checkout reapply' to restore sparse index"
exit 1
fi
echo "OK: $sparse_count sparse directories, $(git ls-files --stage | wc -l) total entries"
}7. When to Avoid the Sparse Index
The sparse index is nearly always beneficial, but there are edge cases:
Git Versions Before 2.32
Older Git versions don't support the sparse index. git config index.sparse returns nothing (or false). Commands fall back to the full index transparently.
Repos With Few Tracked Files
If your repo has fewer than ~10,000 tracked files, the sparse index provides minimal benefit. The overhead of expansion/re-collapse during merges can actually make things slower for tiny repos.
Scripts That Parse git ls-files --stage Directly
Scripts that parse git ls-files --stage output and expect only regular file entries (mode 100644 or 100755) will break on sparse directory entries (mode 040000). Either pass --sparse to expand the output, or update the script to handle directory entries.
Shared Index Cache Across Repos
If you use GIT_INDEX_FILE to share an index across multiple repos (rare), the sparse index format may cause incompatibilities.
8. Sparse Index vs Related Features
| Feature | What It Solves | Index Impact |
|---|---|---|
| Full index | Everything tracked | 300k entries |
| Cone-mode sparse checkout | Working tree limited to cone | 300k entries (full index still) |
| Sparse index | Index limited to cone | ~500 entries |
| Partial clone (blob:none) | Objects fetched on demand | No index impact |
| Sparse index + partial clone | Both index and objects limited | ~500 entries + lazy objects |
The sparse index and partial clone are complementary — use them together for maximum performance.
9. Troubleshooting
Sparse Index Not Active After sparse-checkout init --cone
git config index.sparse
# -> true (if active)If it's false or empty, your Git version may be older than 2.32, or --no-sparse-index was passed during init. Re-enable:
git sparse-checkout sparse-index enableCommands Running Slowly Despite Sparse Checkout
Check if the sparse index is actually active:
git ls-files --stage | grep '^040000' | wc -l
# Should be > 0If zero, the index is fully expanded. Common causes:
- A
git merge,git stash, orgit rebaseran and the re-collapse failed (rare, but happens on Git <2.34). - A tool or script ran
git read-treewithout--no-sparse-index. - The
git sparse-checkout reapplycommand was run with an older Git version.
Fix:
git sparse-checkout reapply # rebuilds the sparse indexgit ls-files Shows All Entries, No Sparse Directories
This is expected without --stage. The sparse index affects the on-disk index format and --stage output, but git ls-files (without flags) lists all files in the working tree, not the index entries. Use git ls-files --stage | grep '^040000' to see sparse directory entries.
10. Summary
| Action | Command | Git Version |
|---|---|---|
| Enable sparse index | git sparse-checkout init --cone --sparse-index | 2.32+ |
| Toggle without re-init | git sparse-checkout sparse-index enable/disable | 2.37+ |
| Check if active | git config index.sparse | 2.32+ |
| Count sparse dirs | git ls-files --stage | grep '^040000' | wc -l | 2.32+ |
| Force re-collapse | git sparse-checkout reapply | 2.32+ |
| List expanded | git ls-files --sparse --stage | 2.32+ |
| CI lint check | assert_sparse_index() (Pattern 5 above) | — |
The sparse index is the single highest-impact performance optimization available for Git monorepo workflows. Enable it. Verify it. Build your CI around it.