Git Reflog Forensics for CI Incident Triage

Authoritative guide to recovering from catastrophic Git operations using the reflog — Git's internal event log — and git fsck --lost-found. This is distinct from the basic reflog rescue guide in docs/troubleshooting/; this document covers forensic analysis, automation, and cleanup for CI pipelines.

Table of Contents

1. How the Reflog Works
2. Recovering Commits After a Bad Reset
3. Recovering Commits After a Forced Rebase
4. Reflog vs Dangling Objects: What fsck --lost-found Finds
5. CI Pipeline Incident Triage Playbook
6. The reflog expire --expire=now Cleanup Lever
7. Automating Reflog Recovery in CI
8. Reflog Pitfalls and Gotchas

---

1. How the Reflog Works

Every time a ref (branch, HEAD, stash, etc.) moves, Git appends a timestamped entry to the reflog for that ref:

HEAD@{2026-08-29 14:23:01}  checkout: moving from main to feat/x
HEAD@{2026-08-29 14:23:45}  commit: feat: add telemetry endpoint
HEAD@{2026-08-29 14:24:10}  reset: moving to HEAD~1
HEAD@{2026-08-29 14:24:10}  commit (amend): feat: add telemetry endpoint

The reflog lives in .git/logs/ and is per-reference: - .git/logs/HEAD — every HEAD movement - .git/logs/refs/heads/main — every branch tip movement - .git/logs/refs/stash — stash operations - .git/logs/refs/original/ — pre-filter-repo backup (if enabled)

Key insight: Reflog entries are local. Cloned repositories do not share reflogs. If a developer pushes after a destructive operation, collaborators who pulled will have different reflog entries.

Reflog entries expire on a schedule (gc.reflogExpireUnreachable, default 90 days). Entries for reachable commits expire at 90 days by default; unreachable (dangling) entries expire at 30 days.

---

2. Recovering Commits After a Bad Reset

Scenario

A developer ran git reset --hard HEAD~5 to remove 5 commits, then continued working. The commits are gone from git log but not from Git's object store.

Step 1: Find the Lost Commits in Reflog

# Show HEAD reflog with timestamps and SHA-1s
git reflog --date=iso

# Filter to find the lost commits by commit message keyword
git reflog | grep -i "feat: telemetry"

# Find commits reachable from the reset target but now dangling
git reflog show HEAD@{1}   # the commit before the reset
git reflog show HEAD@{2}   # two operations ago

Step 2: Identify the Target SHA

git reflog --all --format="%H %s" | grep "reset: moving to"
# Output: abc1234 reset: moving to HEAD~5
#         def5678 reset: moving to abc1234

Step 3: Verify the Commit Is Dangling

git fsck --unreachable --no-reflogs 2>&1 | grep "unreachable commit"
# or with verbose output
git fsck --unreachable --no-reflogs --full 2>&1

Step 4: Recover

# Fastest: reset back to the pre-reset state
git reset --hard HEAD@{1}

# Selective: cherry-pick specific commits by SHA
git cherry-pick abc1234 def5678

# Or branch from a dangling commit
git branch recovery-branch abc1234
git checkout recovery-branch

Validation

# Confirm the branch matches the reflog entry
git log --oneline recovery-branch -10
git log --oneline HEAD@{1} -10  # compare

# Run CI checks before merging recovery branch
git push origin recovery-branch

---

3. Recovering Commits After a Forced Rebase

Scenario

A force-push (git push --force) or an interactive rebase (git rebase -i) rewrote 12 commits. The remote is updated; collaborators who haven't pulled have divergent history.

Step 1: Check Local and Remote-Side Reflog

# Check the remote reflog (requires access to the Git server's .git directory)
# For Forgejo/Gitea: check .git/logs/refs/heads/ on the server

# Check local origin/BRANCH reflog before next fetch
git reflog show origin/main --date=iso

Step 2: Recover from Local Reflog

# List all HEAD movements — look for the pre-rebase state
git reflog --all --format="%H %gd %s" | head -30

# The old commit SHAs are still valid objects until gc runs
git fsck --unreachable --no-reflogs --commits

# Cherry-pick the range that was dropped
OLD_BASE=$(git rev-parse HEAD@{10})   # adjust index
NEW_BASE=$(git rev-parse HEAD)          # current HEAD
git cherry-pick ${OLD_BASE}..HEAD@{8}  # pick dropped commits

Step 3: Remote-Side Recovery (Server-Side)

If you have SSH access to the Git server:

# On the server (Forgejo/Gitea bare repo)
ssh git-server "cd /data/git/repos/owner/repo.git && \
  git reflog show HEAD --date=iso | tail -20"

# Identify the pre-force-push SHA
# Reset the remote branch back (requires direct server write access)
ssh git-server "cd /data/git/repos/owner/repo.git && \
  git update-ref refs/heads/main abc1234def"

Step 4: Re-Establish Correct Remote State

# After recovering locally, force-push the correct history
git push --force-with-lease origin main

# Notify collaborators to re-clone or rebase
git fetch origin
git reset --hard origin/main

---

4. Reflog vs Dangling Objects: What fsck --lost-found Finds

Command	Finds	Lifetime
git reflog	Every ref movement with context	Default 90d reachable / 30d unreachable
git fsck --unreachable --no-reflogs	Objects not reachable from any ref (ignores reflog)	Until git gc --prune=now
git fsck --lost-found	Same as above + writes dangling blobs/commits to .git/lost-found/	Same
git reflog expire --expire=now	Removes reflog entries (objects survive if reachable via another path)	Immediate

Using git fsck --lost-found for Forensic Analysis

# Full forensic scan — finds ALL unreachable objects
git fsck --unreachable --no-reflogs --full 2>&1 | tee fsck-report.txt

# Find dangling commits only
git fsck --unreachable --no-reflogs --commits 2>&1

# Write all dangling objects to lost-found for inspection
git fsck --unreachable --no-reflogs --lost-found

# Inspect individual dangling commits
ls .git/lost-found/commit/
for sha in $(ls .git/lost-found/commit/); do
  git log --oneline $sha -1
done

# Inspect dangling blobs (test artifacts, credentials that were filtered)
ls .git/lost-found/blob/
git cat-file -p $(ls .git/lost-found/blob/ | head -1)

Dangling Blobs After Filter-repo (Security Audit)

If you ran git filter-repo --path-glob '*.env' --invert-paths, removed secrets are still dangling blobs. This is a security audit scenario:

git fsck --unreachable --no-reflogs --lost-found
# Blobs are in .git/lost-found/blob/
# Compare against known leaked SHA list

---

5. CI Pipeline Incident Triage Playbook

Incident: CI Pipeline Accidentally Reset main to Wrong SHA

#!/bin/bash
# ci-reflog-rescue.sh — run this in CI after a bad pipeline reset

set -euo pipefail

AFFECTED_BRANCH="${1:-main}"
REPORT_FILE="reflog-rescue-report.txt"

{
  echo "=== Git Reflog Forensic Report ==="
  echo "Timestamp: $(date -Iseconds)"
  echo "Affected branch: $AFFECTED_BRANCH"
  echo ""

  echo "=== Last 20 HEAD movements ==="
  git reflog show HEAD --date=iso | tail -20
  echo ""

  echo "=== Last 20 branch tip movements ==="
  git reflog show "$AFFECTED_BRANCH" --date=iso | tail -20
  echo ""

  echo "=== Unreachable commits ==="
  git fsck --unreachable --no-reflogs --commits 2>&1 || true
  echo ""

  echo "=== Unreachable blobs (count) ==="
  git fsck --unreachable --no-reflogs --lost-found 2>&1 | grep -c "dangling blob" || echo "0"
  echo ""

  echo "=== Current HEAD vs origin/$AFFECTED_BRANCH ==="
  echo "Local HEAD:  $(git rev-parse HEAD)"
  echo "Remote HEAD: $(git rev-parse origin/$AFFECTED_BRANCH)"
  echo "Diverged:    $(git merge-base HEAD origin/$AFFECTED_BRANCH && echo 'yes' || echo 'no')"

} > "$REPORT_FILE"

cat "$REPORT_FILE"

Incident: Pipeline Did a git rebase -i That Dropped Commits

# Find the pre-rebase HEAD in the reflog
git reflog --date=iso | grep -E "rebase|reset|pick" | tail -20

# Get the SHA before rebase
PRE_REBASE_SHA=$(git reflog | grep "rebase (start)" | awk '{print $1}' | tail -1)

# Cherry-pick the dropped commits
git cherry-pick ${PRE_REBASE_SHA}..HEAD@{1}

# Push the fix
git push --force-with-lease origin "$(git branch --show-current)"

---

6. The reflog expire --expire=now Cleanup Lever

What It Does

git reflog expire removes old reflog entries. With --expire=now, it removes all entries for that ref immediately, regardless of age. This is the cleanup lever after you have successfully recovered your commits.

Critical safety: reflog expire --expire=now only deletes reflog entries, NOT the objects they reference — if a dangling commit is still reachable through another reflog entry or another branch, the object persists until git gc --prune=now runs.

Safe Cleanup Sequence

# Step 1: Verify recovery is complete
git fsck --unreachable --no-reflogs --commits | wc -l
# If > 0, you may still need those objects

# Step 2: Verify the reflog is no longer needed for recovery
git reflog show HEAD --date=iso | tail -5

# Step 3: Expire old entries
git reflog expire --expire=now --all

# Step 4: Run garbage collection to actually remove dangling objects
git gc --prune=now --aggressive

# Step 5: Verify disk space is recovered
du -sh .git/
git count-objects -Hv

When to Use --expire=now

• After a successful recovery: You have cherry-picked or reset to the correct state and want to prevent accidental reuse of stale reflog entries.
• Before a repository transfer: Clean the reflog before handing off to another team.
• CI runner disk pressure: CI runners with shallow clones do not benefit from reflog; expire immediately after confirming no pending recoveries.

# One-liner for CI runner cleanup (after confirming no pending recoveries)
git reflog expire --expire=now --all && git gc --prune=now --quiet

---

7. Automating Reflog Recovery in CI

Scheduled Reflog Snapshot (Backup)

#!/bin/bash
# backup-reflog.sh — run before every destructive CI operation

REPO_DIR="${1:-.}"
BACKUP_DIR="${REPO_DIR}/.git/reflog-backups"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)

mkdir -p "$BACKUP_DIR"

# Snapshot all reflogs
for ref in HEAD refs/heads/* refs/stash; do
  [ -f "${REPO_DIR}/.git/logs/${ref}" ] || continue
  cp "${REPO_DIR}/.git/logs/${ref}" \
     "${BACKUP_DIR}/$(echo $ref | tr '/' '_')_${TIMESTAMP}"
done

echo "Reflog snapshot saved: $BACKUP_DIR/"
ls -lh "$BACKUP_DIR/"

Restore from Snapshot

#!/bin/bash
# restore-reflog.sh

BACKUP_DIR="${1:-.}"
TIMESTAMP="${2:-}"  # e.g. 20260829_142301

if [ -z "$TIMESTAMP" ]; then
  echo "Usage: $0 <backup_dir> <timestamp>"
  exit 1
fi

for backup in "${BACKUP_DIR}"/*_"${TIMESTAMP}"; do
  [ -f "$backup" ] || continue
  ref_name=$(basename "$backup" | sed "s/_${TIMESTAMP}$//" | tr '_' '/')
  target=".git/logs/$ref_name"
  mkdir -p "$(dirname "$target")"
  cp "$backup" "$target"
  echo "Restored: $ref_name"
done

---

8. Reflog Pitfalls and Gotchas

Pitfall	Explanation	Prevention
Reflog is local only	Clone to a new machine and all reflogs are gone	Back up reflogs before risky operations (see Section 7)
Reflog entries expire	Default 90 days for reachable, 30 days for unreachable	Set gc.reflogExpireUnreachable = never for critical repos
git gc --prune=now removes dangling objects	Run after reflog expire and your dangling commits vanish	Always verify with git fsck before gc
Shallow clones have truncated reflogs	git clone --depth N only keeps N latest commits in reflog	Use full clones for recovery-critical workflows
Reflog is not a substitute for backups	Reflog is append-only but can be expired or lost with disk failure	Combine reflog + server-side backups + Git hosting redundancy
Force-push to shared branch loses collaborators' reflogs	When you force-push, teammates who have not pulled will push stale refs	Always use --force-with-lease instead of --force
Object store corruption hides reflog entries	If .git/objects/ is corrupted, dangling objects referenced by reflog may be unreadable	Regular git fsck --full health checks in CI

Configuration for Recovery-Critical Repos

# ~/.gitconfig or repo .git/config

[gc]
  # Never auto-expire unreachable reflog entries
  reflogExpireUnreachable = never

  # Aggressive pruning when gc runs
  prune = now

[fetch]
  # Preserve reflog on fetch (for collaborator recovery)
  prune = false

[push]
  # Warn before force-push to protected branches
  default = simple

---

Summary

Scenario	Command
Find lost commits	git reflog --date=iso
List dangling objects	git fsck --unreachable --no-reflogs --full
Recover a dropped commit	git cherry-pick <sha>
Recover from reset	git reset --hard HEAD@{1}
Recover from rebase	git reset --hard HEAD@{n}
Inspect lost-found	git fsck --lost-found && ls .git/lost-found/
Clean reflog safely	git reflog expire --expire=now --all && git gc --prune=now
Backup reflogs	cp .git/logs/* reflog-backups/

When to use this guide vs. docs/troubleshooting/reflog-rescue.md: That guide covers single-user, single-machine recovery. This guide covers forensic analysis, CI automation, fsck --lost-found, server-side recovery, and cleanup levers — everything a DevOps engineer needs for production incident response.