Git Bundle & Shallow / Partial Clone: Air-Gapped CI & Bandwidth-Constrained Pipelines¶

git bundle packs a repository (or subset of refs) into a single portable file — ideal for air-gapped transfers, offline CI runners, disaster recovery snapshots, and secure one-way pushes through restricted networks. git clone --depth and partial clone filters dramatically reduce bandwidth and disk usage for massive monorepos on CI.

Quick Reference¶

Action	Command	When to Use
Bundle entire repo	`git bundle create backup.bundle --all`	Full offline mirror / DR snapshot
Bundle specific refs	`git bundle create HEAD.bundle HEAD main develop`	Selective push through restricted network
Verify a bundle	`git bundle verify backup.bundle`	Confirm bundle is self-contained before transfer
List refs in bundle	`git bundle list-heads backup.bundle`	Inspect what a bundle contains
Clone from bundle	`git clone backup.bundle ./repo`	Offline clone / restore from bundle
Pull from bundle	`git pull bundle.bundle main`	Merge bundle into existing repo
Shallow clone	`git clone --depth 50 --branch main <url>`	CI runner with limited bandwidth
Incremental shallow fetch	`git fetch --depth 100`	Extend shallow history on CI
Full history	`git fetch --unshallow`	When a late pipeline stage needs full history
Partial clone (no blobs)	`git clone --filter=blob:none <url>`	Large monorepo where you only need source tree
Sparse checkout	`git sparse-checkout set src/app build/`	Only needed directories in partial clone

Part 1: `git bundle` — Offline Transfer and CI Backup¶

Why Bundle?¶

Most CI systems fetch directly from a Git server. But production environments with DMZ isolation, on-premise Forgejo/Gitea behind a VPN, or air-gapped datacenters cannot reach upstream. git bundle solves this by serialising Git objects and refs into a single file that can be transferred via SCP, USB, S3 pre-signed URL, or any file-based mechanism.

Common DevOps scenarios: - DR snapshot: Bundle a production repo before a risky migration or major refactor; store the .bundle file in object storage. - One-way sync through a jump host: git bundle create on the inside, transfer via SCP to the DMZ, then git pull bundle.bundle on the air-gapped runner. - Offline code review: Bundle a feature branch and all dependent commits to share without exposing the full repo. - CI artifact: In a pipeline that cannot reach the Git server (network policy), the previous job bundles --all, publishes as a build artifact, and the next job clones from it.

Creating a Bundle¶

# Full repository bundle (all branches + tags)
git bundle create /backups/repo-$(date +%Y%m%d).bundle --all

# Selective: only the refs you need for the CI job
git bundle create ci-input.bundle HEAD main release/2.4.1

# Incremental: bundle only commits since the last run
git bundle create incremental.bundle ^last-build-commit HEAD main

The ^<commit> <refspec> syntax means "everything reachable from <refspec> but not from <commit>". Use git rev-parse last-build-commit to get the SHA from the previous pipeline run's env var or artifact.

Verifying and Inspecting a Bundle¶

Always verify before transferring — a corrupted or incomplete bundle wastes time on an air-gapped system.

# Check the bundle is self-contained and covers the claimed refs
git bundle verify /path/to/repo.bundle
# Exit 0: bundle is complete and valid
# Exit 1: refs are missing or bundle is corrupt

# List all refs packed into the bundle
git bundle list-heads /path/to/repo.bundle

Cloning from / Pulling into a Bundle¶

# Fresh clone from bundle
git clone /backups/repo-$(date +%Y%m%d).bundle ./workspace

# Clone to a specific branch (if bundle only contains one branch)
git clone -b main /backups/repo-$(date +%Y%m%d).bundle ./workspace

# Pull/merge bundle into an existing repo (common in CI chains)
cd existing-repo
git pull /path/to/ci-input.bundle main

CI Pipeline Example: Two-Job Air-Gapped Setup¶

Job 1 (has Git server access):

#!/bin/bash
set -euo pipefail
# Create bundle of the trigger commit and main branch
GIT_COMMIT=$(git rev-parse HEAD)
git bundle create repo.bundle --tagged-refs \
  HEAD "$GIT_COMMIT" main origin/main
# Upload as pipeline artifact (e.g., Azure Pipelines, GitHub Actions, etc.)
# In GitHub Actions:
echo "artifact_name=repo" >> $GITHUB_ENV

Job 2 (air-gapped runner — no Git server access):

#!/bin/bash
set -euo pipefail
# Download artifact: repo.bundle
# Then:
git clone repo.bundle ./app
cd app
git bundle verify repo.bundle   # sanity check
npm ci
npm run build

Bundle Size and Object Packing¶

Bundles are thin on first creation (they store all reachable objects). Use git repack before bundling for maximum portability:

git bundle create clean.bundle --all
git clone clean.bundle /tmp/check
cd /tmp/check && git repack -ad
# Re-bundle for a smaller file:
git bundle create ../final.bundle --all

Part 2: Shallow Clone — Speed for CI Runners¶

`--depth`: Fixed History Depth¶

# Clone with last 50 commits on main
git clone --depth 50 --branch main https://git.internal/repo.git ./app

# Verify depth
cd ./app
git log --oneline | wc -l   # should be ~50

# Check total object count saved
git count-objects -vH

`--single-branch` (also `--no-single-branch`)¶

# Only history needed for this branch (default with --depth)
git clone --depth 1 --branch main https://git.internal/repo.git ./app

# Clone a single branch without depth limit (no history merge commits)
git clone --single-branch --branch main https://git.internal/repo.git ./app

`--branch` with Tags¶

# Shallow clone a specific release tag (no all-branches history)
git clone --depth 1 --branch v2.4.1 https://git.internal/repo.git ./release

Extending a Shallow Clone¶

# Fetch more history (e.g., from 50 to 200 commits)
git fetch --depth 200

# Fetch all remaining history
git fetch --unshallow

Warning: --unshallow on a shallow clone that contains merge commits may fail with fatal: refusing to merge unrelated histories. Resolve with git pull --allow-unrelated-histories or re-clone without --depth if full history is truly needed.

Shallow Clone in CI: Makefile / Script Pattern¶

# In Dockerfile / CI script
GIT_DEPTH="${GIT_DEPTH:-50}"
git clone --depth "$GIT_DEPTH" --branch "${GIT_BRANCH:-main}" \
  "$REPO_URL" /app
cd /app

# For pipelines that need full history only in certain stages:
# Stage 1: shallow (fast install/build)
# Stage N: fetch full history only when needed
if [ "$NEED_FULL_HISTORY" = "1" ]; then
  git fetch --unshallow
fi

Part 3: Partial Clone — Monorepo Without the Weight¶

For repos over 1 GB, shallow clone alone is insufficient. git clone --filter downloads only the Git object graph (commits, trees), leaving blobs (file content) to be fetched on demand.

`--filter=blob:none` — Tree-Only Clone¶

git clone --filter=blob:none https://git.internal/monorepo.git ./app
# Downloads commit graph and directory structure; file content fetched lazily
# Typical size reduction: 60–90% for repos with large binary assets

`--filter=blob:limit=<n>[kmg]` — Filter by Size¶

# Exclude blobs larger than 10 MB from initial download
git clone --filter=blob:limit=10m https://git.internal/monorepo.git ./app

Sparse Checkout — Only the Directory You Need¶

Partial clone fetches the full tree. Sparse checkout limits the working directory further:

# After partial clone:
cd ./monorepo

# Only check out src/ and build/ directories (no lib/, docs/, etc.)
git sparse-checkout init --cone
git sparse-checkout set src/app build/ terraform/

# Verify sparse checkout
git sparse-checkout list
# Output: src/app build terraform

For non-cone mode (glob patterns):

git sparse-checkout set --no-cone 'src/**' 'build/**' '!tests/**'

Combining Filters¶

# Partial clone + shallow depth + sparse checkout = minimum CI footprint
git clone \
  --filter=blob:none \
  --depth 30 \
  --branch main \
  --sparse \
  https://git.internal/monorepo.git ./app

cd ./app
git sparse-checkout set src/api tests/e2e

Lazy Blob Fetch¶

When a partial clone is checked out, Git fetches blobs on first access (e.g., opening a file in an editor). For CI build scripts that need specific files:

# Force-fetch a specific directory's files (avoids on-demand stalls during build)
git fetch --filter=blob:none --no-recurse-submodules origin -- src/frontend

# Or use rev-list to determine what blobs you need and pre-fetch them:
git rev-list --objects HEAD -- src/frontend | \
  git fetch-pack --stdin --quantize=0 origin | \
  git index-pack --stdin

Part 4: Performance Comparison and Decision Tree¶

Strategy	Network Transfer	Disk Usage	History Available	Best For
Full clone	100% of repo	Full	Yes	Local dev, release builds
Shallow clone `--depth N`	~N× avg commit size	Minimal	Last N commits	CI install/test steps
Partial `--filter=blob:none`	Commit graph only	~5–20%	Yes	Monorepo CI build
Partial + sparse checkout	Commit graph + subset	~1–10%	Yes	Mega-monorepo (NxN builds)
Bundle (full)	~full repo size (once)	Full	Yes	Offline/air-gap CI
Bundle (incremental)	Diff only	Full	Yes	Frequent air-gap sync

Decision tree: 1. Can the runner reach the Git server? → No → Use git bundle 2. Is the repo > 500 MB? → Yes → Use --filter=blob:none 3. Does the pipeline only need one subdirectory? → Yes → Add git sparse-checkout set 4. Is build speed critical? → Yes → Add --depth 50 (adjust to minimum viable) 5. Does a later stage need full history? → Yes → Plan --unshallow only for that stage

Security Considerations¶

Bundles are not encrypted. Transport via HTTPS, SCP with key-based auth, or encrypt with gpg --encrypt before transfer if the bundle contains secrets (e.g., if a dev accidentally bundled a repo with .env files).
Verify bundle signatures if bundled from an untrusted source: git bundle verify only checks completeness, not authenticity — use GPG-signed tags alongside the bundle.
Shallow clones reduce attack surface — fewer old commits mean fewer potential references to leaked credentials in commit messages.
Partial clones do not reduce risk from malicious blobs that may exist in history; use git filter-repo to scrub history before creating bundles.

Cleanup¶

# Remove bundle files after successful transfer / CI run
rm -f repo.bundle incremental.bundle

# Convert shallow clone to full (after confirming no more incremental fetches needed)
git fetch --unshallow

# Convert partial clone back to full (download all remaining blobs)
git fetch --filter=blob:none --refetch origin
# (refetch downloads all blobs not already present)