Skip to content

Git Submodule Management — DevOps Reference ​

Git submodules let a repository embed a fixed-pointer snapshot of another repo at a known commit. Unlike subtrees (which merge full history), submodules:

  • Store only a single commit SHA in the parent repo
  • Keep submodule history independent and pruned by default
  • Require an explicit git submodule update to advance the pointer
  • Work well for pinned dependencies, vendored libraries, or shared tooling

Quick Reference ​

bash
# Add a submodule
git submodule add https://github.com/org/shared-lib.git libs/shared-lib

# Clone a repo WITH submodules
git clone --recurse-submodules https://github.com/org/monorepo.git
git clone                          # then:
git submodule update --init --recursive

# Update a submodule to latest on its default branch
git submodule update --remote libs/shared-lib

# Work inside a submodule (commit in submodule first)
cd libs/shared-lib
git checkout main && git pull
cd ..
git add libs/shared-lib
git commit -m "chore(submodule): advance shared-lib to $(git -C libs/shared-lib rev-parse --short HEAD)"

1. Adding a Submodule ​

bash
# Add and track the default branch (usually main)
git submodule add https://github.com/org/shared-lib.git libs/shared-lib

# Pin to a specific branch
git submodule add -b release/v2 https://github.com/org/shared-lib.git libs/shared-lib

# Add with a specific commit (if already cloned locally)
git submodule add https://github.com/org/shared-lib.git libs/shared-lib
git -C libs/shared-lib checkout <specific-sha>
git add libs/shared-lib
git commit -m "chore(submodule): pin shared-lib at $(git -C libs/shared-lib rev-parse --short HEAD)"

What happens:

  1. .gitmodules is created/updated with the submodule URL and path
  2. The submodule directory contains a gitlink (index entry with mode 160000) instead of files
  3. The first submodule commit is recorded in the parent

2. Cloning and Initialising ​

ScenarioCommand
Clone + all submodulesgit clone --recurse-submodules URL
Clone then initgit clone URL && git submodule update --init
Init non-recursive submodulesgit submodule update --init
Init specific submodulegit submodule update --init libs/shared-lib
Init + checkout current branch in submodulegit submodule update --init --remote
Recursive (submodules of submodules)git submodule update --init --recursive
Clone with depth (submodules get depth too)git clone --recurse-submodules --depth 1 URL

3. Working in a Submodule ​

Three Modes of Submodule Update ​

--remote        → fetch + merge the tracked branch (like a normal pull)
(no flag)       → checkout the commit recorded in the index (detached HEAD)
bash
# MODE A: Point to the recorded SHA (after parent git pull)
git submodule update

# MODE B: Fetch and fast-forward to the latest on the tracked branch
git submodule update --remote
git submodule update --remote --recursive          # all submodules

# MODE C: Checkout a specific branch inside the submodule (for active dev)
cd libs/shared-lib
git checkout feature/my-fix
# work, commit, push
cd ../..
git add libs/shared-lib
git commit  # records the new submodule SHA

DevOps tip: In CI, prefer --remote only when you explicitly want bleeding-edge. Pin to SHA for reproducible builds. Combine with --remote-branch to control which branch is pulled.


4. CI/CD Integration Patterns ​

4a. Shallow Submodule Clones (CI Speed) ​

bash
# Clone parent shallowly + submodules with depth 1
git clone --depth 50 --recurse-submodules --shallow-submodules URL
yaml
# Azure Pipelines example
- bash: |
    git clone --depth 100 --recurse-submodules --shallow-submodules $(repo_url) .
    git submodule update --init --recursive
  displayName: Clone with shallow submodules

4b. Submodule Auth in CI (Private Dependencies) ​

bash
# Option 1: Git credential store (token-based)
git config --global url."https://${GITHUB_TOKEN}@github.com/".insteadOf "https://github.com/"

# Option 2: SSH forwarding
git config --global url."git@github.com:".insteadOf "https://github.com/"

# Option 3: Token in .gitmodules URL (not recommended for public repos)
# Set before clone:
export GITmodules_URL="https://x-access-token:${CI_TOKEN}@github.com/org/repo.git"

4c. Submodule Status Check in CI ​

bash
# Check if submodules are at recorded SHA vs behind
git submodule status

# CI gate: fail if any submodule is dirty or behind remote
if ! git submodule status | grep -q "^ "; then
  echo "ERROR: Submodule at unexpected commit"
  exit 1
fi

4d. Fetching Specific Submodule Branches in CI ​

bash
# Allow submodule to fetch its tracking branch (needed for --remote)
git submodule foreach git fetch origin
git submodule update --remote --init

5. Converting Between Subtrees and Submodules ​

Submodule → Subtree (useful when you need tighter integration) ​

bash
# 1. Ensure the submodule is at the commit you want to start the subtree history from
git -C libs/shared-lib log --oneline -1

# 2. Remove the submodule definition
git submodule deinit -f libs/shared-lib
git rm libs/shared-lib
rm -rf .git/modules/libs/shared-lib

# 3. Add as subtree
git subtree add --prefix=libs/shared-lib \
  https://github.com/org/shared-lib.git main --squash

# 4. Commit
git commit -m "chore: convert shared-lib submodule to subtree"

Subtree → Submodule (useful when you want independent versioning) ​

bash
# 1. Extract the subtree path to a separate bare repo
git subtree split --prefix=libs/shared-lib -b subtree-work

# 2. Clone the subtree work as a new repo (or push to a new remote)
git push git@github.com:org/shared-lib.git subtree-work:main

# 3. Remove the subtree files from the parent
git rm -r libs/shared-lib

# 4. Add as a submodule pointing to the new repo
git submodule add https://github.com/org/shared-lib.git libs/shared-lib

# 5. Commit
git commit -m "chore: convert shared-lib subtree to submodule"

6. Advanced: Nested Submodules ​

bash
# Add a submodule that itself has submodules
git submodule add --recursive https://github.com/org/nested-lib.git ext/nested-lib

# Update all levels
git submodule update --init --recursive

Caveat: Deep nesting makes CI cloning slower and credential management more complex. Consider flattening at 2 levels maximum.


The submodule pointer in the parent is just a gitlink entry in the index:

bash
# View the gitlink (shows the commit SHA the parent points to)
git ls-files --stage libs/shared-lib
# 160000 abc123... 0    libs/shared-lib

# The 160000 mode = commit gitlink

# See diff between recorded SHA and current HEAD in submodule
git diff libs/shared-lib
# Shows: -old SHA +new SHA

# See what changed inside the submodule (detached HEAD vs branch)
git -C libs/shared-lib log --oneline -3

If your pipeline builds a binary and you want reproducible submodule pins:

bash
# Export submodule gitlinks as a JSON artifact
git submodule status --recursive | \
  awk '{print $1, $2}' | \
  jq -Rn 'split("\n") | map(select(length > 0)) | map(split(" ")) | map({path: .[1], sha: .[0]})' \
  > submodule-pins.json

# Store submodule-pins.json as a pipeline artifact
# In downstream pipeline: restore submodules to exact pins
git submodule update --init --recursive

9. Removing a Submodule ​

bash
# 1. Deinit all tracked files and the submodule root
git submodule deinit -f -- libs/shared-lib

# 2. Remove the working directory (if not needed)
git rm libs/shared-lib

# 3. Remove the .git/modules entry
rm -rf .git/modules/libs/shared-lib

# 4. Commit
git commit -m "chore: remove shared-lib submodule"

10. Troubleshooting ​

SymptomCauseFix
fatal: needed be singularSubmodule not initialisedgit submodule update --init
Submodule dir is empty after clone--recurse-submodules not usedgit submodule update --init --recursive
Permission denied (publickey)Submodule uses SSH URL in CISet credential helper or rewrite URL
Submodule appears dirty after git submodule updateLocal branch diverged from recorded SHAgit submodule update --force
Authentication failed in CIToken not propagated to submodule fetchUse url.<base>.insteadOf rewrite
Submodule stuck at old SHAParent repo was not committed after git submodule update --remoteRe-run update and git add the new SHA

Recovering a Lost Submodule Commit ​

bash
# Find the commit in the reflog of the submodule's git dir
git -C .git/modules/libs/shared-lib reflog
# Find the SHA you need, then:
git -C libs/shared-lib checkout <recovered-sha>
git -C libs/shared-lib checkout -b recovery

See Also ​

  • docs/devops-workflows/git-subtree-monorepo.md — subtree approach for merged-history dependency management
  • docs/recipes/subtree-quick.md — subtree one-liner cheat sheet
  • docs/recipes/submodule-quick.md — this guide's companion quick reference
  • scripts/submodule-manage.sh — automation script for common submodule operations

Git Master — Git Reference for Every Skill Level