Skip to content

Nesting Repositories with Git Submodules: A Newbie's Guide

Introduction

Are you facing the challenge of handling multiple code pieces scattered across different repositories in your project, unsure how to seamlessly integrate them?

For developers new to the concept, managing disparate repositories within a single project can be overwhelming. Git submodules offer a guiding light, acting as a map through the maze of organizing and linking these separate codebases or libraries within your projects.

Real-Life Scenario: Aligning Frontend and Backend Strategies

Back in 2022, I found myself as the lead developer overseeing the backend team, while collaborating closely with a talented frontend developer responsible for crafting engaging user interfaces.

Our teams operated independently, each excelling in our specialized domains. However, this independence led to distinct branch strategies. The backend team adopted a unique approach, separate from the frontend's strategy.

Over time, this divergence in branch strategies caused disparities between our repositories' states. Aligning frontend changes with the evolving backend structures became a complex task. Ensuring seamless integration between our frontend branches and specific backend versions posed challenges.

Recognizing these challenges, I engaged in a discussion with the frontend developer. We brainstormed solutions to synchronize versions and segregate our branch strategies effectively.

During our deliberation, we explored the idea of utilizing Git submodules. It wasn't merely about syncing versions but aligning our separate branch strategies while maintaining distinct team autonomy.

The proposal envisioned Git submodules as the bridge between our frontend and backend repositories, facilitating version synchronization and accommodating separate yet aligned branch strategies. This approach aimed to streamline collaboration and ensure smoother integration between our teams' work.

Motivated by the vision of enhanced collaboration and harmonized branch strategies, we collectively agreed to integrate Git submodules. This decision promised a more cohesive development environment, allowing both teams to synchronize versions and align branch strategies seamlessly.

Extending to a Computer Vision Project

Additionally, in a computer vision project, I encountered a similar challenge. Testing code from various repositories required frequent modifications, causing inefficiencies. Managing these disparate codebases led me to prefer a unified repository managed with Git submodules. This approach enabled me to centralize and manage all required codebases efficiently, adapting them as needed.

Think of Git submodules as containers that neatly organize and link external repositories to your main project—providing a solution to the discomfort of handling disjointed pieces of code. Join us as we embark on a journey to explore how to clone, set up, and effortlessly synchronize these submodules within your projects.

Git Submodules

This document simplifies Git submodules in a beginner-friendly way, offering developers new to the concept a clear path to effectively manage multiple repositories as cohesive parts of their projects.

Cloning a Repository with Submodules and Cloning a Specific Submodule

Clone the Main Repository

  1. Open your terminal and navigate to the desired directory for cloning:

    cd /desired/directory/path

  2. Clone the main repository:

    git clone <repository_url>

    Replace <repository_url> with the URL of the main repository.

  3. Change your working directory to the repository:

    cd <repository_directory>

    Replace <repository_directory> with the name of the cloned directory.

Initialize and Update Submodules

  1. Initialize the submodules:

    git submodule init

    This sets up necessary Git configurations for submodules.

  2. Update the submodules:

    git submodule update

    This fetches submodule contents based on references in the main repository.

Clone a Specific Submodule

  1. Clone a specific submodule:

    git submodule update --recursive -- <submodule_path>

    Replace <submodule_path> with the specific submodule path. This command updates only the specified submodule and its dependencies, leaving others unchanged. - The --recursive flag initializes nested submodules within the specified submodule.

Now that you've successfully cloned the main repository along with its submodules, let's explore how to create and manage submodules within an existing repository.

If the update fails, you may want to read this stackoverflow thread

Creating a Git Submodule

  1. Move to the parent repository's root directory:

    cd /path/to/parent/repository

  2. Add the Submodule:

    git submodule add <submodule_repository_url> <submodule_path>
    • <submodule_repository_url>: URL of the submodule repository.
    • <submodule_path>: Path within the parent repository to place the submodule.

    Example:

    git submodule add https://github.com/example/submodule-repo.git path/to/submodule

  3. Commit the Changes:

    git commit -m "Add submodule: <submodule_path>"

    Replace <submodule_path> with the actual path used when adding the submodule.

  4. Push Changes (Optional):

    git push

Now, let's delve into pulling changes from both the main repository and its submodules to keep your local copy up to date.

Pulling Changes from the Main Repository and Submodules

Update the Main Repository

  1. Navigate to the main repository's directory:

    cd /path/to/main/repository

  2. Fetch the latest changes:

    git pull origin main

    This command fetches and merges the latest changes from the remote repository into your local main branch.

Update Submodules

  1. Update submodules to the latest commits:

    git submodule update --remote

    This updates each submodule to the commit specified by the main repository.

  2. Update a specific submodule:

    • Using git submodule update --remote <submodule_path>:

      git submodule update --remote path/to/submodule

    • Or manually in the submodule directory:

      cd path/to/submodule
git pull origin master

Pushing Updated Submodule References (Bonus)

  1. Inside the main repository, after updating submodule references:

    git commit -am "Update submodule references"
git push origin main

  2. If there are changes in the submodules themselves:

    cd path/to/submodule
git commit -am "Update submodule"
git push origin master

Removing a Git Submodule

To remove a submodule from your repository, follow these steps. The following instructions are based on a detailed explanation found on Stack Overflow (source).

  1. Move to the parent repository's root directory:

    Before removing the submodule, move it temporarily to a different location within your working directory.

    mv <submodule_path> <submodule_path>_tmp

  2. Deinitialize the Submodule: Use the following command to deinitialize the submodule:

    git submodule deinit -f -- <submodule_path>

  3. Remove Submodule Configuration: Delete the submodule's configuration from the .git/modules directory:

    rm -rf .git/modules/<submodule_path>

  4. Remove Submodule from Repository: There are two options to remove the submodule from the repository:

    a. If you want to completely remove it from the repository and your working tree:

    git rm -f <submodule_path>

    Note: Replace <submodule_path> with the actual submodule path.

    b. If you want to keep it in your working tree but remove it from the repository:

    git rm --cached <submodule_path>

  5. Restore Submodule (Optional): If you moved the submodule in step 0, restore it to its original location:

    mv <submodule_path>_tmp <submodule_path>

By following these steps, you'll effortlessly manage main repositories and their submodules, ensuring your projects are up to date.

Stay tuned for more Git tips and tricks on our blog for seamless collaboration and version control!