Git: Support nested git repositories

[javier-moreno-tridonic-com]

• VSCode Version: Code 1.18.0 (dcee220, 2017-11-08T21:22:49.932Z)
• OS Version: Windows_NT x64 6.1.7601
• Extensions:

Extension	Author (truncated)	Version
cpptools	ms-	0.14.0

---

Steps to Reproduce:

We use some command line tools from ARM to organise our project. This tools rely on special lib files to establish dependencies between modules. Each module should be an independent repository (git repositories in our case). So once all the references are deployed (the tools will "git clone" the indicated repositories in the lib files) the folder structure would look like this (folders in bold):

• project_folder (git repo)
  • module1.lib
  • module2.lib
  • module1 (git repo)
    • module3.lib
    • module3 (git repo)
  • module2 (git repo)

If we add project_folder as the root folder, the source control tab will only track the changes done in project_folder.

If we use the new multi-root project feature and also add module1, module2, and module3 to the workspace; we still find that only the changes in project_folder are tracked.

Nevertheless, if we add the folder in an strict reverse nesting order in the following manner:

1. With an empty project open module3 as root folder
2. Add folder to workspace... module2
3. Add folder to workspace... module1
4. Add folder to workspace... project_folder

Then we see that the source control view displays the several source control providers correctly.

Reproduces without extensions: Yes

[yh-sb]

Is there any updates about this bug?

[sveinse]

Thank you @javier-moreno-tridonic-com for the procedure in how to circumvent this issue by adding the projects to the workspace in inner->outer order. That helps a lot until this issue is fixed.

[mahmoudajawad]

The reverse order works, but screws up intillisense. My python modules from the child folder won't be able to access parent folder package causing all my child package modules to report errors of undefined module.
I'd suggest adding settings value to the workspace file where it would automatically populate at generation time all the auto-detected git repos. Then, we can manually set the additional paths for the nested git repos we are using.

[zurkoxxx]

Had the same problem.
Fixed it by adding nested repository as a submodule (https://git-scm.com/book/en/v2/Git-Tools-Submodules)

[mahmoudajawad]

Some use cases defy the use of submodules.

…

On Thu, 11 Oct 2018, 13:53 zurkoxxx, ***@***.***> wrote: Had the same problem. Fixed it by adding nested repository as a submodule ( https://git-scm.com/book/en/v2/Git-Tools-Submodules) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#37947 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AATVPqDP4H1nHVvdc94usfZI-QQq4j2uks5ujxUcgaJpZM4QX_sw> .

[rhymu8354]

In all my work, I have nested git repositories that are specifically not submodules, and for this reason: I want the top-level repository to track each sub-level repository via branch, not commit. I made a Python tool, mugit, which uses a manifest file to maintain the relative repository structure, similar to how Google organizes the Android multi-repository structure using their tool, repo.

Android Studio is notable for handling nested, non-submodule, multi-git repository structures quite nicely. It would be great if VSCode could do the same. In the mean time, I've been using the GitLens extension for most things, but still for making commits I'm pretty much forced to use another tool completely (e.g. git, git-gui, GitHub Desktop, SourceTree, etc.) to stage commits.

I'll be watching this issue with interest. I was disappointed to see there were at least two previous issues filed for this exact same subject, but were closed in the belief that multi-repository support was somehow either already done in some "insider" build (36250) or would somehow ride on the coattails of the multi-folder workspace support (12564).

[rahman]

+1 for nested git repo support

[RobertoSnap]

Is this still only possible with submodules? @rhymu8354

In that case
+1 for nested git repo support

[p3t3r5]

could be a workaround for you... I have created .gitmodules in the root of main repo with a content:
[submodule "your_module_name"]
path = relative/path/to/repo
url = git/url/to/repo

Ignore the file in .gitignore, if needed. Its not realy configurated submodule, but VSCode can handle this. Its works recursively to.

VS Code v1.30.2
Git 2.20.1

[benlindsay]

@p3t3r5 Thanks for this info! This feels like kind of a hack, but it's a good interim solution until repos in subdirectories are supported.

For anyone using this .gitmodules hack with no intention of using submodules, also note that the url = line, while necessary for this hack to work, does not require a real URL. Based on some quick testing on my macbook, it looks like any non-whitespace string will work. (I currently have it working with url = dummy_string_to_satisfy_vscode)

[Morgy93]

Any news on this? 🙏
I'd love to have one of these options, maybe one is easy to implement:

main_repo/* <- Git1
main_repo/vendor/module1 <- Git2
main_repo/vendor/module2 <- Git3

(Pretty common for like php composer stuff)

A) Get the current path from terminal
The git integration checks the current path upon refresh.
So it uses Git1, but if I'm cd main_repo/vendor/module1 then it shows me Git2 instead

• Probably not easy because of different shells and stuff

B) Use the current selection from the explorer
The git integration uses the current selected path upon refresh
So as long as my selection is somewhere inside of Git3 this will be used for git integration:

C) Set the current git path
Slightly different from B) if the "use current selection" is difficult:
Just right click and "set this as git path" so that this folder/path is used for the git integration.

For me it seems that the git integration does just like git status and this is of course always from the root of the opened folder.
If you could set a git path manually (via right click and select [C] or just use selection [B]) you could simply "add a prefix" do thinks like: cd PATH && git status (there are probably better ways but it would work)

I hope at least one is able to understand this mess 😁
Not being able to simply/quickly select/use a nested repo sadly is a deal breaker for me.

[benlindsay]

Did you try the solution above where you make a .gitmodules file as though you were going to use submodules but don't actually check it in? It's not ideal but at least it's a one time setup that allows integration of all your git repos

[Morgy93]

@benlindsay
I work on like 20 of those "main repos" and all of them have like 20 sub git repos which also have dependencies and my co-workers use those main repos as well, so I cannot track what sub repos are added.
I'd love to use a workaround, but this one is sadly not maintainable for me.

[gogobd]

Any news?

[nteymory]

I already mentioned this in #157347, but

If you temporarily move the .git folder from your workspace root, then open the workspace, then put the .git folder back, vscode detects all the repositories correctly. Note, this does not work if you do it while the workspace is currently open.

It seems to me there is a check that happens when a workspace is opened:

• Is there a repository in the root of the workspace?
• If yes, then for the rest of this session don't scan subdirectories for additional repositories
• If no, then scan all subdirectories for repositories

Assuming such a check is actually happening, I think all we need is a configuration option to disable it.

[nteymory]

Ok, this bothered me enough that I decided to spend the afternoon debugging it and I've figured it out. The problem lies within the function Model.getOpenRepository inside model.ts

The problematic part is this for loop

for (const liveRepository of this.openRepositories.sort((a, b) => b.repository.root.length - a.repository.root.length)) {
	if (!isDescendant(liveRepository.repository.root, resourcePath)) {
		continue;
	}

	for (const submodule of liveRepository.repository.submodules) {
		const submoduleRoot = path.join(liveRepository.repository.root, submodule.path);

		if (isDescendant(submoduleRoot, resourcePath)) {
			continue outer;
		}
	}

	return liveRepository;
}

What is happening here is that the resourcePath (the current potential directory that could be a git repository) is being checked to see if it's a subdirectory (isDescendent) of some already discovered repository. If it is, the for loop checks if it corresponds to a submodule of the parent and if so, will not consider it part of this repo. Otherwise, it assumes that the path being tested is part of the parent repository. This sort of makes sense..

This explains why the submodule "hack" above works, why if you load repositories in reverse order it works, and also why my workaround of removing the parent .git folder and readding afterwards works as well -- as long as you haven't discovered the parent repro before you add the child repro everything works.

Now, I'm not exactly sure what the right way to fix this is, but I propose the following change: check if resourcePath contains a .git folder, if it does, it should be considered it's own repo. That's it. There is probably a "more correct" way to do this, but this is what I did in my local build and it fixed the problem for me:

for (const liveRepository of this.openRepositories.sort((a, b) => b.repository.root.length - a.repository.root.length)) {
	if (!isDescendant(liveRepository.repository.root, resourcePath)) {
		continue;
	}

	for (const submodule of liveRepository.repository.submodules) {
		const submoduleRoot = path.join(liveRepository.repository.root, submodule.path);

		if (isDescendant(submoduleRoot, resourcePath)) {
			continue outer;
		}
	}

        // NEW CODE
	// We are a descendant, and we are not a submodule
	// If we also are not the exact path for the repository being tested
	if (!pathEquals(liveRepository.repository.root, resourcePath)) {
		// Then check if we have our own .git folder
		// If we do, this implies we are our own repository
		// and therefore and not just a subfolder of this outer repository
		if (fs.existsSync(path.join(resourcePath, ".git"))) {
			continue;
		}
	}
        // ~NEW CODE

	return liveRepository;
}

I'm willing to make a pull request if this solution is not considered horrible @lszomoru. Otherwise, I'm happy to leave it to you to fix "properly".

If anyone doesn't want to wait for the fix, you can patch this into your build right away. Here is how to do it:

Locate your vscode install and navigate to resources\app\extensions\git\dist\main.js. For example for me it was located here

C:\Users\<username>\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\git\dist\main.js

This file is optimized so it's a bit tricky to edit. Make sure you are using a decent editor. Look for this bit of code (0,s.isDescendant)(e.repository.root,t). You can manually expand the braces to see the logic above:

			e: for(const e of this.openRepositories.sort(((e,t)=>t.repository.root.length-e.repository.root.length)))
			{
				if((0,s.isDescendant)(e.repository.root,t))
				{
					for(const r of e.repository.submodules)
					{
						const i=c.join(e.repository.root,r.path);
						if((0,s.isDescendant)(i,t))
						{
							continue e
						}
					}
					return e
				}
			}

Patch it by adding this bit into the for loop

			e: for(const e of this.openRepositories.sort(((e,t)=>t.repository.root.length-e.repository.root.length)))
			{
				if((0,s.isDescendant)(e.repository.root,t))
				{
					for(const r of e.repository.submodules)
					{
						const i=c.join(e.repository.root,r.path);
						if((0,s.isDescendant)(i,t))
						{
							continue e
						}
					}
					
                                        // NEW CODE
					if (!s.pathEquals(e.repository.root, t)) {
						if (l.existsSync(c.join(t, ".git"))) {
							continue e
						}
					}
                                        // ~NEW CODE

					return e
				}
			}

Reload your workspace and it should now work the way you expect

There is one more thing to consider here, there is a setting that you may also want to double check. By default, vscode will only search 1 folder deep for additional repos. If your repos are deeper, this alone won't fix it, update this setting as well:

git.repositoryScanMaxDepth

[JanJakes]

@nteymory Awesome! The patch seems to work for me. I think a pull request would move any further discussion to the code and could help move this forward; what do you think?

[lszomoru]

@nteymory, thank you very much for the investigation. Since the issue has been labeled with help wanted, I would suggest that you create a PR so that we can move the conversation there. Adjusting the repository discovery code is a great start but we will have to see if there are other areas in the git extension that need to be tweaked to fully support nested repositories.

[nteymory]

Sure, PR created: #157812

[BrockHallenbeck]

@nteymory is this a git specific solution? I am observing identical behavior using subversion and am hopeful this effort will benefit SVN users as well.

[nteymory]

@nteymory is this a git specific solution? I am observing identical behavior using subversion and am hopeful this effort will benefit SVN users as well.

Unfortunately, yes, it is

[lszomoru]

I know that it has been a long time coming, but today's Insiders release (2022-09-08) contains changes to the repository discovery code that detects nested git repositories. Depending on how deep the nested repositories are, one might have to tweak the git.repositoryScanMaxDepth setting.

At the moment we have only tweaked the repository discovery code, but I do expect some additional paper-cuts on the way so I would like to ask you to update to the latest Insiders release, try out this new capability and let us know if you are running into any issues. Thank you very much!

[wesleybl]

@lszomoru

I tested with the following version:

Version: 1.72.0-insider
Commit: 5cecbf4746a7b95dedaf9fc4534b9786b9a30ace
Date: 2022-09-09T16:25:41.046Z
Electron: 19.0.12
Chromium: 102.0.5005.167
Node.js: 16.14.2
V8: 10.2.154.15-electron.0
OS: Linux x64 4.15.0-176-generic
Sandboxed: Yes

It worked for me! I had to be setting git.repositoryScanMaxDepth to 2 to work for me. My root folder is a git repository, and inside it is a folder that has two git repositories. Changes in 3 repositories are detected.

After changing git.repositoryScanMaxDepth, it only started working after restarting VSCode. This is expected, correct?

Thanks for the fix!

[lszomoru]

@wesleybl, thanks for testing out the changes.

After changing git.repositoryScanMaxDepth, it only started working after restarting VSCode. This is expected, correct?

That is currently expected behaviour.

[hemmingsv]

@lszomoru installing code-insiders with latest snap on ubuntu 22.04 => works perfectly, all git subfolders detected, inline git diffs show up as expected. Fantastic!

[lszomoru]

@hemmingsv, thank you very much for trying out and validating the new functionality.

[lszomoru]

With the changes to the repository discovery code checked into main, and slated to be included in the next Stable release, I will go ahead and close this issue. Any other issues that are being discovered with nested git repositories should be handled in separate issues. Thank you all for your help, and for your patience.

[abhijit-chikane]

I just tried the nested git repo Initially it did show me the git subfolders but after I reload the vscode it is no more working
anything I am missing or any setting that I may have changed?

[lszomoru]

@abhijitchikane, could you please create a separate issue so that we continue the investigation? Thanks!