Skip to content

Multiple LLVM Workspaces

Jump to bottom
Renato Golin edited this page Jun 24, 2024 · 5 revisions

If you find yourself constantly switching between different LLVM source trees to work on separate projects, you may consider using Git Work-tree.

From their docs: "A git repository can support multiple working trees, allowing you to check out more than one branch at a time".

This has many advantages, as now you can:

  • have a separate source and build directory for each branch while sharing disk space and metadata
  • work in multiple source directories as effectively as you work on a single one (rebase, cherry-pick etc. just work)
  • quickly work on an upstream review fix while not breaking your current editor setup / running build
  • build one branch while working on another without fear of changing sources mid-way
  • edit branches directly before a cherry-pick into another (and even run tests!)
  • have a different compile_commands.json file for each branch
  • help your editor not to go crazy re-indexing thousands of files when you change branches
  • keep a tracking branch clean (for example tpp-mlir), while working on other stuff

Workspace

To create the workspace, you'll need a common directory for your repository and your work-trees. You shouldn't create the work-trees inside the repository, as it will make git think they're new files.

Clone the repository

Create the "workspace" and clone your fork inside as origin, adding the main repo as upstream:

# Create the workspace
mkdir workspace && cd workspace

# Clone your fork
git clone git@github.com:<username>/llvm-project.git

# Add upstream
cd llvm-project
git remote add upstream git@github.com:llvm/llvm-project.git

# Fetch all
git fetch --all

This is where main is, too, you can't create a work-tree for main, which is a good thing, as you don't want to develop on the main branch anyway.

If you use Github's CLI, you need to clone it differently:

gh repo clone llvm/llvm-project

This will already create both upstream and origin in the right order, but be careful, as it makes main track upstream, not origin, so make sure you don't accidentally push to upstream instead of origin.

Work-tree makes it slightly easier, as you don't work on the same source directory (you should never work in the main repo source anyway), but you have to follow the pattern, or you'll mix branches and make a mess.

Create work-trees

For each separate tracking or feature branch, create a new work-tree. If the tracking branch doesn't exist in origin, create it.

# Crate the branch in the origin, if needed
git push origin main:<name>

# Create the work-tree
git worktree add --track -b <name> ../<name> origin/<name>

You don't need to track a branch from origin, you can have a quick local work-tree. If you do, you can skip most of it and just:

# Create a local work-tree
git worktree add ../<name>

Even if you have a sub-directory collecting all work-trees, git is smart enough to name it correctly:

$ git worktree add ../worktrees/temp
Preparing worktree (new branch 'temp')

Now you can create the local build directory too.

# Go into the work-tree and create its own build directory
cd ../<name>
mkdir build

If you've created a work-tree for the tracking branch tpp-mlir and a few development branches, your directory structure is similar to:

workspace/
├── llvm-project
|   ├── llvm
|   └── ...
├── tpp-mlir
|   ├── build
|   ├── llvm
|   └── ...
├── dev1
|   ├── build
|   ├── llvm
|   └── ...
└── dev2
    ├── build
    ├── llvm
    └── ...

Building

Once you have the work-trees, you can create a local build directory inside each one and build them independently of each other. This is very convenient, as you can edit one source and build another, then edit the other and build the one.

You can also interplay this with any other project that uses LLVM. If your project builds on top of LLVM, you can install your current worktree locally (ex. -DCMAKE_INSTALL_PREFIX=install), then run ninja install and then move it to a common directory with the hash of the commit.

Example:

// In your LLVM worktree branch build
SHARED_INSTALL=~/install/llvm
COMMIT=$(git rev-parse HEAD)
ninja install
mv install $SHARED_INSTALL/$COMMIT

// In your project
cmake ... -DMLIR_DIR=$SHARED_INSTALL/$COMMIT/lib/cmake/mlir ...
ninja

This means you can build any number of LLVMs in parallel, install and develop multiple projects against different LLVM builds without conflict.

Work-cycle

With multiple branches and build directories, you can keep different editors open on different branches, build one branch while continue working on another, etc.

With this, you can use the same CMake command to build all branches, since they'll all point to the same relative location (../llvm).

When you push to your origin, it'll be the same as if you were working on a branch (because you are).

# Business as usual
git ci -m "Some message" .
git push

Same for rebase, cherry-pick, etc.

Listing your work-trees

Listing work-trees show which names you have and what are the commits they're in:

$ git worktree list
.../llvm/llvm-project        f101196d695e [main]
.../llvm/worktrees/pack      f101196d695e [pack]
.../llvm/worktrees/temp      f101196d695e [temp]
.../llvm/worktrees/tpp-mlir  cb0d2887ab8f [tpp-mlir]

Removing a work-tree

Once your change is merged or your branch is no longer necessary, you can remove the work-tree.

git worktree remove temp

This does not remove the origin branch, you'll have to remove it yourself, either by going on the Github UI or by:

git push origin :temp

Note: the pattern :<name> means "push empty to branch", which is the same as delete.

Clone this wiki locally