Contributing

See Adding Tests page for technical details on how to implement new test cases.

Branches

In addition to main branch, VK-GL-CTS repository hosts several API-specific release branches. They follow the convention of <CTS name>.<API major>.<API minor>.<CTS major> where <CTS name> is one of:

• Vulkan CTS: vulkan-cts
• OpenGL CTS: opengl-cts
• OpenGL ES CTS: opengl-es-cts

If you are contributing new tests or other improvements, they must always target main branch. Bug fixes on the other hand must be targeted to the earliest release branch where the bug occurs.

Coding Conventions

The VK-GL-CTS is written in C++. Most of the coding conventions are enforced by clang-format. In general, it is enough to match the style of the code surrounding new code.

Note: the coding conventions of VK-GL-CTS were dramatically improved in 2024. Any change that was in-progress before the repo was reformatted would see non-trivial conflicts when directly rebased on or merged with tip of tree. The following procedure should be followed to make this rebase or merge both safe and trivial.

How to align existing branches to new format

Assume the branch new-tests was written against the older versions of VK-GL-CTS, and was intended to be applied to the vulkan-cts-1.3.8 branch. Assume the git commit that reformatted the vulkan-cts-1.3.8 branch is tagged as reformat-vulkan-cts-1.3.8 and the last commit on that branch before is was reformatted is tagged as before-reformat-vulkan-cts-1.3.8.

The process of aligning a non-reformatted branch to the new formatting involves:

1. Align with the parent branch before reformatting (this can be done by merging or rebasing, see below).
2. Usual conflict resolution (manual).
3. Merge or rebase with the parent branch after reformatting, using the changes in the branch new-tests when it comes to conflict resolution. These changes will contain everything new introduced by the branch compared to the parent branch, albeit not reformatted.
4. Reformat (this will fix all the new changes).
5. Amend the last merge commit.

Pre-requisites

To run the script, the following prerequisites are needed:

• clang-format: Since clang-format's output is not fixed between versions, we use clang-format 16.0.4. Prebuilt binaries can be downloaded from here: https://github.com/llvm/llvm-project/releases/tag/llvmorg-16.0.4

  • For Linux: https://github.com/llvm/llvm-project/releases/download/llvmorg-16.0.4/clang+llvm-16.0.4-x86_64-linux-gnu-ubuntu-22.04.tar.xz

  • For MacOS: https://github.com/llvm/llvm-project/releases/download/llvmorg-16.0.4/clang+llvm-16.0.4-arm64-apple-darwin22.0.tar.xz

  • For Windows: https://github.com/llvm/llvm-project/releases/download/llvmorg-16.0.4/LLVM-16.0.4-win64.exe

• Once downloaded, set the environment variable CTS_CLANG_FORMAT to /path/to/clang-16.0.4/bin/clang-format

• On windows, git bash is likely the best way to get bash.

Rebase method

# Back up the branch!
$ git branch backup

# Bring the branch up to date with the commit _before_ the reformat commit
$ git rebase before-reformat-vulkan-cts-1.3.8

# If there are conflicts, resolve as usual

# Rebase over the reformat change.  Almost every file will conflict, but we resolve it by undoing the formatting.
$ git rebase -Xtheirs reformat-vulkan-cts-1.3.8

# Now redo the formatting.  Ignore the python error about `git diff`, that is expected.
$ ./format_all.sh

# Amend the commit
$ git commit -a --amend

# Further rebase to tip-of-tree as usual

Merge method

# Back up the branch!
$ git branch backup

# Bring the branch up to date with the commit _just before_ the reformat commit
$ git merge before-reformat-vulkan-cts-1.3.8

# If there are conflicts, resolve as usual

# Merge the reformat change.  Almost every file will conflict, but we resolve it by undoing the formatting.
# Note that -Xtheirs is used for rebase, but -Xours for merge!
$ git merge -Xours reformat-vulkan-cts-1.3.8

# Now redo the formatting.  Ignore the python error about `git diff`, that is expected.
$ ./format_all.sh

# Amend the commit
$ git commit -a --amend

# Further rebase to tip-of-tree as usual

For dev/ branches for in-flight extensions, please follow the merge strategy.

Blame: In the event that git blame shows the reformat change, use the --ignore-rev= flag to exclude the reformat changes.

Pull Requests

Pull requests for new tests or features should target main, and bug fixes should target the release branch where the bug was first introduced in (for example vulkan-cts-1.0.1 for a bug in test that was added in Vulkan CTS 1.0.1).

History

To keep git history meaningful, merge requests should generally contain just one, or a few atomic commits, meaning that all intermediate steps build & function correctly. When you need to update an existing pull request (to address feedback for example), please do so by modifying the existing commit(s) rather than adding additional commits on top.

To update one commit pull request, you can do:

git checkout <pull request branch>
# edit files
git add <modified files>
git commit --amend
git push --force origin <merge request branch>

--amend flag modifies topmost commit instead of creating a new one.

Testing

See Testing New Tests for documentation how any new test code should be tested.

In addition, dEQP has self-test module under modules/internal. Any framework changes should be tested by running internal tests. For any new framework features appropriate tests should be added as well.