WIP: CMake only PR. #1947

gpx1000 · 2023-06-14T22:56:52Z

This allows other tooling such as CLion to work with building this project. It also allows for building with XCode directly. It should make maintaining the project easier in the long run.

…uilding this project. It also allows for building with XCode directly. It should make maintaining the project easier in the long run.

billhollings

Thanks so much for submitting this. Automated builds are very helpful.

The existing Makefile builds MoltenVK by using the Xcode projects embedded in the repo content, so an automated build process would currently fetch the MoltenVK repo, then run fetchDependencies and make.

I assume the resulting CMake-based build process still runs fetchDependencies and make, after CMake? Otherwise how are the dependencies and projects built?

So, I'm not sure what this CMake build process adds to the process. Can you clarify a use case that requires CMake instead of just running fetchDependencies and make? What does CLion need?

I've also made some inline review comments about the content of this PR.

External/CMakeLists.txt

MoltenVK/CMakeLists.txt

MoltenVKShaderConverter/CMakeLists.txt

billhollings · 2023-06-15T15:18:12Z

mvkGitRevDerived.h.in

@@ -0,0 +1,2 @@
+// Auto-generated by MoltenVK
+static const char* mvkRevString = "@MVK_GIT_REV@";


I'd like to avoid cluttering the top level directory if possible. Can we find a different home for this file? Maybe under Templates?

I'll move it there.

This file should be moved to a new Templates/cmake directory.

gpx1000 · 2023-06-15T16:17:24Z

You actually don't need fetchDependencies if we switch to submodules. CMake is a pretty smart build system that correctly determines dependencies and only builds what's asked for when it's asked for. So if we restructure the project a bit this would be the entire process from clone to built project:
git clone --recurse-submodules -j8 https://github.com/KhronosGroup/MoltenVk
mkdir -p MoltenVk/build && cd MoltenVk/build
cmake ..
cmake --build ..

You can add -G XCode to the first cmake command and it will create an XCode project complete with the dependencies linked correctly and ready to be built as needed. Otherwise, you can use -G ninja or -G Makefile for those command line build systems.
Now, let's say you're a consumer of MoltenVK, instead of downloading a built library, you want to work with MoltenVK within your project. Here's all you'd have to do with a CMake project:
git submodule add https://github.com/KhronosGroup/MoltenVk
git submodule update --init --recursive <-- only needed if using submodules instead of fetchDependencies, if fetchDependencies then run it here instead.
in your CMakeLists.txt:
add_subdirectory(MoltenVk)
target_link_libraries(myexectuablename PUBLIC moltenVk)

Because the dependencies for MoltenVk are set to PUBLIC in the link_libraries command in our CMakeLists.txt, they will get promoted automatically to the consuming target (myexecutablename) So working with MoltenVK becomes easier for end users if they have a need/want to build moltenvk.
The cool thing there is their entire build process can also use cpack which will create an installer and package things in OS independent way (i.e. if my project targets more than one platform, I can have CI/CD correctly create frameworks as needed and make them available correctly for the CD). When it comes time to actually publish, use cmake to generate XCode and simply submit validated / tested applications including to the ios store (there's more magic behind the scenes to make it that usable but MoltenVK isn't a self-contained executable so no need for magic here). Other platforms can include or not MoltenVK based upon need from the CMakeLists.txt level.

billhollings · 2023-06-15T18:04:34Z

Thanks for clarifying the larger integration building use case. I can see benefit in consistency of plugging into a larger CMake process, although the `fetchDependencies/make' scripts are fairly easily integratable into a larger build process at a script level.

You actually don't need fetchDependencies if we switch to submodules.

Long ago, we used submodules, but moved to a script to provide options to manage the build complexity. But I am not adverse to moving back to submodules for the dependencies, if we can cover the building effectively (and I assume move it into the main build sequence).

You can add -G XCode to the first cmake command and it will create an XCode project

Yup. The focus from the start was to use Xcode projects as the primary driver for editing and building. It's critical that those of us working on MoltenVK internals every day have an effective Xcode environment. But again, I am not adverse to modernizing this and making it more command-buildable moving forward.

Keep in mind there are several build scripts, including the dylib build scripts, that will need to be integrated (or replaced) into any potential replacement build process.

As I mention, I am no CMake expert, and am busy with other priorities. So, that's a large project that I'll have to rely on other experts to tackle, and I can take a role of validating it against internal dev requirements.

Once this basic PR is ready to go, I'll pull it in, so we have some basics. Beyond that, I suggest we create both a tracking issue, and a branch to this repo that can absorb a larger and longer effort to move everything to CMake. We can validate there, and then merge to main once both internal and external requirements are effective.

spnda

Some minor things I noticed

CMakeLists.txt

MoltenVKShaderConverter/CMakeLists.txt

MoltenVK/CMakeLists.txt

TellowKrinkle · 2023-07-19T20:30:07Z

I'd recommend making a add_library(MoltenVK-Flags INTERFACE) putting all your include directories, compile options, link libraries, etc on that, then you can just target_link_libraries(MoltenVK-iOS PRIVATE MoltenVK-Flags) for each of the individual targets and it'll add all your stuff automatically

Formatting nitpick: most files on MoltenVK use either 4-space tabs or hard tabs (or both, it's a bit messy around here), but none use 8-space tabs. If you like 8-space tabs, hard tabs have the option to be 8 spaces while also fitting in with the rest of the project.

…e for a CMake project; but it will work for our needs.

billhollings

This PR has not been updated in the past year, and so it is structurally a bit out of date. I've suggested inline some changes that at least allow this PR to perform a MoltenVK build.

Having said that, I feel this functionality is far from complete:

As mentioned in the comments below, the builds are building for the wrong platform, or the wrong functionality, none of which are usable.
I expect that we will receive future requests to output dynamic frameworks, a macOS dylib, and possibly an XCFramework. All this will definitely be true for anyone using the Vulkan SDK, but wanting to build their own MoltenVK. Although we don't necessarily need those pieces right now, I want to have an action plan for how that might be accomplished.
We will definitely need some user documentation added to the README.md document to cover:
- How does the user invoke CMake?
- How is the user intended to integrate this into a build plan?
- If running standalone:
  - How does the user build MoltenVK?
  - Which target(s) should they build?
  - Where is the build output, and what should they do with it?

I remain concerned about the overall maintainability of all of the above if this becomes part of the officially-supported MoltenVK distro.

Another option might be to move this effort to a different repo somewhere. I'm thinking sort of a MoltenVK tools repo, where this could be maintained and managed separately from MoltenVK, by those users who need CMake integration. It could use an install script to put the files in this PR into the right locations within MoltenVK (or perhaps it could sit separately, and just import/reference the MoltenVK directories).

I've moved this PR back to WIP status, while we figure this out.

billhollings · 2024-03-27T20:15:39Z

External/CMakeLists.txt

+add_subdirectory(glslang)
+set(SPIRV_TOOLS_BUILD_STATIC ON)
+add_subdirectory(SPIRV-Cross)
+add_subdirectory(Vulkan-Headers)


The contents of External are not currently saved to repo. .gitignore needs to be adjusted so that External/CMakeLists.txt is added to the repo.

billhollings · 2024-03-27T20:17:56Z

MoltenVK/CMakeLists.txt

+        ${CMAKE_CURRENT_LIST_DIR}/MoltenVK/Commands/MVKCmdDraw.h
+        ${CMAKE_CURRENT_LIST_DIR}/MoltenVK/Commands/MVKCmdPipeline.h
+        ${CMAKE_CURRENT_LIST_DIR}/MoltenVK/Commands/MVKCmdQueries.h
+        ${CMAKE_CURRENT_LIST_DIR}/MoltenVK/Commands/MVKCmdRenderPass.h


Suggested change

${CMAKE_CURRENT_LIST_DIR}/MoltenVK/Commands/MVKCmdRenderPass.h

${CMAKE_CURRENT_LIST_DIR}/MoltenVK/Commands/MVKCmdRendering.h

billhollings · 2024-03-27T20:18:53Z

MoltenVK/CMakeLists.txt

+        ${CMAKE_CURRENT_LIST_DIR}/MoltenVK/Commands/MVKCmdDraw.mm
+        ${CMAKE_CURRENT_LIST_DIR}/MoltenVK/Commands/MVKCmdPipeline.mm
+        ${CMAKE_CURRENT_LIST_DIR}/MoltenVK/Commands/MVKCmdQueries.mm
+        ${CMAKE_CURRENT_LIST_DIR}/MoltenVK/Commands/MVKCmdRenderPass.mm


Suggested change

${CMAKE_CURRENT_LIST_DIR}/MoltenVK/Commands/MVKCmdRenderPass.mm

${CMAKE_CURRENT_LIST_DIR}/MoltenVK/Commands/MVKCmdRendering.mm

billhollings · 2024-03-27T20:19:15Z

MoltenVK/CMakeLists.txt

+        ${CMAKE_CURRENT_LIST_DIR}/MoltenVK/GPUObjects/MVKVulkanAPIObject.mm
+        ${CMAKE_CURRENT_LIST_DIR}/MoltenVK/Layers/MVKExtensions.mm
+        ${CMAKE_CURRENT_LIST_DIR}/MoltenVK/Layers/MVKLayers.mm
+        ${CMAKE_CURRENT_LIST_DIR}/MoltenVK/OS/CAMetalLayer+MoltenVK.m


Suggested change

${CMAKE_CURRENT_LIST_DIR}/MoltenVK/OS/CAMetalLayer+MoltenVK.m

${CMAKE_CURRENT_LIST_DIR}/MoltenVK/OS/CAMetalLayer+MoltenVK.mm

billhollings · 2024-03-27T20:22:36Z

MoltenVK/CMakeLists.txt

+
+add_library(MoltenVK-iOS STATIC
+        ${MOLTEN_VK_SOURCES}
+        )


The MoltenVK-iOS target builds a macOS binary.

billhollings · 2024-03-27T20:22:56Z

MoltenVK/CMakeLists.txt

+
+add_library(MoltenVK-tvOS STATIC
+        ${MOLTEN_VK_SOURCES}
+        )


The MoltenVK-tvOS target builds a macOS binary.

billhollings · 2024-03-27T20:24:02Z

MoltenVK/CMakeLists.txt

+
+add_executable(MoltenVK-macOS
+        ${MOLTEN_VK_SOURCES}
+        )


The MoltenVK-macOS target builds the MoltenVKShaderConverter tool, not a MoltenVK static library.

billhollings · 2024-03-27T20:33:58Z

MoltenVK/CMakeLists.txt

+        ${CMAKE_CURRENT_LIST_DIR}/include
+        )
+
+target_compile_features(MoltenVK-macOS PRIVATE cxx_std_17)


To get any of the MoltenVK-* targets to build, I also had to add the following build settings:

CURRENT_PROJECT_VERSION=1.2.9
GCC_PREPROCESSOR_DEFINITIONS=MVK_FRAMEWORK_VERSION=${CURRENT_PROJECT_VERSION}

billhollings · 2024-03-27T20:35:26Z

mvkGitRevDerived.h.in

@@ -0,0 +1,2 @@
+// Auto-generated by MoltenVK
+static const char* mvkRevString = "@MVK_GIT_REV@";


This file should be moved to a new Templates/cmake directory.

TellowKrinkle · 2024-03-30T05:49:04Z

You might also want to add a CI build for it so people know if break it, as they may not notice if they're using the xcodeproj locally.

zmarlon · 2024-04-14T15:16:27Z

What is still standing in the way of the merge? I would like to implement VK_NV_memory_decompression, but would be very reluctant to work without CMake, as I use Clion.

gpx1000 · 2024-04-14T18:39:40Z

I'm still working on getting the CI and ancillary projects working together. Basically me having free time is what's standing in the way of this getting merged. I use CLion and the branch as it stands is good enough to get a MoltenVK built using CLion.

MarkCallow · 2024-05-03T07:54:52Z

git clone --recurse-submodules -j8 https://github.com/KhronosGroup/MoltenVk
mkdir -p MoltenVk/build && cd MoltenVk/build
cmake ..
cmake --build ..

There is no need for the mkdir step. CMake will happily create the build directory for you. What I do is

git clone --recurse-submodules -j8 https://github.com/KhronosGroup/KTX-Software
cmake -G XCode -B build
cmake --build build

You can add -G XCode to the first cmake command and it will create an XCode project complete with the dependencies linked correctly and ready to be built as needed. Otherwise, you can use -G ninja or -G Makefile for those command line build systems.

You can easily use the Xcode projects for command line builds too, as shown by my example above. You can append

[--config <build config>] [--target <some target>]

to the last command to set those aspects of the build. You can even set Xcode specific build options by appending them to the end of the command separated by --. E.g,

cmake --build build -- CODE_SIGN_IDENTITY= CODE_SIGN_ENTITLEMENTS= CODE_SIGNING_REQUIRED=NO CODE_SIGNING_ALLOWED=NO

Now, let's say you're a consumer of MoltenVK, instead of downloading a built library, you want to work with MoltenVK within your project. Here's all you'd have to do with a CMake project:
git submodule add https://github.com/KhronosGroup/MoltenVk
git submodule update --init --recursive <-- only needed if using submodules instead of fetchDependencies, if fetchDependencies then run it here instead.
in your CMakeLists.txt:
add_subdirectory(MoltenVk)
target_link_libraries(myexectuablename PUBLIC moltenVk)

...
The cool thing there is their entire build process can also use cpack which will create an installer and package things in OS independent way

Alternatively if users want to build using the VulkanSDK, the SDK can provide a CMake find module that would be, in part, generated by the CMake install commands that underlie the mentioned cpack. I say in part, because of all the other pieces of the Vulkan SDK. Users use this in their project via something like find_package(Vulkan) in their own CMakeLists.txt. It will provide the correct include paths, library paths and link commands.

Long ago, we used submodules, but moved to a script to provide options to manage the build complexity. But I am not adverse to moving back to submodules for the dependencies, if we can cover the building effectively (and I assume move it into the main build sequence).

Submodules are more convenient for users wishing to include MoltenVK into their own projects as a submodule. If that is not a consideration you can continue with the script. It can be called from the install section of a .travis.yml or from an install step in a GitHub Action .yml file to install all the dependencies on the CI runner before starting the build. To use with CMake it would have to be run before the CMake configure step (the first cmake command in the examples) so the dependencies are there to be found during configuration.

Yup. The focus from the start was to use Xcode projects as the primary driver for editing and building. It's critical that those of us working on MoltenVK internals every day have an effective Xcode environment. But again, I am not adverse to modernizing this and making it more command-buildable moving forward.

I remain concerned about the overall maintainability of all of the above if this becomes part of the officially-supported MoltenVK distro.

Mac is my primary development environment and I only use Xcode for developing KTX-Software. We use CMake to generate the projects and we build for iOS, iPadOS and macOS. The CMake-generated projects provide a completely effective Xcode environment. The CMakeLists.txt and subsidiary files are the only build files we maintain.

We will definitely need some user documentation added to the README.md document to cover:

How does the user invoke CMake?

How is the user intended to integrate this into a build plan?

If running standalone:

How does the user build MoltenVK?

Which target(s) should they build?

Where is the build output, and what should they do with it?

Most of these things are well known to CMake users. In KTX-Software we have a BUILDING.md that gives all the information needed about building including some example cmake commands. (Don't be put off by the length of it. This is due to the complexity of things like dealing with signatures, that it covers multiple platforms and that it lists all the dependencies for each platform individually instead of just saying use the install_* script for your platform.)

MarkCallow · 2024-05-03T08:00:40Z

Another option might be to move this effort to a different repo somewhere. I'm thinking sort of a MoltenVK tools repo, where this could be maintained and managed separately from MoltenVK, by those users who need CMake integration. It could use an install script to put the files in this PR into the right locations within MoltenVK (or perhaps it could sit separately, and just import/reference the MoltenVK directories).

This will ultimately end up being a lot more work than adopting CMake to build MoltenVK. The sole advantage is that @billhollings will not have learn CMake. 😄

mokafolio · 2024-09-10T00:56:04Z

I'd love to see this take shape. Is the main remaining issue that the wrong targets are being built?

zmarlon · 2024-10-13T10:48:12Z

Good question, im always looking forward to this PR. Because AppCode got canceled and I really want wo work in a JetBrains IDE :)

CMake only PR. This allows other tooling such as CLion to work with b…

267c318

…uilding this project. It also allows for building with XCode directly. It should make maintaining the project easier in the long run.

billhollings requested changes Jun 15, 2023

View reviewed changes

spnda reviewed Jun 26, 2023

View reviewed changes

CMakeLists.txt Outdated Show resolved Hide resolved

MoltenVKShaderConverter/CMakeLists.txt Outdated Show resolved Hide resolved

MoltenVK/CMakeLists.txt Outdated Show resolved Hide resolved

MoltenVK/CMakeLists.txt Outdated Show resolved Hide resolved

billhollings changed the title ~~CMake only PR.~~ WIP: CMake only PR. Jun 26, 2023

IsaacMarovitz mentioned this pull request Nov 21, 2023

WIP: Transform feedback #1943

Draft

reckenrode mentioned this pull request Jan 27, 2024

Build failure: darwin.moltenvk 1.2.4 NixOS/nixpkgs#284281

Closed

gpx1000 added 3 commits March 18, 2024 17:19

Merge branch 'main' into support_CMake_building

488cedd

update cmake to use GLOB as per request. This is NOT the best practic…

6505227

…e for a CMake project; but it will work for our needs.

accept requested changes.

c8ab021

gpx1000 changed the title ~~WIP: CMake only PR.~~ CMake only PR. Mar 19, 2024

billhollings changed the title ~~CMake only PR.~~ WIP: CMake only PR. Mar 27, 2024

billhollings requested changes Mar 27, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: CMake only PR. #1947

WIP: CMake only PR. #1947

gpx1000 commented Jun 14, 2023

billhollings left a comment

billhollings Jun 15, 2023

gpx1000 Jun 15, 2023

billhollings Mar 27, 2024

gpx1000 commented Jun 15, 2023

billhollings commented Jun 15, 2023

spnda left a comment

TellowKrinkle commented Jul 19, 2023

billhollings left a comment

billhollings Mar 27, 2024

billhollings Mar 27, 2024

billhollings Mar 27, 2024

billhollings Mar 27, 2024

billhollings Mar 27, 2024

billhollings Mar 27, 2024

billhollings Mar 27, 2024

billhollings Mar 27, 2024

billhollings Mar 27, 2024

TellowKrinkle commented Mar 30, 2024

zmarlon commented Apr 14, 2024

gpx1000 commented Apr 14, 2024

MarkCallow commented May 3, 2024 •

edited

Loading

MarkCallow commented May 3, 2024 •

edited

Loading

mokafolio commented Sep 10, 2024

zmarlon commented Oct 13, 2024

		@@ -0,0 +1,2 @@
		// Auto-generated by MoltenVK
		static const char* mvkRevString = "@MVK_GIT_REV@";

	${CMAKE_CURRENT_LIST_DIR}/MoltenVK/Commands/MVKCmdRenderPass.h
	${CMAKE_CURRENT_LIST_DIR}/MoltenVK/Commands/MVKCmdRendering.h

	${CMAKE_CURRENT_LIST_DIR}/MoltenVK/OS/CAMetalLayer+MoltenVK.m
	${CMAKE_CURRENT_LIST_DIR}/MoltenVK/OS/CAMetalLayer+MoltenVK.mm

WIP: CMake only PR. #1947

Are you sure you want to change the base?

WIP: CMake only PR. #1947

Conversation

gpx1000 commented Jun 14, 2023

billhollings left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gpx1000 commented Jun 15, 2023

billhollings commented Jun 15, 2023

spnda left a comment

Choose a reason for hiding this comment

TellowKrinkle commented Jul 19, 2023

billhollings left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TellowKrinkle commented Mar 30, 2024

zmarlon commented Apr 14, 2024

gpx1000 commented Apr 14, 2024

MarkCallow commented May 3, 2024 • edited Loading

MarkCallow commented May 3, 2024 • edited Loading

mokafolio commented Sep 10, 2024

zmarlon commented Oct 13, 2024

MarkCallow commented May 3, 2024 •

edited

Loading

MarkCallow commented May 3, 2024 •

edited

Loading