Update cmake #275

chillenzer · 2025-01-15T15:00:31Z

This is a major reorganisation of the repository including setting up proper CMake support. It is strongly inspired by ModernCppStarter with some notable changes resulting from discussion with and preferences of @psychocoderHPC. As kind of a byproduct, also includes some modern features that are however not yet tweaked to production quality.

This PR is tested with KitGenBench and picongpu. Hint for the review: The first commit removes alpaka and is quite unwieldy. Viewing the diff from the second commit on shows that it's really just cmake stuff and files moved.

I'll try to summarise the new layout and features:

The main directory now provides a CMakeLists.txt that provides the library and optionally includes tests and examples via mallocMC_BUILD_TESTING/EXAMPLES.
Directories containing source code are include, examples and test. The latter two are self-contained cmake subdirectories that can be built independently.
All external source code is removed from the repository (except for a patched version of CPM).
CPM is used for adding dependencies. This means that given internet access the library is fully self-contained and can download and build all dependencies itself if it must.
Fine-grained control over the dependencies' origin is provided by variables of the form mallocMC_USE_... allowing ON (no downloading), ON_ALLOW_FETCH (default if required), ON_ALWAYS_FETCH (for best reproducibility), AUTO (default for optional dependencies) and OFF. Alternatively, the variable is allowed to contain a path pointing to the source location for this dependency.
All details about the dependencies are specified in cmake/package-lock.cmake. Only ever touch this, if you want to change a version or the like. (Updating alpaka should be significantly less painful now.)
Without internet connection one can simply download dependencies on another computer specifying the -DCPM_SOURCE_CACHE=/some/convenient/path, copy that to the offline system and point the same variable there for the build -DCPM_SOURCE_CACHE=/where/you/copied/the/source/cache. This is probably the closest we can get to offline support without shipping external code.
In copying over from ModernCppStarter, we acquired a number of tools including platform-independent sanitizer, static analysis and code coverage support. At the time of writing, this is not yet properly configured and would need some tinkering with to produce useful output. (Currently, it can be run but most warnings I've seen are about alpaka and Catch2. It's hard to say at this point if there are useful messages about mallocMC among the noise.)

psychocoderHPC

mostly variables without prefix mallocMC_ were exposed.

psychocoderHPC · 2025-01-17T14:43:42Z

.github/workflows/ci.yml

@@ -1,5 +1,8 @@
 name: Continuous Integration
 on: [push, pull_request]
+env:
+  CPM_SOURCE_CACHE: ${{ github.workspace }}/cpm_modules


out of interest: Is this variable also required in normal workflows or will the cache by default places somewhere relative to the build folder?

Offline discussion: This is not required but recommended. If not set, CPM will redownload for every new build directory. The downloaded files will be put into a local folder inside the build directory (if I recall correctly). The discussion led to another comment: We should document the option to not cache unstable dependencies (like the alpaka dev branch).

psychocoderHPC · 2025-01-17T14:45:05Z

.github/workflows/ci.yml

@@ -18,26 +21,25 @@ jobs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
+      - uses: actions/cache@v3


why is all this configuration required?

Not required but supposed to slightly speed up CI runs by caching downloads. The CI here is only provisional at this point anyways. But concerning caching we should document the option to explicitly disable caching for unstable dependencies.

psychocoderHPC · 2025-01-17T14:54:26Z

CMakeLists.txt

+  NAMESPACE ${PROJECT_NAME}
+  BINARY_DIR ${PROJECT_BINARY_DIR}
+  INCLUDE_DIR ${PROJECT_SOURCE_DIR}/include
+  INCLUDE_DESTINATION include/${PROJECT_NAME}-${PROJECT_VERSION}


after installing, are all files in include/malloc-3.0.0/mallocMC/*?

psychocoderHPC · 2025-01-17T14:58:24Z

cmake/tools.cmake

+
+# enables sanitizers support using the the `USE_SANITIZER` flag available values are: Address,
+# Memory, MemoryWithOrigins, Undefined, Thread, Leak, 'Address;Undefined'
+if(USE_SANITIZER OR USE_STATIC_ANALYZER)


the variables should have the prefix mallocMC_

psychocoderHPC · 2025-01-17T14:58:59Z

cmake/tools.cmake

+endif()
+
+# enables CCACHE support through the USE_CCACHE flag possible values are: YES, NO or equivalent
+if(USE_CCACHE)


should be mallocMC_USE_CCACHE

psychocoderHPC · 2025-01-17T15:03:10Z

examples/vectorAdd/CMakeLists.txt

+
+# --- Import tools ----
+
+include(../../cmake/tools.cmake)


should all ../.. start with ${CMAKE_CURRENT_LIST_DIR}/../..?

psychocoderHPC · 2025-01-17T15:06:36Z

test/multithreaded/CMakeLists.txt

+
+# --- Import tools ----
+
+include(../../cmake/tools.cmake)


maybe ${CMAKE_CURRENT_LIST_DIR}/../.. ?

psychocoderHPC · 2025-01-17T15:09:24Z

test/unit/CMakeLists.txt

+
+# ---- Options ----
+
+option(ENABLE_TEST_COVERAGE "Enable test coverage" OFF)


Those variables should start with mallocMC_ to clearly show the users that mallocMC behaviour is changed.
This will also group all variables of mallocMC which helps finding all possible cmake switches.

chillenzer force-pushed the update-cmake branch 6 times, most recently from f677e22 to d09d548 Compare January 16, 2025 07:45

chillenzer marked this pull request as ready for review January 16, 2025 07:51

chillenzer requested a review from psychocoderHPC January 16, 2025 07:51

psychocoderHPC added the refactoring label Jan 17, 2025

psychocoderHPC added this to the 3.0.0 milestone Jan 17, 2025

psychocoderHPC requested changes Jan 17, 2025

View reviewed changes

Third Party and others added 19 commits January 20, 2025 15:11

Remove alpaka

527d022

Add all and cmake

9a6b18d

Move include to toplevel

15f3dcd

Update toplevel CMakeLists.txt

9be648a

Remove thirdParty

021e643

Unused variables

2e4defe

Update test folder

e1de0e0

Update examples

0ebf37f

Remove docs

48554f1

Fix nvcc Werror interface

168a79e

Fix nvcc Werror interface 2

2598926

Remove all/

efaef82

Use add_controlled

b67fd87

Tweak package-lock

43de18a

Remove unnecessary excludes

23088db

Update CI to new build system

c8f1927

Update version

04f89aa

Guard against re-adding an already exisitng target

6518a98

Add comment about disabling caching

c714794

chillenzer added 3 commits January 20, 2025 15:13

Be explicit about CMAKE_CURRENT_LIST_DIR everywhere

cd5bb15

Prefix all cmake variables with project name

0c81ba3

Add comments to shortcut custom targets

8a80fb1

chillenzer force-pushed the update-cmake branch from 5680693 to 8a80fb1 Compare January 20, 2025 14:13

chillenzer mentioned this pull request Jan 22, 2025

Add gallatin cuda after reorganisation #277

Draft

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update cmake #275

Update cmake #275

chillenzer commented Jan 15, 2025 •

edited

Loading

psychocoderHPC left a comment

psychocoderHPC Jan 17, 2025

chillenzer Jan 20, 2025

psychocoderHPC Jan 17, 2025

chillenzer Jan 20, 2025

psychocoderHPC Jan 17, 2025

psychocoderHPC Jan 17, 2025

psychocoderHPC Jan 17, 2025

psychocoderHPC Jan 17, 2025

psychocoderHPC Jan 17, 2025

psychocoderHPC Jan 17, 2025


		# ---- Options ----

		option(ENABLE_TEST_COVERAGE "Enable test coverage" OFF)

Update cmake #275

Are you sure you want to change the base?

Update cmake #275

Conversation

chillenzer commented Jan 15, 2025 • edited Loading

psychocoderHPC left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chillenzer commented Jan 15, 2025 •

edited

Loading