Skip to content

Developing TileDB in CLion

Jump to bottom
Konstantinos Tsitsimpikos edited this page Nov 12, 2020 · 9 revisions

Developing TileDB in CLion

CLion is an IDE for C++ development that has good support for CMake-based project. However, it does not handle very well the CMake structure used by TileDB, referred to as a "superbuild" (see this issue).

Superbuild structure

In the superbuild, all of the dependencies of TileDB are managed using CMake external projects. This allows us to automatically and portably install TileDB dependencies without relying on external package managers, or requiring superuser privileges.

During the initial configuration phase, when you first invoke cmake, we check for the dependencies in standard system locations. If a dependency is not found, it is added as an external project (EP). We also add TileDB itself as an EP, to allow us to express a dependency graph across EPs (the TileDB EP depends on the dependency EPs). Next, when you start the build by invoking make or cmake --build ., any EPs that were added are first downloaded, configured, built and installed in the TileDB build directory.

The TileDB EP is processed last. The TileDB EP simply invokes cmake inside the tiledb build directory, as there is no download step. This configures and starts the build for TileDB itself, now that the dependencies have been installed in a known location.

Configuring CLion

CLion does not know about the build targets within the TileDB external project. Therefore, when you first set up the TileDB project in CLion, you should follow these steps:

  1. Run the default Build All target once in the Debug profile. This will invoke the superbuild and build the dependencies. If you don't have a Build All target you can create one via Edit Configurations -> Application -> Add new configuration.
  2. Edit the CMake "Profile" in CLion for the profile you just built (Debug) and add the CMake build option -DTILEDB_CMAKE_IDE=ON. This will disable the superbuild for the Debug configuration, which will cause the normal TileDB build targets to be available to CLion.
  3. Repeat this process as required for the other profiles (like Release).

Running the unit tests in CLion

CLion has special support for tests using the Catch framework. Follow these steps:

  1. Edit your run configurations (Run menu -> Edit configurations).
  2. Add a new "Catch" configuration by clicking the "+" button.
  3. Choose the tiledb_unit target for the Catch configuration.

If -DTILEDB_HDFS=ON

  1. (macOS Only) If building / testing against HDFS, enable JVM signal chaining for Catch:
  • Add environment variable: DYLD_FORCE_FLAT_NAMESPACE=1

Note: Catch should automatically properly link to the libjsig library during the build process

Now you should be able to run and debug the unit tests within CLion.

Developing special configurations

If the build configuration you're developing requires a specific build option (e.g. TILEDB_S3 or TILEDB_SERIALIZATION) you may run into the issue that CLion does not pick up the corresponding preprocessor definitions. This means that portions of code with #ifdef TILEDB_SERIALIZATION will be marked as unreachable by CLion, even if you set the option -DTILEDB_SERIALIZATION=ON during configuration time. Code highlighting, syntax checking, etc. will be disabled in these regions.

Method 1

The method to fix this is to manually define the corresponding definition on the toplevel CMakeLists.txt file. E.g. for serialization, edit TileDB/CMakeLists.txt and below the section with all of the option(...) lines, add the explicit definition:

# Don't commit this!
set(TILEDB_SERIALIZATION ON)
add_definitions(-DTILEDB_SERIALIZATION)

Then reload the project in CLion with Tools -> CMake -> Reset Cache and Reload Project.

Method 2

A second method can be used in case that the build configuration you're developing requires a specific build option (e.g. TILEDB_S3 or TILEDB_SERIALIZATION). It will require though some manual steps, but it will save you from modifying toplevel CMakeLists.txt file.

  1. Create a directory for your build. mkdir build
  2. Jump into the directory you just created cd build
  3. Run the bootstrap script to create a superbuild of the project ../bootstrap .. && make -j4. This will install all the external packages
  4. Create a debug profile File -> Settings -> Build, Execution, Deployment -> Cmake and point the Build directory to your build directory
  5. In the CMake Options placeholder you can now insert -DCMAKE_BUILD_TYPE=Debug reassuring that you will be in Debug mode and any other options that you might need e.g -DTILEDB_SERIALIZATION=ON accompanied with -DTILEDB_CMAKE_IDE=ON needed for Code highlighting, syntax checking, etc..
  6. Let the Cmake build again the project
  7. Portions of code with #ifdef TILEDB_SERIALIZATION should now be reachable.

You can reload the project in CLion with Tools -> CMake -> Reset Cache and Reload Project, but this step is optional and only for making sure that indeed works as it should.

Remote development

It's possible to set up a Clion project to remote deploy / debug a project on external servers using your local Clion IDE. This is useful for building / deploying / testing on beefier servers on EC2 for instance.

See the JetBrains documentation on how to set this up: https://www.jetbrains.com/help/clion/remote-development.html

Note: It is still required to use the Clion superbuild workaround described above.

Clone this wiki locally