Integration Quickstart Guide

Integration Quickstart Guide

projectM is released as either a shared or static library and provides a C-based API for all exposed functionality. This means that developers can use projectM in any application or with any programming language that supports calling native C functions and provide a proper OpenGL rendering context, including the platform-specific drawing surface.

The quickstart guide explains how to get started with projectM in a C/C++ application. If you need to use it in other languages, the best approach would be writing a small C-to-your-language layer, which will be very similar to using it directly in a C application. Since there are many different programming languages out there, we cannot cover them all

No operating system specific details like creating an OpenGL context are handled in this guide. libprojectM and its API are generally platform-agnostic and require the application to provide the rendering environment.

Preparing your application to render projectM visuals

projectM uses OpenGL to render its visuals. As of version 4.1, it renders to the thread's current default framebuffer and native surface. Before initializing or rendering with projectM, the application must:

• Create a native window/rendering surface using host OS APIs.
• Initialize an OpenGL context and connect it to the window/surface.
• Make the OpenGL context which should be used to render the projectM visuals current.
• Not use different threads to initialize projectM and call the projectm_render_frame() method.

Since OpenGL initialization is highly OS- and application-dependent, projectM will not perform any of those operations. Since implementing with the OS-native APIs directly is often very complicated, using a wrapper library such as GLFW or SDL if highly recommended. Both libraries provide plenty of examples on how to create OpenGL contexts. Qt can also create OpenGL sufaces, but additional care must be taken to use the proper rendering backend.

Adding libprojectM to your application

The recommended way to integrate projectM into your application is by building libprojectM as a static or shared library and use the public C API (by including projectM.h) to instantiate and control it. This is the recommended way to use projectM. While the exposed functionality is limited to the public API libprojectM provides, you can easily update to recent releases without effort. If libprojectM is linked as a shared library, updating is as easy as replacing the library file - you don't even need to recompile your application.

If you need more control over projectM's internals, for example if you're building an advanced preset editor for which you require access to projectM's internal parser or rendering code, you can also directly integrate the source files into your application. Be aware that this comes with a more involved process of updating to newer libprojectM releases.

Also be aware that either statically linking libprojectM or directly including the sources into your codebase requires your application to be released under the GPL or LGPL licenses.

This integration guide will only cover integration via the official C API and linking the static or shared library.

Build the library

First, download and build libprojectM and install the build results in a location where you can access them easily.

Add the libraries and include dirs

Using CMake

The recommended way to use libprojectM is by using CMake to build your application. In your CMakeLists.txt, you need to find the libprojectM package and then link the correct library target to your executable or library:

# After the project() command.
# In the case projectM is optional in your build, leave out the REQUIRED.
# If you need a specific libprojectM version, you can specify it after the package name,
find_package(projectM4 REQUIRED)

add_executable(MyApp
    main.cpp
)

target_link_libraries(MyApp
    PRIVATE
    libprojectM::projectM
)

That's all. If CMake finds libprojectM, it'll link the correct library and also add any required include dirs and compiler flags to your application's build configuration. To have CMake find your copy of libprojectM, you might need to add it via CMAKE_PREFIX_PATH:

cmake -S /path/to/source \
      -B /path/to/build \
      -GNinja \
      -DCMAKE_PREFIX_PATH=/path/to/libprojectM

If libprojectM is installed in a standard path your toolchain is configured to look into, you won't need it.

To add other dependencies in non-standard search paths, append them to CMAKE_PREFIX_PATH separated by semicolons. If any path contains a space, put quotes around the paths.

Using pkgconfig for other build systems

If you can't or don't want to use CMake, you need to gather the library and include dirs on your own and add them manually to your project configuration.

On Linux and macOS, libprojectM will also create a libprojectM.pc file for use with pkgconfig, which will be the tool of choice for autotools-based builds for example. If you want to use the debug library, use libprojectM-debug as the pkgconfig package name instead.

The optional playlist library can be found using either libprojectM-playlist or libprojectM-playlist-debug, respectively.

Call projectM in your code

One word about memory allocation

Before we start using the API, it is important to know how the API works in regard to memory allocation. The C API is technically a C wrapper around C++ code. So even if you call C functions, they internally execute as C++ code in the library. In addition to that, the library, if loaded as a shared/dynamic library and depending on the platform, might use a different heap area and C/C++ runtime than your host application.

To make sure all allocated memory is properly being disposed of after use, it must be freed in the same context where it has been allocated: if your application reserved memory, it has to release it. If libprojectM reserved memory, it must be released in the library code.

Currently, this applies to all strings returned by projectM's getter functions. As a rule of thumb, here is a quick reference:

• Returned data can be freed at any time, even after destroying any projectM/playlist instances.
• If a pointer (e.g. char*) is returned by an API function, always use the appropriate projectm_free_string() function if you're done using the data.
• If your code passes any self-allocated pointer to the API, it is safe to free the allocated memory immediately after the call. Do not use the projectm_free_string() function on these pointers.
• It is safe to pass temporary pointers to any API call, e.g. using std::string::c_str() in an argument. the functions will always make a copy of the contents if needed.
• You can use projectm_alloc_string to allocate memory for strings, but you must then use projectm_free_string to free it after use.
• Any data pointers passed to callbacks are only valid until the callback returns. If you need the data afterwards,.make a copy. Your code must not call free or delete on the passed data pointers inside the callback.
• Your application has to make sure that registered callbacks (and the user_data pointer) always reference allocated memory and existing functions.
• The playlist library is separate from the core projectM library and comes with its own free functions. Use those for all projectm_playlist_*() API return values.
• The playlist library can return arrays of strings (char**). Call projectm_playlist_free_string_array() to free those pointers.

Create a new projectM instance

Now that you have your project files configured properly, you can immediately start using projectM. As stated above, this guide assumes your application already takes care of creating a proper OpenGL rendering context. This must be done before calling any projectM functions.

First, include the API header and create a new projectM instance:

#include <projectM-4/projectM.h>

/* In your setup code: */

/* Create a projectM instance with default settings */
projectm_handle projectMHandle = projectm_create();
if (!projectMHandle)
{
    /* Something went wrong, most probably OpenGL isn't configured. */
}

The opaque handle returned by projectm_create() identifies your instance and must be used as the first parameter to all API calls that have an instance parameter.

It is safe to include the header from both C and C++ files. It wraps extern "C" around declarations automatically.

Make sure to clean up the instance

Now is a good time to make sure the newly created instance is deleted after your application is done using it:

/* In your shutdown code: */
projectm_destroy(projectMHandle);
projectMHandle = NULL;

If you make sure to set projectMHandle to NULL after destroying it, any further calls to projectm_destroy() will simply be a no-op: it is safe to pass a null pointer.

Note: It is not safe to pass NULL or already-destroyed instance handles in instance to any other API call! Make sure your code doesn't do that.

Set the canvas size

Once your rendering context is ready and the dimensions of the target surface are known, you must provide the size to projectM once after initializing, and again every time the surface was invalidated or changed size, e.g. after a windows was resized, minimized and restored:

/* Initialize these with your current surface dimensions. */
size_t renderWidth = 800;
size_t renderHeight = 600;

projectm_set_window_size(projectMHandle, renderWidth, renderHeight);

For performance reasons, make sure to only call projectm_set_window_size() if really needed, as this will currently cause a full recreation of projectM's internal rendering pipeline, including shader compilation.

Render a frame

projectM is now ready to start rendering frames. This needs to be done in your application's rendering loop:

projectm_render_frame(projectMHandle);
/* Swap buffers here */

libprojectM does not have any FPS limiting capabilities. This means you can render frames as fast as projectM can draw them. In end-user applications, this might not be a good thing as it will fully utilize one or more CPU cores while the display possibly cannot display frames at the same speed. By enabling VSync for buffer swaps, it will automatically limit FPS to the refresh rate. You might further consider FPS limiting to a certain target framerate to safe resources on the user's device.

Supplying audio data

With the above setup, the application will only render the default idle preset (the wandering "M" logo), but not do anything fancy. To make it react to some audio playing, the application must pass audio sample data into the library.

Where the application sources the audio data is up to the actual implementation, e.g. capturing external audio via some system API, directly decoding an audio file or using data from an underlying player application.

The API currently supports a few different data formats. All functions start with projectm_pcm_add_, followed by the sample data type, the number of channels and the data structure type accepted to pass in the actual data. For best performance and visuals, it is recommended to always use the projectm_pcm_add_float_2ch_data() function. It requires your data to be in this format:

• 32 bit floating-point samples
• 2 channels (LR)
• A simple data pointer, pointing to the first data byte
• The number of samples

The actual sample frequency is not part of the interface, but projectM is optimized for 44.1 kHz, same as Milkdrop. It will also work with other sample frequencies, but the beat detection and presets drawing spectrum data might not behave as expected. Using 48 kHz is fine though as the difference is minimal.

The number of samples is the count of a full complement of data for all channels. This means if sample_count is 1, then the data must at least contain 16 bytes or 2 floats (General formula: numBytes = sizeof(sample data type) * channels).

Ideally, if the application is gathering audio data in a separate thread than the renderer, it should not pass audio data only while projectm_render_frame is running. libprojectM does not use mutexes internally to prevent mutual access to the audio buffer. If there are race conditions, it won't cause crashes though, but may negatively impact the rendered visuals.

With all that said, let's say your application has a wrapper function that gets properly formatted audio data as a basic byte (unsigned char*) buffer:

/* audio_data is passed in float/2ch format */
void add_audio_to_projectm(const unsigned char* audio_data, unsigned int length)
{
projectm_pcm_add_float(projectMHandle, (const float*)audio_data, length / sizeof(float), 2 /* channels */);
}

Now projectM should start reacting to the audio.

Using the playlist library

The core projectM library uses callbacks to inform the application if it wants a new preset, e.g. if the display duration has timed out. The application can then load another preset.

If only a simple playlist management is required, the playlist library implements an easy-to-use interface to do so. an application can create any number of playlist instances, filling them with different preset files. However, only one playlist can be connected to a projectM instance at any time. Only the connected playlist will receive the callbacks from the core library to switch to the next preset. If the application has previously registered the callbacks, it will also no longer receive the calls after connecting a playlist. A playlist can be disconnected either by connecting another playlist, or setting the preset_switch_requested and preset_switch_failed callbacks in the projectM instance to a different pointers.

To add the playlist library and include paths, change the find_package() call in your CMakeLists.txt to the following and remember to link the libprojectM::playlist CMake target to the application to prevent undefined symbol errors:

find_package(projectM4 REQUIRED COMPONENTS Playlist)

...

target_link_libraries(MyApp
    PRIVATE
    libprojectM::playlist
)

The core projecTM library will be linked automatically as a dependency. You can now include the playlist header, create an instance after initializing projectM and fill it with presets:

#include <projectM-4/playlist.h>

...

/* Create a projectM playlist and immediately connect it to the projectM instance */
projectm_playlist_handle playlistHandle = projectm_playlist_create(projectMHandle);

/* Add a preset path recursively, don't allow duplicates */
projectm_playlist_add_path(playlistHandle, "/path/to/presets", true. false);

/* Load a random preset in the playlist */
projectm_playlist_set_shuffle(playlistHandle, true);
projectm_playlist_play_next(playlistHandle, true);

/* Create another, unconnected projectM playlist */
projectm_playlist_handle playlistHandleUnconnected = projectm_playlist_create(NULL);

/* Swap playlists */
projectm_playlist_connect(playlistHandleUnconnected, projectMHandle);
projectm_playlist_connect(playlistHandle, NULL);

Implement and register the callbacks defined in the playlist.h file to react to preset switches and failures.

‼️ Make sure to not overwrite the preset switch callbacks provided by the projectM core library (callbacks.h), as doing so will prevent the playlist manager from switching presets. If your application needs to temporarily receive these callbacks, make sure to call projectm_playlist_connect() again to reconnect the playlist manager to the projectM instance.

Audio data caveats

The application does not need to care about how much data is stored inside projectM's internal buffer. If more data is added than projectM can consume on each frame, it is implemented as a ring buffer and will simply be overwritten. Each frame only renders the last added audio samples available in the buffer. This in turn means that if audio data is added only sporadically in large batches, multiple frames will use the same audio data for rendering, effectively "freezing" waveforms on screen.

Depending on the frequency and amount of audio data the application gets on each update (e.g. in callbacks from an external audio driver), it might be necessary to spread out adding the data over multiple frames instead of adding it all at once directly in the callback. Knowing the sample frequency and actual frame rendering time, the application can calculate the number of new audio samples to pass to projectM before each frame. If audio data comes in faster than frames are rendered, only the last few samples of the available audio data need to be passed. The application can query projectM's internal sample buffer size with the projectm_pcm_get_max_samples() function for optimization.

In addition to all that, the application should make sure not to get too much behind the actually played audio, as it might introduce visible lag between what the user hears and what projectM renders. As a rule of thumb, latencies over 35ms will potentially be noticeable by the user.

What next?

Now that the application is able to render visualizations with projectM, it should also take care of configuring additional features like adding presets, properly setting the different options projectM supports and handling user input as required.

projectM also supports loading presets from a text buffer instead of local files. This is useful if the preset contents aren't stored locally, but streamed/loaded from other sources like the network or ZIP files. Use the projectm_load_preset_data() instead of projectm_load_preset_file() to pass in the preset data. For now, preset data is assumed to be in Milkdrop preset format.

Frequently asked questions

Here are a few questions we get asked frequently on GitHub and Discord, regarding projectM integration into applications. This Q&A section is valid for projectM release 4.1.

How do I build projectM?

Please see the build documentation. You may also be able to find projectM binaries (or build scripts) in other package managers of your preferred language/ecosystem and use those instead of building projectM yourself.

I don't know how to use (or don't want to/cannot use) CMake. Is there an alternative?

Since version 4.0, projectM's build system is based on CMake. It takes care of all build checks, creating the native build system files (e.g. Makefiles) and also installs the compiled libraries along with the include files. this means at least to build projectM, you'll need CMake. It is of course possible to build projectM in your preferred build system, but you need to port the build yourself and then maintain it. The projectM team won't be able to support in this case.

To integrate projectM into your own application, it is recommended to use CMake, but it's not mandatory. On UNIX-based systems, the projectM install dir will also contain PkgConfig (.pc) files which can be used to find and link projectM, for example with GNU autotools-based (M4) builds. To use libprojectM in any other build system, you have to do the integration yourself, e.g. write the proper detection scripts or add the correct include/library paths, compiler and linker flags to your project. In this case, make sure to check for any changes between projectM releases.

Is it possible to render presets at a slower/higher speed?

Preset rendering speed is influenced mainly by the preset code itself. How each value is incremented between frames depends on how the preset author wrote the code, so except for changing the frame rate, there's no way to change preset "speeds". Additionally, projectM keeps an internal timer which is based on the system time and provides the value to presets, so authors can also use this time value to pace their calculations. The time passed to presets can currently not be modified from the outside, and will always reflect the time passed since starting projectM or the preset. This means rendering presets at a different real framerate than the target one (e.g. rendering at 30 FPS into a move clip while targeting 60 FPS video framerate) will make some presets run faster or slower.

The projectM team has added an API method in version 4.2 to provide the time value from the outside, replacing the system time entirely. This enables presets render at a targeted frame rate, no matter how fast or slow the actual render/encode process is in real time.

Can I run multiple projectM instances in a single application?

This is absolutely possible. There are a few caveats due to OpenGL restrictions which have to be considered though:

• All projectM function calls must be done from the same thread used to create the OpenGL context and creating the projectM instance.
• When using multiple OpenGL contexts, e.g. when rendering to multiple windows, always make sure to activate ("make current") the correct OpenGL context for the projectM handle passed to the respective projectM API function call.

When using multiple, unrelated GL contexts, each context/projectM instance can run in a separate thread. projectM only uses very few statically allocated objects, which are all considered to be thread-safe. All API calls are fully reentrant.

Is it possible to record other application's audio output on macOS?

Sadly, no. MacOS heavily restricts audio recording and only allows applications to record from external sources the user has allowed the application to use, like a microphone, line-in jacket or a USB sound card.

Users can install additional kernel drivers to record application audio and then loop it back using a virtual recording device. These drivers aren't free though and since Apple is currently deprecating all user-space kernel extensions, this loophole will probably be entirely unavailable in a future macOS release.

Does projectM run on the Raspberry Pi?

On some, sort of - projectM requires the OpenGL ES 3.0 standard on embedded platforms like the Pi or smartphones. Due to hardware and software limitations, projectM will only run at decent speeds on at least a Raspberry Pi 4. The Pi 3 may be able to run projectM, but the actual frame rate will not be inside a useful range.

Even on the Raspberry Pi 4, some precautions need to be taken to get a decent frame rate:

• Render in 720p at most, then scale the image to the native resolution.
• Reduce the per-vertex mesh resolution to a small value, like 48x32, which is Milkdrop's default.
• Pick only presets without shaders (those without any warp_ and comp_ lines), or test each preset for shader performance.
• Don't use CPU-heavy presets with complex math calculations. This one is hard to tell from just looking at the milk file, so as with shaders, testing presets is key.