mdc_encoder

Spirit MDC-based point cloud encoder

This repository contains the code to build a library that can be used in a Unity application (or any other programming environment that support Dlls) to encode point clouds that have been generated by the capturer library.

This library uses an multiple description coding (MDC) approach to encode the point cloud into multiple descriptions that each contain a subset of the points. For a more detailed explanation of the system, we refer to our recent publications [1, 2].

[1] M. De Fré, J. van der Hooft, T. Wauters, and F De Turck. "Demonstrating Adaptive Many-to-Many Immersive Teleconferencing for Volumetric Video", Proceedings of the 15th ACM Multimedia Systems Conference, 2024 (available here)

[2] M. De Fré, J. van der Hooft, T. Wauters, and F De Turck. "Scalable MDC-Based Volumetric Video Delivery for Real-Time One-to-Many WebRTC Conferencing", Proceedings of the 15th ACM Multimedia Systems Conference, 2024 (available here)

Building

This repository uses CMake to build the library. For Windows you can use the following command to generate the solution

cmake -G "Visual Studio 17 2022" -A x64 -S . -B "x64" -DBUILD_SHARED_LIBS=ON -DCMAKE_CONFIGURATION_TYPES=Release

After this you will to open the generated solution, change the build type to Release and press Ctrl+Shift+B to build the Dll.

Usage

Creates the encoding queue:

This queue holds up to two point clouds (i.e., the frame that is currently encoding and one buffered frame).

initialize()

To encode the point cloud from the capturer:

This method is non-blocking, and will simply add the point cloud to the queue. Once the encoders are ready for the next frame they will retrieve it from the queue and split it into multiple descriptions. You need to make sure you have called register_description_done_callback at the start of your application to receive the encoded description.

encode_pc(PointCloud* pc)

To register the callback that will be used when a description has been encoded:

register_description_done_callback(DescriptionDoneCallback cb)

This callback has to have the following function definition:

func(Description* dsc, char* raw_data_ptr, uint32_t n_points_in_total, uint32_t dsc_size, uint32_t frame_nr, uint32_t dsc_nr)

To free the encoded description once you are done with it:

free_description(Description*)

To decode the raw data of a single description into a point cloud:

This method is blocking, and will only return once decoding is fully complete.

DracoMDCDecoder* decode_pc(char* data, uint32_t size)

Get the number of points in the decoded description:

get_n_points(DracoMDCDecoder*)

Get a raw pointer to an array containing the position data of decoded description:

Points are saved as <X, Y, Z>, with each axis having a 32bit value.

get_point_array(DracoMDCDecoder*)

Get the number of points in the decoded encoded description:

Colors are saved as <R, G, B>, with each color channel having a 8bit value.

get_color_array(DracoMDCDecoder*)

To free the decoded description once you are done with it:

free_decoder(DracoMDCDecoder*)

To know when you can free the captured point cloud:

register_free_pc_callback(FreePointCloudCallback cb)

This callback has to have the following function definition:

func(PointCloud* pc)

To stop the encoder threads after you are done:

cleanup()

Tested operating systems

• Windows 10/11

Dependencies

• Draco