Compressing assets

The DirectX Tool Kit is designed for ease-of-use and to fit neatly into the 'Venn diagram' of all Microsoft platforms. Therefore, the library does not include an asset management system, file-system-in-a-file WAD-style packaging, etc. On Windows 10 and Xbox Series X|S, for example, technologies like DirectStorage can greatly improve on-disk compression and throughput. Generally these concerns are out-of-scope for the tool kit.

That said, for very large models SDKMESH and even block-compressed DDS files can have a very large disk footprint. For SDKMESH, some space savings can come from using tricks like the Samples Content Exporter -compressvertexdata:+ option which uses more compact DXGI formats for vertex data, and this also translates into in-memory savings as well. This still only gets you so far. The good news is that SDKMESH and DDS files compress well using classic lossless compression schemes.

If your system requirements are Windows 8 or later, then you can make use of the Compression API as a quick & dirty solution. For some example tooling see:

• xbcompress command-line tool which generates a SZDD/KWAJ-style compressed file using the Compression API.
• ReadCompressedData .cpp / .h which loads these compressed binary blob files and expands them.
• CompressedTextureFactory which is a simple implementation of EffectTextureFactory for DirectX 12 which looks for compressed DDS files as part of the model loading process.

Examples

Loading a 'compressed' bin file:

auto blob = DX::ReadCompressedData(L"mylargedata.bi_");

Loading a 'compressed' SDKMESH file:

{
    auto modelBlob = DX::ReadCompressedData(L"AbstractCathedral.sdkmes_");
    m_model = Model::CreateFromSDKMESH(device, modelBlob.data(), modelBlob.size());
}

Loading a 'compressed' DDS file:

{
    auto ddsBlob = DX::ReadCompressedData(L"Map3DColor.dd_");
    DX::ThrowIfFailed(
        CreateDDSTextureFromMemory(device, resourceUpload,
            ddsBlob.data(), ddsBlob.size(), m_map.ReleaseAndGetAddressOf()));
}

Remarks

For Windows 7 compatibility or just for personal preference, you can use open source solutions like zlib, zopfli, etc. in the same basic way.

Credits

The SZDD/KWAJ-style file format has a long history, and dates back to the days of MS-DOS. The compress command would take a file like "myfile.exe" and compress it into "myfile.ex_". The uncompress tool did the opposite.

The xbcompress tool is inspired by this simple design.

File offset	Field length	Description
0	8	Magic byte sequence to uniquely identify file format. The magic sequence is 0x41, 0x46, 0x43, 0x57, 0x47, 0x50, 0x53, 0x4d.
9	1	Compression mode. COMPRESS_ALGORITHM_LZMS (5) or COMPRESS_ALGORITHM_MSZIP (2)
10	1	File format version. Currently 0x41 ('A')
11	2	Last character (UTF-16LE) that was changed to '' when the compressed name was determined. This value is 0 if '.' was added instead.
13	4	Size in bytes of the original uncompressed data block. To keep the code simple, this file format only supports up to 4 GB file sizes.