Add KHR_texture_astc extension #2369

wasimabbas-arm · 2024-03-18T15:37:46Z

Add an extension for native ASTC encoded ktx2 images. Extension supports LDR/HDR and 3D images.

wasimabbas-arm · 2024-03-18T15:39:20Z

extensions/2.0/Khronos/KHR_texture_astc/README.md

+
+Regardless of the format used, these additional restrictions apply for compatibility reasons:
+
+- Swizzling metadata (`KTXswizzle`) MUST be `rgba` or omitted.


TBC, where we need this meta data or not.

I don't think glTF uses swizzle anywhere so probably not needed.

janharaldfredriksen-arm · 2024-03-21T11:29:08Z

extensions/2.0/Khronos/KHR_texture_astc/README.md

+
+This extension adds the ability to specify textures using KTX v2 images with (ASTC compression)[https://github.com/ARM-software/astc-encoder/blob/main/Docs/FormatOverview.md]. An implementation of this extension can use such images as an alternative to the PNG or JPEG images available in glTF 2.0 for more efficient asset transmission, reducing GPU memory footprint, higher texture throughput and faster execution.
+
+When this extension is used, it's allowed to use value `image/astc` for the `mimeType` property of images that are referenced by the `source` property of `KHR_texture_astc` texture extension object.


The example below uses mimeType image/ktx2. Is that intentional?

Yes, the extension will only allow image/ktx2. image/astc is a typo however.

janharaldfredriksen-arm · 2024-03-21T11:31:11Z

extensions/2.0/Khronos/KHR_texture_astc/README.md

+  > **Note:** Red textures from the core glTF 2.0 specification include:
+  > - `occlusionTexture` (standalone)
+
+- **Luminance-Alpha:** A texture that uses only Luminance and Alpha channels. To sample these textures use `.ga` swizzle in your shaders. Such textures MUST NOT be encoded with sRGB transfer function.


Do we need Luminance-Alpha types here?
(I don't know what other glTF extensions do, but if Red-Green was sufficient that'd be better.)

There is support for L+A in the encoder so it makes sense for gltf to have support too. Although my note below

**Note:** The core glTF 2.0 specification has no examples of luminance-alpha textures.

So maybe we remove it.

janharaldfredriksen-arm · 2024-03-21T11:34:30Z

extensions/2.0/Khronos/KHR_texture_astc/README.md

+
+### KTX header fields for ASTC payloads
+  - `supercompressionScheme` MUST be `0` (None).
+  - ASTC HDR vs LDR as well as block sizes can be determined from the vkFormat of the image


nit: vkFormat => VkFormat
(in a few other places)

The field in the KTX v2 header is called "vkFormat". So here "value of vkFormat" would be correct. I haven't looked at the other instances of its use.

janharaldfredriksen-arm · 2024-03-21T11:36:30Z

extensions/2.0/Khronos/KHR_texture_astc/schema/texture.KHR_texture_basisu.schema.json

+    "$schema": "http://json-schema.org/draft-04/schema",
+    "title": "KHR_texture_astc glTF Texture Extension",
+    "type": "object",
+    "description": "glTF extension to specify textures using the KTX v2 images with astc compression.",


nit: astc => ASTC

MarkCallow

Looks good but I do think you'll need to address the fragmentation of ASTC implementations into LDR-only, LDR+HDR and LDR+HDR+3D possibly by having multiple extensions. I'm no glTF expert but perhaps you can just have different extension names in this single extension document. That's what the GL extensions did IIRC.

MarkCallow · 2024-08-30T02:12:34Z

extensions/2.0/Khronos/KHR_texture_astc/README.md

+
+## Overview
+
+This extension adds the ability to specify textures using KTX v2 images with (ASTC compression)[https://github.com/ARM-software/astc-encoder/blob/main/Docs/FormatOverview.md]. An implementation of this extension can use such images as an alternative to the PNG or JPEG images available in glTF 2.0 for more efficient asset transmission, reducing GPU memory footprint, higher texture throughput and faster execution.


"using ASTC compressed images in KTX v2 containers" would be more accurate.

MarkCallow · 2024-08-30T02:13:34Z

extensions/2.0/Khronos/KHR_texture_astc/README.md

+
+When this extension is used, it's allowed to use value `image/astc` for the `mimeType` property of images that are referenced by the `source` property of `KHR_texture_astc` texture extension object.
+
+At runtime, engines can directly use the ktx images unlike EXT_texture_basisu which requires doing transcoding to other compressed formats. This also means you get support for all ASTC block formats and quality presets.


MarkCallow · 2024-08-30T02:14:19Z

extensions/2.0/Khronos/KHR_texture_astc/README.md

+
+## glTF Schema Updates
+
+The `KHR_texture_astc` extension is added to the `textures` object and specifies a `source` property that points to the index of the `image` which defines a reference to the KTX v2 image with ASTC compression.


"KTX v2 file with the ASTC compressed image."

MarkCallow · 2024-08-30T02:27:10Z

extensions/2.0/Khronos/KHR_texture_astc/README.md

+  > **Note:** Red textures from the core glTF 2.0 specification include:
+  > - `occlusionTexture` (standalone)
+
+- **Luminance-Alpha:** A texture that uses only Luminance and Alpha channels. To sample these textures use `.ga` swizzle in your shaders. Such textures MUST NOT be encoded with sRGB transfer function.


Why must it not be encoded with sRGB? It is quite normal to encode luminance in sRGB?

MarkCallow · 2024-08-30T02:31:27Z

extensions/2.0/Khronos/KHR_texture_astc/README.md

+
+### KTX header fields for ASTC payloads
+  - `supercompressionScheme` MUST be `0` (None).
+  - ASTC HDR vs LDR as well as block sizes can be determined from the vkFormat of the image


The field in the KTX v2 header is called "vkFormat". So here "value of vkFormat" would be correct. I haven't looked at the other instances of its use.

MarkCallow · 2024-08-30T02:32:51Z

extensions/2.0/Khronos/KHR_texture_astc/README.md

+
+Regardless of the format used, these additional restrictions apply for compatibility reasons:
+
+- Swizzling metadata (`KTXswizzle`) MUST be `rgba` or omitted.


I don't think glTF uses swizzle anywhere so probably not needed.

MarkCallow · 2024-08-30T02:35:09Z

extensions/2.0/Khronos/KHR_texture_astc/README.md

+    - `colorPrimaries` MUST be `KHR_DF_PRIMARIES_UNSPECIFIED`;
+    - `transferFunction` MUST be `KHR_DF_TRANSFER_LINEAR`.
+- When a texture refers to a sampler with mipmap minification, the KTX image MUST contain a full mip pyramid.
+- When a texture referencing a KTX v2 image with ASTC compression is used for glTF 2.0 material maps (both color and non-color), the KTX v2 image MUST be of **2D** type as defined in the KTX v2 Specification, Section 4.1.


"referencing a KTX v2 file with ASTC compressed image"

MarkCallow · 2024-08-30T02:37:44Z

extensions/2.0/Khronos/KHR_texture_astc/README.md

+
+## KTX v2 Images with Astc compression 
+
+To cover a broad range of use cases, this extension allows different ASTC block sizes as well as LDR, HDR and 3D images. These can be determined from the vkFormat of the KTX image.


Given the fragmented nature of ASTC implementations you may need to have different extensions for LDR only, LDR+HDR and 3D.

abbaswasim · 2024-09-07T15:17:10Z

Looks good but I do think you'll need to address the fragmentation of ASTC implementations into LDR-only, LDR+HDR and LDR+HDR+3D possibly by having multiple extensions. I'm no glTF expert but perhaps you can just have different extension names in this single extension document. That's what the GL extensions did IIRC.

I am not sure why we need to make the distinction? If all of these can be specified from the vkFormat then why have multiple extensions?

We probably need to add a bit more details of what other texture types could be supported and the formats like RGBM or RGB9E5 etc.

wasimabbas-arm closed this Mar 18, 2024

wasimabbas-arm changed the title ~~Add KHR_texture_astc extension~~ WIP: Add KHR_texture_astc extension Mar 18, 2024

wasimabbas-arm commented Mar 18, 2024

View reviewed changes

wasimabbas-arm reopened this Mar 18, 2024

wasimabbas-arm closed this Mar 18, 2024

janharaldfredriksen-arm reviewed Mar 21, 2024

View reviewed changes

MarkCallow reviewed Aug 30, 2024

View reviewed changes

MarkCallow reopened this Sep 5, 2024

abbaswasim force-pushed the KHR_texture_astc branch from 1d83f0e to 6b6d872 Compare September 7, 2024 14:42

abbaswasim added 3 commits September 7, 2024 15:43

Add KHR_texture_astc extension

75ad1fe

Add WebGL astc extension restriction of npot textures back in

0b5690b

Fix review comments

94e4293

abbaswasim force-pushed the KHR_texture_astc branch from 6b6d872 to 94e4293 Compare September 7, 2024 14:44

Review comments

b7b6dd2

wasimabbas-arm changed the title ~~WIP: Add KHR_texture_astc extension~~ Add KHR_texture_astc extension Sep 7, 2024

donmccurdy mentioned this pull request Sep 15, 2024

Expected positive integer for width but received 0 of type number when running optimize donmccurdy/glTF-Transform#1500

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add KHR_texture_astc extension #2369

Add KHR_texture_astc extension #2369

wasimabbas-arm commented Mar 18, 2024

wasimabbas-arm Mar 18, 2024

MarkCallow Aug 30, 2024

janharaldfredriksen-arm Mar 21, 2024

wasimabbas-arm Jul 24, 2024

janharaldfredriksen-arm Mar 21, 2024

abbaswasim Sep 7, 2024

janharaldfredriksen-arm Mar 21, 2024

MarkCallow Aug 30, 2024

janharaldfredriksen-arm Mar 21, 2024

MarkCallow left a comment

MarkCallow Aug 30, 2024

MarkCallow Aug 30, 2024

MarkCallow Aug 30, 2024

MarkCallow Aug 30, 2024

MarkCallow Aug 30, 2024

MarkCallow Aug 30, 2024

MarkCallow Aug 30, 2024

MarkCallow Aug 30, 2024

abbaswasim commented Sep 7, 2024


		Regardless of the format used, these additional restrictions apply for compatibility reasons:

		- Swizzling metadata (`KTXswizzle`) MUST be `rgba` or omitted.


		This extension adds the ability to specify textures using KTX v2 images with (ASTC compression)[https://github.com/ARM-software/astc-encoder/blob/main/Docs/FormatOverview.md]. An implementation of this extension can use such images as an alternative to the PNG or JPEG images available in glTF 2.0 for more efficient asset transmission, reducing GPU memory footprint, higher texture throughput and faster execution.

		When this extension is used, it's allowed to use value `image/astc` for the `mimeType` property of images that are referenced by the `source` property of `KHR_texture_astc` texture extension object.


		## Overview

		This extension adds the ability to specify textures using KTX v2 images with (ASTC compression)[https://github.com/ARM-software/astc-encoder/blob/main/Docs/FormatOverview.md]. An implementation of this extension can use such images as an alternative to the PNG or JPEG images available in glTF 2.0 for more efficient asset transmission, reducing GPU memory footprint, higher texture throughput and faster execution.


		When this extension is used, it's allowed to use value `image/astc` for the `mimeType` property of images that are referenced by the `source` property of `KHR_texture_astc` texture extension object.

		At runtime, engines can directly use the ktx images unlike EXT_texture_basisu which requires doing transcoding to other compressed formats. This also means you get support for all ASTC block formats and quality presets.


		## glTF Schema Updates

		The `KHR_texture_astc` extension is added to the `textures` object and specifies a `source` property that points to the index of the `image` which defines a reference to the KTX v2 image with ASTC compression.


		## KTX v2 Images with Astc compression

		To cover a broad range of use cases, this extension allows different ASTC block sizes as well as LDR, HDR and 3D images. These can be determined from the vkFormat of the KTX image.

Add KHR_texture_astc extension #2369

Are you sure you want to change the base?

Add KHR_texture_astc extension #2369

Conversation

wasimabbas-arm commented Mar 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MarkCallow left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abbaswasim commented Sep 7, 2024