Skip to content

Commit

Permalink
Merge pull request KhronosGroup#51 from javagl/ext-mesh-features-revi…
Browse files Browse the repository at this point in the history 
…sion-spec

Restricting `EXT_mesh_features` to the definition of feature IDs
  • Loading branch information
@lilleyse
lilleyse authored Feb 11, 2022
2 parents 04b71b3 + 3c89d72 commit 0ed57a0
Show file tree
Hide file tree
Showing 11 changed files with 21,080 additions and 0 deletions.
213 changes: 213 additions & 0 deletions extensions/2.0/Vendor/EXT_mesh_features/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
<!-- omit in toc -->
# EXT_mesh_features

<!-- omit in toc -->
## Contributors

* Peter Gagliardi, Cesium
Expand All @@ -11,13 +13,224 @@
* Samuel Vargas, Cesium
* Patrick Cozzi, Cesium

<!-- omit in toc -->
## Status

Draft

<!-- omit in toc -->
## Dependencies

Written against the glTF 2.0 specification.

<!-- omit in toc -->
## Optional vs. Required

This extension is optional, meaning it should be placed in the `extensionsUsed` list, but not in the `extensionsRequired` list.

<!-- omit in toc -->
## Table of Contents

- [Overview](#overview)
- [Features](#features)
- [Feature IDs](#feature-ids)
- [Feature ID by Vertex](#feature-id-by-vertex)
- [Feature ID by Texture Coordinates](#feature-id-by-texture-coordinates)
- [Using Feature IDs](#using-feature-ids)
- [Referencing Property Tables with Feature IDs](#referencing-property-tables-with-feature-ids)
- [Referencing External Resources with Feature IDs](#referencing-external-resources-with-feature-ids)
- [Schema](#schema-1)
- [Examples](#examples)
- [Revision History](#revision-history)

## Overview

This extension defines a means of assigning identifiers to geometry and subcomponents of geometry within a glTF 2.0 asset.

In most realtime 3D contexts, performance requirements demand minimizing the number of nodes and meshes in an asset. These requirements compete with interactivity, as applications may wish to merge static objects while still supporting interaction or inspection on those objects. Common performance optimizations for GPU rendering — like merging geometry or GPU instancing to reduce CPU overhead — may destroy references to distinct objects, their attributes, and their behaviors.

By defining a representation of conceptual objects ("features") distinct from rendered geometry, this extension allows applications to preserve important details of 3D assets for inspection and interaction without compromising runtime performance and draw calls.

See [Examples](#examples) for a more detailed list of use cases for this extension.


### Features

A **feature** is a conceptual object in a virtual environment. Similar concepts exist in various industries and domains. In Geographic Information Systems (GIS) a feature is an entity such as a point, polyline, or polygon that represents some element on a map. In another domain like CAD/BIM, a feature might be a component of a design model, such as a pipe. A feature could also be a 3D building in a city, a tree in a forest, a sample point in a weather model, or a patch of texels on a 3D asset.

### Feature IDs

*Defined in [mesh.primitive.EXT_mesh_features.schema.json](./schema/mesh.primitive.EXT_mesh_features.schema.json) and [featureId.schema.json](./schema/featureId.schema.json)*

Features are identified within a 3D asset by **Feature IDs**. A mesh primitive may specify multiple sets of feature IDs. These feature ID sets might (for example) identify features at different levels of abstraction: there may be feature IDs that identify individual buildings, and feature IDs that identify different parts of each building.

Each feature ID set is defined as a set of values that are associated with the conceptual parts of the model. The definition of the feature ID set may include a `nullFeatureId`, which is a value that indicates that a certain part is not considered to be an identifiable object. The definition also includes a `featureCount` value, which is the number of unique features that are identified.

Feature IDs can be associated with parts of a model in one of two ways:

* **Feature ID by Vertex:** Feature IDs that are stored as vertex attribute, using a standard glTF accessor. The `featureId.attribute` refers to this accessor, and allows defining feature IDs for each individual vertex.
* **Feature ID by Texture Coordinates:** Feature IDs that are stored in the channels of a standard glTF texture. The `featureId.texture` refers to this texture, and allows defining feature IDs for regions on the surface of a mesh.

These concepts are explained in more detail in the following sections.

#### Feature ID by Vertex

*Defined [featureIdAttribute.schema.json](./schema/featureIdAttribute.schema.json).*

Per-vertex feature IDs are defined in a vertex attribute accessor.

Names of feature ID attribute semantics follow the naming convention `_FEATURE_ID_n` where `n` must start with 0 and continue with consecutive positive integers: `_FEATURE_ID_0`, `_FEATURE_ID_1`, etc. Indices must not use leading zeroes to pad the number of digits (e.g., `_FEATURE_ID_01` is not allowed).

The attribute's accessor `type` must be `"SCALAR"` and `normalized` must be false. The accessor's `componentType` is not restricted.

> **Implementation note:** Because glTF accessors do not support `UNSIGNED_INT` types for 32-bit integers in vertex attributes, `FLOAT` may be used instead allowing integer feature IDs up to 2<sup>24</sup>. For smaller ranges of feature IDs, `UNSIGNED_BYTE` or `UNSIGNED_SHORT` may be used. As with other vertex attributes, each element of a feature ID accessor must align to 4-byte boundaries.
> **Implementation note:** For a primitive with feature ID attributes, points in the interior of a triangle or line segment should be considered to belong to the feature associated with the nearest vertex.
Per-vertex feature IDs can be used to identify individual objects that have been combined into a single mesh primitive, as shown in the example below.

> **Example:** A primitive defines two quads, where each quad is a distinct feature. The quads are composed of four vertices, distinguished by different `_FEATURE_ID_0` vertex attribute values.
>
> Note that the `attribute` value is a set index `n` that refers to the `_FEATURE_ID_n` vertex attribute semantic. For example, `"attribute": 0` corresponds to `_FEATURE_ID_0`.
>
> ![Feature ID by attribute](figures/feature-id-by-attribute.png)
>
> ```jsonc
> {
> "primitives": [
> {
> "attributes": {
> "POSITION": 0,
> "_FEATURE_ID_0": 1
> },
> "indices": 2,
> "mode": 4,
> "extensions": {
> "EXT_mesh_features": {
> "featureIds": {
> "exampleRectangles" : {
> "featureCount": 2,
> "attribute": 0
> }
> }
> }
> }
> }
> ]
> }
> ```
#### Feature ID by Texture Coordinates
*Defined in [featureIdTexture.schema.json](./schema/featureIdTexture.schema.json).*
Feature ID textures classify the pixels of an image into different features. Some use cases include image segmentation or marking regions on a map. Often per-texel feature IDs provide finer granularity than per-vertex feature IDs, as in the example below.
> **Example:** A building facade, represented by a single quad. The primitive's `baseColorTexture` displays the visible appearance of the building, and its feature ID texture identifies regions of the quad (walls, door, roof, window) as distinct features. Both textures use the same texture coordinates, `TEXCOORD_0`, in this example.
>
> <img src="figures/feature-id-by-texture.png" alt="Feature ID Texture" width="800">
>
> ```jsonc
> {
> "primitives": [
> {
> "attributes": {
> "POSITION": 0,
> "TEXCOORD_0": 1
> },
> "indices": 2,
> "material": 0,
> "extensions": {
> "EXT_mesh_features": {
> "featureIds": {
> "buildingComponents" : {
> "featureCount": 4,
> "texture" : {
> "index": 0,
> "texCoord": 0,
> "channels": [0]
> }
> }
> }
> }
> }
> }
> ]
> }
> ```
The `texture` object of a `featureId` extends the glTF [`textureInfo`](../../../../specification/2.0/schema/textureInfo.schema.json) object. The `channels` array contains non-negative integer values corresponding to channels of the source texture that the feature ID consists of. Channels of an `RGBA` texture are numbered 0–3 respectively, although specialized texture formats may allow additional channels.
The values from the selected channels are treated as unsigned 8 bit integers, and represent the bytes of the actual feature ID, in little-endian order.
> **Example:**
> If a `featureID.texture` defines `"channels": [0, 1]`, then the actual feature ID can be computed as `id = channel[0] | (channel[1] << 8);`.
> If a `featureID.texture` defines `"channels": [1, 0, 2]`, then the actual feature ID can be computed as `id = channel[1] | (channel[0] << 8) | (channel[2] << 16);`.
Texture filtering must be `9728` (NEAREST), or undefined, for any texture object referenced as a feature ID texture.
### Using Feature IDs
The feature ID sets that are associated with mesh primitives can be accessed by client applications, and be used to look up addition information that is associated with thse features. Two possible ways of associating features with additional information are presented here.
#### Referencing Property Tables with Feature IDs
When combined with the `EXT_structural_metadata` extension, feature ID sets can be associated with property tables. A property table maps each feature ID to a set of values that are associated with the respective feature. The feature ID in this case serves as an _index_ for the row of the table. The index of the property table that a certain set of feature IDs is associated with is stored in the `propertyTable` of the feature ID set definition.
> **Example:** This example assumes that an array of property tables is defined in the asset, using the `EXT_structural_metadata` extension. The example shows a primitive with multiple feature ID sets. The first one uses a feature ID texture that contains 4 different features. The second one is defined via a vertex attribute, and defines 2 different features. The first ID set is associated with the property table with index 1. The second one is associated with the property table with index 0.
>
> ```jsonc
> // Primitive:
> "extensions": {
> "EXT_mesh_features": {
> "featureIds": {
> "firstFeatureId" : {
> "featureCount": 4,
> "texture" : {
> "index": 0,
> "texCoord": 0,
> "channels": [0]
> },
> "propertyTable": 1
> },
> "secondFeatureId":
> {
> "featureCount": 2,
> "attribute": 0,
> "propertyTable": 0
> }
> }
> }
> }
> ```
#### Referencing External Resources with Feature IDs
Feature IDs may identify features for use in other extensions or in custom applications. Use cases for these IDs could include identifying features for styling or picking, or looking up metadata externally in a REST API or database.
> <img src="figures/feature-id-lookup.png" alt="Feature ID lookup" width="800">
## Schema
* [mesh.primitive.EXT_mesh_features.schema.json](./schema/mesh.primitive.EXT_mesh_features.schema.json)
* [featureId.schema.json](./schema/featureId.schema.json)
* [featureIdAttribute.schema.json](./schema/featureIdAttribute.schema.json)
* [featureIdTexture.schema.json](./schema/featureIdTexture.schema.json)
## Examples
_This section is non-normative_
The examples below show possible use cases for this extension.
Example|Description|Image
--|--|--
Triangle mesh|Feature IDs are assigned to each vertex to distinguish components of a building.|![Building Components](figures/feature-id-building-components.png)
Multi-point features|A point cloud where groups of points that represent distinct components of the building receive the same feature ID|![Multi-point features](figures/feature-id-building-points.png)
Material classification|A textured mesh using a feature ID texture to identify regions on the surface of the mesh as distinct features|![Material Classification](figures/feature-id-texture-ids.png)
## Revision History
The revision history of this extension can be found in the [common revision history of the 3D Tiles Next extensions](../next/REVISION_HISTORY.md).
Binary file added extensions/2.0/Vendor/EXT_mesh_features/figures/feature-id-building-components.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 0ed57a0

Please sign in to comment.