Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--Convert materials to use magnum materials. #2083

Merged
merged 27 commits into from
Apr 28, 2023
Merged

--Convert materials to use magnum materials. #2083

merged 27 commits into from
Apr 28, 2023

Conversation

jturner65
Copy link
Contributor

@jturner65 jturner65 commented Apr 25, 2023

Motivation and Context

This PR replaces the habitat gfx::MaterialData with Magnum's material implementations. This will expand support and potential capabilities, particularly for multi-layer materials and extensions, and make improvements to the PBR rendering in particular be much more easily realized.

How Has This Been Tested

Existing tests pass.

Types of changes

  • Docs change / refactoring / dependency upgrade
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have completed my CLA (see CONTRIBUTING)
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@facebook-github-bot facebook-github-bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Apr 25, 2023
@jturner65 jturner65 force-pushed the materialToMagnum branch 3 times, most recently from 427bf82 to 83dce89 Compare April 26, 2023 07:55
src/cmake/dependencies.cmake Show resolved Hide resolved
src/esp/assets/ResourceManager.cpp Outdated Show resolved Hide resolved
src/esp/assets/ResourceManager.cpp Outdated Show resolved Hide resolved
src/esp/assets/ResourceManager.cpp Outdated Show resolved Hide resolved
src/esp/assets/ResourceManager.cpp Outdated Show resolved Hide resolved
src/esp/gfx/PbrDrawable.cpp Outdated Show resolved Hide resolved
src/esp/gfx/PbrDrawable.cpp Show resolved Hide resolved
src/esp/gfx/PbrDrawable.cpp Outdated Show resolved Hide resolved
src/esp/metadata/attributes/AttributesEnumMaps.cpp Outdated Show resolved Hide resolved
src/esp/metadata/attributes/AttributesEnumMaps.h Outdated Show resolved Hide resolved
@jturner65 jturner65 force-pushed the materialToMagnum branch 3 times, most recently from 8a5154b to 026dfb1 Compare April 26, 2023 17:11
@jturner65 jturner65 marked this pull request as ready for review April 26, 2023 19:31
@jturner65 jturner65 changed the title WIP : Convert materials and lights to use magnum materials and lights. --Convert materials and lights to use magnum materials and lights. Apr 26, 2023
@jturner65 jturner65 changed the title --Convert materials and lights to use magnum materials and lights. --Convert materials to use magnum materials. Apr 26, 2023
Copy link
Contributor

@0mdc 0mdc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good so far! This is a great clean-up and nicely removes a gap between simulator and batch rendering.

I left some minor comments.

src/tests/AttributesConfigsTest.cpp Outdated Show resolved Hide resolved
src/esp/assets/ResourceManager.h Outdated Show resolved Hide resolved
src/esp/assets/ResourceManager.h Outdated Show resolved Hide resolved
src/esp/assets/ResourceManager.cpp Show resolved Hide resolved
Copy link
Contributor

@0mdc 0mdc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

src/esp/gfx/PbrDrawable.cpp Outdated Show resolved Hide resolved
@jturner65 jturner65 merged commit 7d04cd5 into main Apr 28, 2023
@jturner65 jturner65 deleted the materialToMagnum branch April 28, 2023 16:28
@Jianghanxiao
Copy link

Jianghanxiao commented Apr 28, 2023

The semantic sensor in the viewer.cpp doesn't work after this commit. I test the previous commit, the semantic sensor works

@jturner65
Copy link
Contributor Author

jturner65 commented May 1, 2023

The semantic sensor in the viewer.cpp doesn't work after this commit. I test the previous commit, the semantic sensor works

Addressed here : #2090

ykarmesh pushed a commit that referenced this pull request May 16, 2023
* skip unsupported 3D primitives (#2054)

* bugfix for setArticulatedObjectModelFilename causing garbage log output (#2053)

* Gfx-replay polish: workaround for material-overrides; new keyframe getter (#2035)

* gfx-replay polish: workaround for material-overrides; new keyframe getter API needed for a python application

* --Have pre-commit use node version compatible with Ubuntu 18.04 (#2058)

* --use system node install if present
* --change to hook-specific node version specification
* --change to nearest official release version

* Fix viewer.py framebuffer size mismatch. (#2055)

* Fix viewer.py framebuffer size mismatch.

* Change int tuples to mn.Vector2.

* Minor simplifications.

* Replay renderer: add line-rendering and unproject() (#2057)

* integrate DebugLineRender into replay renderers
* add ClassicReplayRenderer.unproject
* stub implementation for BatchReplayRenderer.unproject
* lighting hack in ClassicReplayRenderer to get ReplicaCAD stages to render correctly

* bugfix: cylinder proimitive collsion shape did not use halflength (#2060)

* --have disabled renderer test only execute 1 time (#2065)

* --(Bugfix) Fix normal transformation calc and address backface culling in case of negative scaling/reflections (#2062)

* --don't use co-factor matrix for transforming pre-calculated normals
* --address backface winding in Generic and Phong drawables

TODO :  Still need to set this up in batch renderer.
* --revert change in batch renderer

The normal matrix derivation will be changed in magnum to be appropriate, so we will keep the original verbiage for clarity.  We keep the changes in the default and PBR renderer calls since we need the determinant to determine if winding order must be reversed for backface culling.

* --test negative scaling along 3 axis
* --make copy of observation buffer for ground truth observation

The buffer is shared, so without a copy subsequent observations would overwrite the gt observation buffer.

* Update Magnum submodules to latest. (#2066)

* Update magnum submodules. (#2073)

* Add runtime perf stats for troubleshooting perf problems (#2070)

* add ResourceManager::getDrawableCountAndNumVertices, Sim.get_runtime_perf_stat_values, and related helpers

* --Refactor Semantic Mesh loading/flattening to remove deprecated functionality and improve efficiency  (#2079)

* --refactor to remove deprecated functionality
* --reviewer suggestions;
* --fixed inappropriate alloc

* Articulated object skinning (#2076)

* Add rig property to RenderAssetInstanceCreationInfo.

* Add render_asset to ao_config.json.

* Create render asset from ao_config.json when creating an articulated object.

* Store bone names into MeshTransformNode.

* Add skinning asset loading and phong rendering.

* Assorted minor fixes.

* Add flag to render articulated object primitives while having a skinned mesh for debugging.

* Add skinned articulated object test.

* Fix MetadataMediatorTest

* Cache joint transformations in drawables, other minor fixes.

* Fix test asset path + other minor fixes.

* Fix skinned mesh caching. Make ao_config.json render_asset path relative to the file. Code clean-up.

* --Add access to Scene Instance-level user defined attributes. (#2081)

* --add access to user defined in scene instance configs.

* --If SceneDataset or SceneInstance do not exist, return nullptr.


---------

Co-authored-by: Alexander Clegg <alexanderwclegg@gmail.com>

* --Don't make needless copy of scene instance attributes when retrieving user defined values (#2082)

* --don't make needless copy of scene instance attributes
* --add a test

* gfx-replay: fix to remove reflection when converting node transform matrix to rotation/translation (#2085)

* Articulated object semantics (#2086)

* Parse semantic_id from ao_config.json.

* Propagate semantic_id to skinned mesh.

* Add skinned mesh semantics test.

* Propagate semantics for non-skinned articulated object.

* --Convert materials to use magnum materials. (#2083)

* --add Mn::MaterialTools; WIP
* --address test issue;
* --convert materials to magnum materials
* --address fallback material not having defaults
* --fix many attribute accessor bugs.
* --appropriately make new attribute name with lowercase letter
* --Flat/Phong and Semantic textures work. Have to fix building PBR layers
* --improve ptr attribute naming; organize custom attribute assignment
Still need to build texture pointer attribute array with layers if they exist in base material
* --use owning material constructors
* --cleanup; clang-tidy; fix flat vs phong ambient txtr map in phong shader
Flat now has ambient Texture mapped directly from source material, so  no need to check baseColorTexture anymore.
* --get layers working properly
* --support noneRoughnessMetallic texture; handle per-vert objectID for PBR drawables
* --cache all material quantities in drawable to speed up draw access
* --prepopulate normal texture

* --address inappropriate bitflag check; add vertexID flag to PBR shader (#2090)

* enable frustum-culling for classic replay renderer (#2096)

* enable recompute_navmesh when creating sim with create_renderer==false (#2097)

* --[BUGFIX] Reverse directional lights before sending to uniform; (#2094)

* --reverse directional lights before sending to uniform;
Also correct existing light setups
* --fix lighting tutorial lights
* --correct orientations of sample  and default light with names

* Update Magnum submodules. (#2100)

* --[BUGFIX] Single channel texture support (#2102)

* --Address uncompressed single and double channel textures by modifying swizzles to be rrr1 and rrrg respectively.

* add intertiaFromURDF option to URDF loading API (#2098)

* add additional magnum importer plugins to python build dependencies

* refactor to reduce redundancy

---------

Co-authored-by: Eric Undersander <eundersander@fb.com>
Co-authored-by: John Turner <7strbass@gmail.com>
Co-authored-by: Mikaël Dallaire Côté <110583667+0mdc@users.noreply.github.com>
Co-authored-by: Alexander Clegg <alexanderwclegg@gmail.com>
Co-authored-by: Vladimír Vondruš <mosra@centrum.cz>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed Do not delete this pull request or issue due to inactivity.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants