renderer: add support for rendering high dimensional textures for classification/segmentation use cases #1248
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For 3D segmentation problems it's really useful to be able to train the models from multiple viewpoints using Pytorch3D as the renderer. Currently due to hardcoded assumptions in a few spots the mesh renderer only supports rendering RGB (3 dimensional) data. You can encode the classification information as 3 channel data but if you have more than 3 classes you're out of luck.
This relaxes the assumptions to make rendering semantic classes work with
HardFlatShader
andAmbientLights
with no diffusion/specular. The other shaders/lights don't make any sense for classification since they mutate the texture values in some way.This only requires changes in
Materials
andAmbientLights
. The bulk of the code is the unit test.Test plan:
Added unit test that renders a 5 dimensional texture and compare dimensions 2-5 to a stored picture.