Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: USD in the Scene Pipeline #92

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Conversation

AMZN-Gene
Copy link

Resolves: o3de/o3de#18248

Signed-off-by: AMZN-Gene <genewalt@amazon.com>
Signed-off-by: AMZN-Gene <genewalt@amazon.com>
Signed-off-by: Gene Walters <32776221+AMZN-Gene@users.noreply.github.com>
Signed-off-by: Gene Walters <32776221+AMZN-Gene@users.noreply.github.com>
Adding build time stats if we wanted to move assimp out of 3rd party.
Comment on lines 189 to 194
2. Contribute back to Assimp and update scene builder to load USD files with references. Stages are flattened by adding in reference geometry. O3DE then processes USD like FBX and other scene file formats.
1. Input: USD files which contains a hierarchy of references to other geometry inside of other USD files 
2. Output: Flattened azmodel containing the geometry from the multiple USD files inside one azmodel file
3. Contribute back to Assimp giving API option to flatten stages or keep reference geom separate. Update O3DE to create a procedural prefab that mimics the relationship graph of USD files by matching models and transforms. This could be expanded later to allow O3DE developers to translate USD properties into O3DE component properties as well as add advanced shaders/materials.
1. Input: USD files which contains a hierarchy of references to other geometry inside of other USD files
2. Output: Procedural prefabs generated per USD. Root prefab preserves the hierarchy of the original USD.
Copy link

@nick-l-o3de nick-l-o3de Aug 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that its a good way to start phasing this in, but might actually be in conflict with both the purpose of stages and references, as well as a possible long term usage.
Files referencing other files are used for a number of reasons,

  1. Just to arrange things more neatly or split work apart between different people. The final scene is intended to be re-assembled as a whole by loading everything.
  2. To present alternative "variants" of a Thing in the scene, for the purposes of alleviating CPU or GPU or Memory load. For example, there might be an Oak Tree object in the scene, but there are 2 available variants, a full detailed render-ready oak tree with a billion verts, or a billboard oak tree to switch it out for, when teh author is working on something other than rendering and would like their DCC tool to remain responsive. In this case, the 2 variants are never expected to ever render at the same time or exist in the same scene at the same time.
  3. To duplicate objects without duplicating data. In this case, the same object is repeated in the scene over and over, but its the same object, so each one just references an existing USD but specifies a new transform for it.
  4. To provide a procedural object library - an example being a "chair" USD which references 20 different variants of a chair, but only loads a selected one. This allows an artist to populate their scene with 100 chairs, but then choose a different variant for each such chair instance, to make them look unique, as well as from a data organizing perspective, allow other artists to add more available variants to the "chair" USD over time, expanding the library of avilable chair variants to use without invalidating existing scenes or data.

The basic approach of "flattening" every mesh it finds into one azmodel file can only actually potentially cover the first option, and does not cover the other options at all. It might be a good start though, depending on how its written.

Its also probably why ASSIMP and others may not have tackled the problem - its not a binary problem, and "what parts of this USD do you want to realize into the scene" is not solvable by automation or brute force, its the choice of the given artist at the time of the given usage, and the combinatorics can explode too quickly to do a "for each variant, for each other variant, export each combination" situation...

for example,

car.usd pseudocode

  (root xform)
     VARIANT A:  "car_low_detail_fake_geometry_for_preview.usd"
     VARIANT B: 
        xform "car"
            VARIANT 1:  "lambo.usd"
            VARIANT 2:  "audi.usd"
       .. hundeds of cars, you can use in a parking garage

Workflow in your pixar DCC tool is to duplicate the car over and over in the parking garage and just click on each instance and choose a variant like "audi" or "lambo". The world can also substitute all of them for variant A at an individual or global level to drop detail to nothing on the cars to save CPU and GPU and MEM. Switching them back to Variant B will restore all the cars to full render detail.

from above, you can see that simply exporting everything into one atom model would be a mistake and would not be useful to anyone. Especially if someone then has ParkingGarage.USD with hundreds of instances of the car.

As for longer term, There's only a couple ways I think we can actually support what is essentially the superpower of USD (points 2, 3, and 4 above).

One way would be to make it so that the scene processing pipeline is actually reactive to the usage of the USD in the scene. What this means is, there's no merging (or little marging, but no merging across variants) so in the above example, the most that gets exported is an atom mesh seperately for car.usd, lambo.usd, audi.usd, and car_low_detail_fake_geometry_for_preview.usd.

This would essentially mean no changes to assimp, those files would just be extracted into their own USDs and would not attempt to follow references, just flattening everything into the same USD into one.

But the magic would be when it generates the procprefab for car, it would make components on the transforms that represented the objects - when it encounters a USD variant that would switch which USD it actually used, it could actually generate a component that allowed the user to switch the procprefab (output by the pipeline) was selected as a child, and switch between the available variants that way when editing individual entities. (if not that, then at least switch between available flattened atom meshes that represent sub-usds)

A further optimization would be those switcher components being allowed to, when edited, also edit the associated scene export settings, so that variants unused are simply not present or compiled (and recompile and become usable when selected).

Whats interesting about that, is that the functionality for having a component or a property of the Prefab Component (which is the component on an entity which stores the reference to the sub prefab to spawn the heirarhcy of as a child to itself) is not a USD thing. Adding this functionality would support USD variants, but also allow our prefabs to use that functionality without USD being involved, too.

Any thoughts here?

One problem is variants can be within a USD, and don't always refer to other USDs. You can make one USD that has a variant override for a color value, offers 3 different variants, and only overrides that color value based on what variant you choose. thats going to be very difficult to handle on an Atom Model / Scene basis, but extracting each variant as its own prefab or even as a prefab patch that shows up on the prefab UI as something you can toggle bettween might be the best, and not a huge change to our existing prefab pipeline.

To be more specific about this,

in USD, an override is a set of patches that when applied, change variables in the scene usually by path.
These overrides are stored in the USD file, and look like over "path to override" { variable=value }

in PREFAB, an override is a set of patches that when applied alter variables in the scene usually by path. The overrides are stored in the PREFAB file, in the root element as a list of path to override" : { new value }

in USD. a variant is a set of overrides (ie, patches) that apply to the scene optionally. When the scene has multiple variants available, the user can choose ONE variant from the list to apply to the scene, and only those overrides apply.

in PREFAB, there is no concept of "variants", but, since it already has the ability to store a set of overrides, it would not be a large stretch to have a set of "optional" overrides that are not always applied. but stored in the file, which the user can either toggle on and off in the GUI (turning on and off layers of overrides), or toggled like a radio button, where turning one on, turns the others off, grouped.

If such a feature exists, it would mean extracting variant over {} blocks into these optional overrides that are exported into the procprefab.

Comment on lines +366 to +370
Should we move Assimp to a Gem instead of using O3DE 3rdParty system?
---------------------------------------------------------------------

Moving forward with USD via Assimp will require regularly pulling new versions of Assimp as new features come online (example: references). Going through the 3rdParty library system is slower as it requires its own approval process via [O3DE's 3rd Party Package repo](https://github.com/o3de/3p-package-source). The downside to moving Assimp into a gem directly, is that we'd reimplement what's already in 3rdParty, for example, [git pulling a particular Assimp commit](https://github.com/o3de/3p-package-source/blob/main/package-system/assimp/build_config.json#L2-L3), [building](https://github.com/o3de/3p-package-source/blob/main/package-system/assimp/build_assimp_windows.cmd), and [packaging](https://github.com/o3de/3p-package-source/blob/main/package-system/assimp/install_assimp_windows.json) the library. This would also increase build time when compiling the editor source; Assimp takes 2 minutes to download and compile on my local 16-Core 3.4GHz PC.

Copy link

@nick-l-o3de nick-l-o3de Aug 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would actually suggest moving Assimp into a fetchcontent type library, if it only takes 2 minutes to compile. There's nothing in its build scripts I see that would make it incompatible with this. It doesn't need to be a gem (although it could be, but we would never commit the code of actual assimp into our repo, we'd use fetchcontent instead - so the only thing in the gem would be the cmake scripts that handle doing that, and if we want to make custom plugins or custom diffs...)

Copy link
Author

@AMZN-Gene AMZN-Gene Aug 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Just moved assimp build with SDKWrapper. WiP tracking here: o3de/o3de#18273

@guillaume-haerinck
Copy link

One question that I haven't seen in the RFC is the ability for O3DE to modify the imported USD so that the source DCC can get some changes and you end up with a back-and-forth workflow. In my opinion that would be fairly complicated but given that USD addition was pushed by some specific people, it might be best to ask if it is something they need or not.

Remedy does it but to do so, they decided to use USD as their scene file format which is a bold move (and creates a bunch of issues). Given that USD was created for communication between tools, but not really for long term stability of a scene (a change can creates a bunch of unexpected modification in a scene as shown in their GDC pres, some free info there as well)

I believe your approach of making a prefab out of the USD, so that its kept as an import format is best for stability and better control of the feature, easier diff etc.

@michalpelka
Copy link

I went through RFC and like the idea of stages and contributions back to Assimp.
I originally developed support for another, much simpler description format called URDF, it is now a Robot Importer tool in ROS 2 gem (https://www.docs.o3de.org/docs/user-guide/interactivity/robotics/importing-robot/) and I would like to point out few issues that we had developing this tool.

1. Problem with file referencing outside source asset directory.
The URDF is essentially a list of meshes that are referenced from different places in the User's file system.
This can be an OBJ or DAE that lives in /opt/ros/humble/share.
To give a good experience we take an "importer" approach, the User is a walkthrough through steps:

  • Find a URDF/SDF file
  • Configure Robot variant
  • Copy all referenced meshes from the User's file system to the User's project source directory. Educate and inform the user if we fail to resolve.
  • Wait for Assetprocessor to process those meshes
  • Procedurally create Prefabs.

The mesh in URDF/SDF can be referenced with number of ways:

  • as simple reference relative to imported file, e.g. 'meshes/body.dae'
  • global reference /home/Michal/my_experimental_robot/meshes/body.dae
  • using prefixes like package://my_production_readyrobot_decription_package/assets/body.dae

I've taken a look at USD exported from Nvidia's Isaac Sim and there is a similar issue.

    def Xform "A1" (
        prepend references = @omniverse://localhost/NVIDIA/Assets/Isaac/2023.1.1/Isaac/Robots/Unitree/a1.usd@
    )

In USD from Isaac, the other USD file is referenced to an asset hosted with the Nucleus database. I do not use OpenUSD and USD in daily work, but I assume that there are multiple ways to reference other meshes and USD files here and User need to be informed that they need to copy to their Project source USD and all referenced files. Without that, import can be incomplete and the User needs to dig through Asset Processor logs to find out why.

2. Destructive reimport.
This issue is not resolved in Robot Importer.
Let's assume that the O3DE Editor User imports a scene from USD. It will create a number of prefabs that will instantiate each other. The O3DE Editor User will add some O3DE-specific components. The next day DCC User comes with the new version of scene USD and there is no way to preserve the O3DE Editor User work.
It can be circle-navigate around with prefab overrides.

… the importor. Breakout references and variants into separate phases

Signed-off-by: AMZN-Gene <genewalt@amazon.com>
Signed-off-by: AMZN-Gene <genewalt@amazon.com>
@AMZN-Gene
Copy link
Author

One question that I haven't seen in the RFC is the ability for O3DE to modify the imported USD so that the source DCC can get some changes and you end up with a back-and-forth workflow. In my opinion that would be fairly complicated but given that USD addition was pushed by some specific people, it might be best to ask if it is something they need or not.

Remedy does it but to do so, they decided to use USD as their scene file format which is a bold move (and creates a bunch of issues). Given that USD was created for communication between tools, but not really for long term stability of a scene (a change can creates a bunch of unexpected modification in a scene as shown in their GDC pres, some free info there as well)

I believe your approach of making a prefab out of the USD, so that its kept as an import format is best for stability and better control of the feature, easier diff etc.
Thanks!
I'll give mention to the possibility of O3DE exporting to USD in order to build a back-and-forth workflow between DCC under "Convert O3DE prefabs to USD" section.
Good to know that keeping USD separate from the native scene is a safer move.

Signed-off-by: AMZN-Gene <genewalt@amazon.com>
@AMZN-Gene
Copy link
Author

2. Destructive reimport.
This issue is not resolved in Robot Importer.
Let's assume that the O3DE Editor User imports a scene from USD. It will create a number of prefabs that will instantiate each other. The O3DE Editor User will add some O3DE-specific components. The next day DCC User comes with the new version of scene USD and there is no way to preserve the O3DE Editor User work.
It can be circle-navigate around with prefab overrides.

Exactly right. The procprefab generated by the AssetProcessor will only change based on the contents of the source USD, and not be modifiable by designers inside O3D. Designers can instead use that procprefab in their levels as a base prefab, in order to add/modify O3DE components to the original. If the base prefab changes, such as a new child entity is added, the designer's prefab will be updated to receive the new child entity, but still have their custom O3DE components.

…hould generate AssetProcessor errors to inform users to copy external USD references to their local project folder.

Signed-off-by: AMZN-Gene <genewalt@amazon.com>
@AMZN-Gene
Copy link
Author

User need to be informed that they need to copy to their Project source USD and all referenced files

Thanks! I've made note of this under "Phase 2 - Update Assimp to support USD with references" section

@moudgils
Copy link

@nick-l-o3de Can we merge this PR since the work is approved and we are already implementing parts of this? Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature Request: Create an RFC Documenting Universal Scene Description (USD) Support
5 participants