Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to render a renderable in different positions of one scene at a time #1513

Closed
flyingonly opened this issue Aug 13, 2019 · 22 comments · Fixed by #2607
Closed

How to render a renderable in different positions of one scene at a time #1513

flyingonly opened this issue Aug 13, 2019 · 22 comments · Fixed by #2607
Assignees
Labels
gltf Specific to glTF support

Comments

@flyingonly
Copy link

In our practical use, we need to render a renderable several times on one scene with different transforms at the same time. For now, we have tried to load a glb file for multiple times so that we have several filamentAssets at the same time. But it turns out to be a burden on the memory if the glb file has many textures, since each filament asset will load its own buffer and textures.
We wish to have an easy way to render a renderable more than once on one scene at a time, or to copy filamentAssets without load glb for multiple times. Is there a possible solution to this?

@prideout
Copy link
Contributor

We might be able to add a simple instancing feature to gltfio. Out of curiosity how many instances will you be creating, ballpark? Hundreds? Thousands?

@prideout prideout added the gltf Specific to glTF support label Aug 13, 2019
@prideout prideout self-assigned this Aug 13, 2019
@romainguy
Copy link
Collaborator

Renderables are fairly lightweight wrappers around vertex buffers, index buffers and material instances. It should be fairly trivial to add an API to at least clone a renderable. That would solve this request without using a ton of memory.

@flyingonly
Copy link
Author

We might be able to add a simple instancing feature to gltfio. Out of curiosity how many instances will you be creating, ballpark? Hundreds? Thousands?

Thanks for the reply. For now we just need about a dozen or so instances.

@gpyalt
Copy link

gpyalt commented Sep 3, 2019

@romainguy, Would it be possible instead to allow a renderable to store an optional array of mat4 transforms, which would allow its efficient rendering at various locations using OpenGL instancing?

@romainguy
Copy link
Collaborator

Instancing is something we'd like to do at some point but it's more complicated than just having multiple transforms since it could be useful to vary other attributes as well.

@gpyalt
Copy link

gpyalt commented Sep 3, 2019

Yes, that is true. Supporting a transform would be a good start though :-)

@prideout
Copy link
Contributor

prideout commented Sep 3, 2019

I've been thinking about this. Currently, FilamentAsset has unique ownership over vertex buffers, index buffers, and textures. If I changed these to use shared ownership, then I could add a clone() method to FilamentAsset that would create new a set of entities with new renderable & transformable components.

I think this would be a simple solution. Each clone would have unique transforms and material instances, but they would share their materials and vertex buffers.

@MingweiSamuel
Copy link
Contributor

How can I help add this feature?

I see currently it is possible to do this with Filamesh (since it exposes Vertex and IndexBuffer), just not with gtfio, is that right? What changes need to be made in gtfio?

@prideout
Copy link
Contributor

I still think that adding a public clone() method to FilamentAsset is the way to go. On the implementation side, perhaps all Filament objects owned by FilamentAsset could be wrapped in a shared_ptr with a custom deleter.

@MingweiSamuel
Copy link
Contributor

Wouldn't cloning FilamentAsset potentially end up cloning a lot more than necessary? Cloning Renderable instances seems more precise

@prideout
Copy link
Contributor

It would not be a deep clone, the textures etc would be shared, which is why I mentioned the potential use of shared_ptr to help with the implementation. The cloned FilamentAsset would have a separate set of entities, components, and material instances. The textures, vertex buffers, and index buffers would be shared.

@MingweiSamuel
Copy link
Contributor

MingweiSamuel commented May 23, 2020

It just seems like a waste if you need to clone a particular mesh renderable a dozen times that you would also have to clone the texture material instances and light etc handles to just to be thrown out.

For my current use I actually only need to clone Renderables' meshes which would only require making the VertexBuffers and IndexBuffers accessible. But then the question is how to match up the entities (since not every one is necessarily a Renderable) and also things like what PrimitiveType is it ..

std::vector<filament::VertexBuffer*> mVertexBuffers;
std::vector<filament::IndexBuffer*> mIndexBuffers;

@romainguy
Copy link
Collaborator

But you wouldn't clone the textures or lights, just the renderables.

@prideout
Copy link
Contributor

Yes, the Renderable and Transformable components would be cloned (i.e. the entity hierarchy) as well as Material Instances. Textures and buffers would gain shared ownership semantics so that we can release them properly.

Using the clone() name for the API might be a bit confusing because the above behavior wouldn't be obvious to users, plus there are other ambiguities (e.g. would overridden parameter values get cloned too?).

Perhaps a better way to expose this functionality would be to add a createAssetInstances(...) method to AssetLoader which would allow users to create a number of instances. This would avoid confusion about parameter values and "clone" terminology.

@romainguy
Copy link
Collaborator

You only need to clone Material instances if you want to modify the materials of the clones. Otherwise they can be shared too.

@romainguy
Copy link
Collaborator

Also @prideout clone() has a specific meaning in Java, we should avoid using that name.

@MingweiSamuel
Copy link
Contributor

MingweiSamuel commented May 26, 2020

So I can see FilamentAsset keeps its cgltf_data* srcAsset at least until releaseSourceAsset() is called. So it would be relatively simple to pass that to AssetLoader.createAsset(srcAsset) to create copies. However AssetLoader also stores a lot of temporary state which gets cleared on each createAsset() call:

// The loader owns a few transient mappings used only for the current asset being loaded.
FFilamentAsset* mResult;
MatInstanceCache mMatInstanceCache;
MeshCache mMeshCache;

If mMatInstanceCache and mMeshCache were instead stored in FilamentAsset alongside mSrcAsset, and there was also a new third mTextureCache then you'd have everything you need recreate assets using those caches.

Then there could be a function AssetLoader.createAssetFromAsset(FilamentAsset) as @romainguy suggested in #2344 which returns a new "cloned" FilamentAsset. The method would fail if releaseSourceAsset() had already been called on the input asset.

This way it doesn't require any new loading/entity creation logic. But we would need to change some things in FilamentAsset:

  • There are now mSrcAsset, the moved mMeshCache & mMatInstanceCache, and the new mTextureCache as the original source assets.
    • We can have createAssetFromAsset(...) only work on the original copy. So the clone(s) don't store mSrcAsset & caches, as if releaseSourceAsset() had been called. This means we don't have to mess with the mSourceAssetRefCount logic used by ResourceLoader.cpp.
  • These guys need shared_ptr. I think only one ptr for everything, put them in a new struct?
    std::vector<filament::VertexBuffer*> mVertexBuffers;
    std::vector<filament::IndexBuffer*> mIndexBuffers;
    std::vector<filament::Texture*> mTextures;
    filament::Aabb mBoundingBox;
  • I don't think they are needed on the transient source data since pointers in the previous bullet keep these alive, but I'm not confident.

Ok that was a wall of text hopefully makes sense.

prideout added a commit that referenced this issue May 27, 2020
This new API concept uses a "Master Asset + Instances" ownership model
which lets us avoid complex shared ownership semantics. Instances have
their own API, which turns out to make a lot of sense. For example,
light sources and material instances are not instanced, and therefore
are not accessible through the FilamentInstance interface.

Issue #1513
prideout added a commit that referenced this issue May 28, 2020
This adds createInstancedAsset() to AssetLoader, which creates a master
asset and a set of slave instances. Vertex buffers, index buffers,
textures, and material instances are shared. Entities and components are
duplicated.

Instances have their own API object that is very simple. Light sources
and material instances are not instanced, so are not accessible through
the instance API object.

The master-slave ownership model lets us avoid complex shared ownership
semantics. The existing cache structures in AssetLoader allow the
implementation of this feature to be fairly simple.

The master asset exposes the union of all entities and allows clients to
modify all instances en masse if they wish. This design also works
naturally with ResourceLoader, which does not need to know about
instancing. For example, asynchronous loading is completely unchanged;
the dependency graph simply contains the union of entities across all
instances.

Fixes #1513.
@prideout
Copy link
Contributor

I have this implemented locally (sneak peek in 72fe00d) but I need to do more testing, add Java / JavaScript bindings, etc.

The API is quite different from earlier proposals but it is simple and works quite well. For example, it does not use shared_ptr in its implementation.

prideout added a commit that referenced this issue May 28, 2020
This adds createInstancedAsset() to AssetLoader, which creates a master
asset and a set of slave instances. Vertex buffers, index buffers,
textures, and material instances are shared. Entities and components are
duplicated.

Instances have their own API object that is very simple. Light sources
and material instances are not instanced, so are not accessible through
the instance API object.

The master-slave ownership model lets us avoid complex shared ownership
semantics. The existing cache structures in AssetLoader allow the
implementation of this feature to be fairly simple.

The master asset exposes the union of all entities and allows clients to
modify all instances en masse if they wish. This design also works
naturally with ResourceLoader, which does not need to know about
instancing. For example, asynchronous loading is completely unchanged;
the dependency graph simply contains the union of entities across all
instances.

Fixes #1513.
prideout added a commit that referenced this issue May 28, 2020
This adds createInstancedAsset() to AssetLoader, which creates a master
asset and a set of slave instances. Vertex buffers, index buffers,
textures, and material instances are shared. Entities and components are
duplicated.

Instances have their own API object that is very simple. Light sources
and material instances are not instanced, so are not accessible through
the instance API object.

The master-slave ownership model lets us avoid complex shared ownership
semantics. The existing cache structures in AssetLoader allow the
implementation of this feature to be fairly simple.

The master asset exposes the union of all entities and allows clients to
modify all instances en masse if they wish. This design also works
naturally with ResourceLoader, which does not need to know about
instancing. For example, asynchronous loading is completely unchanged;
the dependency graph simply contains the union of entities across all
instances.

Fixes #1513.
prideout added a commit that referenced this issue May 28, 2020
prideout added a commit that referenced this issue May 28, 2020
When animation is applied to the master asset, all instances are
animated.

Instances can also be individually animated via the Animator in
FilamentInstance.

Fixes #1513.
prideout added a commit that referenced this issue May 28, 2020
This adds createInstancedAsset() to AssetLoader, which creates a master
asset and a set of slave instances. Vertex buffers, index buffers,
textures, and material instances are shared. Entities and components are
duplicated.

Instances have their own API object that is very simple. Light sources
and material instances are not instanced, so are not accessible through
the instance API object.

The master-slave ownership model lets us avoid complex shared ownership
semantics. The existing cache structures in AssetLoader allow the
implementation of this feature to be fairly simple.

The master asset exposes the union of all entities and allows clients to
modify all instances en masse if they wish. This design also works
naturally with ResourceLoader, which does not need to know about
instancing. For example, asynchronous loading is completely unchanged;
the dependency graph simply contains the union of entities across all
instances.

Support for animation is added in a subsequent commit.

Fixes #1513.
prideout added a commit that referenced this issue May 28, 2020
When animation is applied to the master asset, all instances are
animated.

Instances can also be individually animated via the Animator in
FilamentInstance.

Fixes #1513.
prideout added a commit that referenced this issue May 28, 2020
prideout added a commit that referenced this issue May 28, 2020
prideout added a commit that referenced this issue May 28, 2020
When animation is applied to the master asset, all instances are
animated.

Instances can also be individually animated via the Animator in
FilamentInstance.

Fixes #1513.
prideout added a commit that referenced this issue May 28, 2020
prideout added a commit that referenced this issue May 28, 2020
This adds createInstancedAsset() to AssetLoader, which creates a master
asset and a set of slave instances. Vertex buffers, index buffers,
textures, and material instances are shared. Entities and components are
duplicated.

Instances have their own API object that is very simple. Light sources
and material instances are not instanced, so are not accessible through
the instance API object.

The master-slave ownership model lets us avoid complex shared ownership
semantics. The existing cache structures in AssetLoader allow the
implementation of this feature to be fairly simple.

The master asset exposes the union of all entities and allows clients to
modify all instances en masse if they wish. This design also works
naturally with ResourceLoader, which does not need to know about
instancing. For example, asynchronous loading is completely unchanged;
the dependency graph simply contains the union of entities across all
instances.

Support for animation is added in a subsequent commit.

Fixes #1513.
prideout added a commit that referenced this issue May 28, 2020
When animation is applied to the master asset, all instances are
animated.

Instances can also be individually animated via the Animator in
FilamentInstance.

Fixes #1513.
prideout added a commit that referenced this issue May 28, 2020
prideout added a commit that referenced this issue May 28, 2020
This adds createInstancedAsset() to AssetLoader, which creates a master
asset and a set of slave instances. Vertex buffers, index buffers,
textures, and material instances are shared. Entities and components are
duplicated.

Instances have their own API object that is very simple. Light sources
and material instances are not instanced, so are not accessible through
the instance API object.

The master-slave ownership model lets us avoid complex shared ownership
semantics. The existing cache structures in AssetLoader allow the
implementation of this feature to be fairly simple.

The master asset exposes the union of all entities and allows clients to
modify all instances en masse if they wish. This design also works
naturally with ResourceLoader, which does not need to know about
instancing. For example, asynchronous loading is completely unchanged;
the dependency graph simply contains the union of entities across all
instances.

Support for animation is added in a subsequent commit.

Fixes #1513.
prideout added a commit that referenced this issue May 28, 2020
When animation is applied to the master asset, all instances are
animated.

Instances can also be individually animated via the Animator in
FilamentInstance.

Fixes #1513.
prideout added a commit that referenced this issue May 28, 2020
prideout added a commit that referenced this issue May 29, 2020
This adds createInstancedAsset() to AssetLoader, which creates a master
asset and a set of slave instances. Vertex buffers, index buffers,
textures, and material instances are shared. Entities and components are
duplicated.

Instances have their own API object that is very simple. Light sources
and material instances are not instanced, so are not accessible through
the instance API object.

The master-slave ownership model lets us avoid complex shared ownership
semantics. The existing cache structures in AssetLoader allow the
implementation of this feature to be fairly simple.

The master asset exposes the union of all entities and allows clients to
modify all instances en masse if they wish. This design also works
naturally with ResourceLoader, which does not need to know about
instancing. For example, asynchronous loading is completely unchanged;
the dependency graph simply contains the union of entities across all
instances.

Support for animation is added in a subsequent commit.

Fixes #1513.
prideout added a commit that referenced this issue May 29, 2020
When animation is applied to the master asset, all instances are
animated.

Instances can also be individually animated via the Animator in
FilamentInstance.

Fixes #1513.
prideout added a commit that referenced this issue May 29, 2020
@MingweiSamuel
Copy link
Contributor

@prideout Great stuff, thanks so much!

@davidwarford
Copy link

davidwarford commented Nov 3, 2020

Hay sorry to comment on such an old issue, But i came across this issue while trying to create an array of meshes and i just can't figure it out. and i don't want to make a new issue. @prideout

@romainguy
Copy link
Collaborator

If you are using gltfio, we recently added an API to create multiple instance of the asset. See #3225

@prideout
Copy link
Contributor

prideout commented Nov 3, 2020

@davidwarford gltfio has had support for software instancing since May and you should be able to use createInstancedAsset from JavaScript although I think our only demo for this is in C++ (gltf_instances.cpp).

The JS version of createInstancedAsset takes a pre-sized array for its second argument, and fills it in with instance objects.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
gltf Specific to glTF support
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants