Proposal for Improvements in GDCubism next version.

[MizunagiKB]

As we develop GDCubism v0.7, there are two areas we are considering for improvement.

If you are using GDCubism or are interested in it, please feel free to comment.

Comments can be in either Japanese or English.

Specifying the Size of Mask Rendering Textures

GDCubism focuses solely on displaying Live2D models, which results in significant GPU memory consumption. Especially when displaying at 4K (3840 x 2160) resolution, the memory usage quickly reaches several gigabytes.

This is because the mask images created during the rendering of Live2D models are generated at the same size as the final output, and since the mask images are realized with SubViewport, the more masks used in a single rendering, the more GPU memory is consumed (the back buffer cannot be reused).

While higher quality masks result in smoother edges, masks with a resolution higher than the actual output resolution do not seem to add much value. Therefore, we are considering allowing users to change the size of the mask images.

For example, even if rendering at a size of 4096x4096, it would be possible to set the mask to 256x256.

As an item, we plan to prepare a Vector2i item where (0, 0) would maintain the current behavior, and sizes of (1, 1) or larger would generate the mask image at that size.

Rendering Without Using SubViewport

The GDCubismUserModel used to render Live2D models in GDCubism is a class that inherits from SubViewport. This is to handle multiple MeshInstance2D as a single image, but we are considering whether it might be more convenient to separate this.

To achieve the goal of "displaying Live2D models," there seems to be a somewhat high barrier, and if the user understands SubViewport, they could place SubViewport as the parent of GDCubismUserClass themselves. (Combining purpose-specific Nodes rather than creating a highly functional and large Node seems to align more with the Godot Engine's philosophy.)

If SubViewport is not used, it would likely be necessary to adjust the mask to screen coordinates, which may require shader modifications.

---

GDCubism v0.7 を開発するにあたり、改善を考えている箇所が二箇所あります。

もしGDCubismを利用されていたり、興味がもっているのでしたらお気軽にコメントをお願いします。

日本語・英語、どちらでもコメント可能です。

マスク描画用テクスチャのサイズ指定

GDCubismはLive2Dモデルを表示する事だけに注力しているため、GPUメモリをかなり消費します。特に4K（3840 x 2160）解像度での表示を行うとすぐに数GBに達してしまいます。

これはLive2Dモデルの描画時に作成するマスク画像を最終出力サイズと同じサイズで生成していることと、マスク画像をSubViewportで実現しているため、一度の描画で使用されるマスクが多いほどGPUメモリを消費するためです。（バックバッファを使いまわせない）

マスクの品質が高い方がエッジが滑らかになるのですが、実際の解像度以上のマスクは出力結果にあまり意味をなさない気がするため、マスク画像のサイズを利用者側で変更出来る様にすることを考えています。

例えば4096x4096のサイズで描画していても、マスクは256x256にするといった事が可能になります。

項目としては、Vector2iの項目を用意して(0, 0)の場合は現状と互換の動作、サイズが(1, 1)以上であれば、そのサイズでマスク画像を生成するといったものにしようとしています。

SubViewportを使用せずに描画を行う方法

GDCubismでLive2Dモデルを描画するのに使用しているGDCubismUserModelは、SubViewportを継承したクラスにしています。これは複数のMeshInstance2Dから生成されたものを一枚の画像として扱うためなのですが、これを分離してしまう方が便利なのではないかと考えています。

「Live2Dモデルを表示したい」という目的を達成するには、やや高い壁がある様な気がしており、SubViewportを理解しているのであれば、利用者自身でGDCubismUserClassの親にSubViewportを配置すれば解決出来るだろうと考えます。（高機能で巨大なNodeを作るよりも目的特化Nodeを組み合わせる方がGodot Engineの流儀にもあっている気もします）

SubViewportを使用しない場合は、マスクをスクリーン座標にあわせて用意する必要があるため恐らくシェーダーの修正が必要になりそうです。直接描画時は描画結果も若干変わってしまうかもしれません。

[ALOBugTea]

I tried it on Android, it freeze the game.

I used the GDCubism at 4.3 is worked on window platform, and I want to run at android,
Then, it caused the freeze on android platform. I'm not sure what parts made game can't be runnable.
I know the android support is experimental, but still want to get a try on android platform.

When I build for android platform

Basically, I never try the godot-cpp old version to build for android platform,
it always caused "Required toolchain not found".
So... I build it after godot 4.3 updated.
godot-cpp have a update, and then I can build android platform now.

The issues when use

I try to install android template and output, it have some errors i got (they solved I guess... or not?).
The first of error called "GDCubismUserModel not found", I resolved it by built android output.
(Yup, this problem is caused by me never build the android libs)
Seconds of error called "No suitable library found for GDExtension", I resolved it on changing gd_cubism.gdextension's path,

# fixed to full path and it finally found.
android.debug.arm64 = "res://addons/gd_cubism/bin/libgd_cubism.android.debug.arm64.so"
android.release.arm64 = "res://addons/gd_cubism/bin/libgd_cubism.android.release.arm64.so"

Finally, it should be work if I'm not wrong...
It freeze the viewer scene and never moving on.

I can't handle the issue, hope it can be easy to imporve...

[MizunagiKB]

@ALOBugTea

This issue pertains to the features to be implemented in version 0.7.
The comment you made is a new issue, so I have prepared a separate issue for it elsewhere.

#104

[MizunagiKB]

It is now possible to make the viewport size for masks adjustable. This behavior is compatible with version 0.6, so it has been implemented as version 0.6.

https://github.com/MizunagiKB/gd_cubism/tree/0.6

This is an example where the SubViewport is used at 4096x4096, while the mask viewport is specified at 1024x1024 and 512x512.

[cjmaxik]

When I try to use Mask Viewport, it stops rendering the eyes (and other facial features such as blush), and I am not sure why.

UPD: "Adjust Scale" option is the culprit, if you have the scale less than 1, the mask viewport will be shifted towards (0;0) in scale. Mask Viewport does not follow the Scale properly.

[MizunagiKB]

@cjmaxik
I have created a new issue to address this bug. #108

[MizunagiKB]

About Rendering Without Using SubViewport

We have started experimenting with rendering methods that do not use SubViewport.
The current experimental implementation allows for direct rendering but comes with certain limitations.

If direct rendering is feasible, inheriting from SubViewport will no longer be necessary, and knowledge of SubViewport will be required. This will likely result in a lighter class.

However, this approach changes the way nodes are assembled, so it may be necessary to switch options at build time or provide two nodes.

Nonetheless, having two nearly identical nodes might be confusing for users.

[MizunagiKB]

In order to provide a more practical demonstration rather than just an explanation, I have committed the current test version.
https://github.com/MizunagiKB/gd_cubism/tree/feature_direct_rendering

Please note that this is intended for functionality verification, and it is not yet decided whether this will become the new version.

The implementation is as follows:

• There are some incompatibilities with the currently released GDCubismUserModel.
• This version has only been tested with demo_simple.tscn.
• GDCubismUserModel has been changed from inheriting SubViewport to inheriting Node2D.
• AutoScale, which was intended for SubViewport, does not function correctly (it only works with AutoScale = False).
• MaskViewport Size is only valid when x = 0, y = 0 (it generates a mask the same size as the screen, consuming a lot of memory).
• Transform is also applied to the SubViewport prepared as a Mask, so it correctly inherits the Transform of the parent node.