Load the renderer only when <model-viewer> is present

[Arecsu]

I'm building an online store and I'm using Swup.js to have animated page transitions. It uses ajax to replace the contents of the website without doing a full reload.

This store has 3D objects only in each product page. Nonetheless, I have to load the script as a module in every page because of the way Swup.js works. I cannot have a <script> for the product page and get it to load manually at some point without touching the code of the source script. They have to be bundled with the rest of the site.

I've noticed a good amount of stuttering when any page loads from cold. Tracing and profiling the site, I've noticed Renderer._singleton gets called on every page load, even though there may not be any <model-viewer> present in the page at that moment.

Renderer._singleton initializes a bunch of calls related to threejs and the canvas renderer, which takes a unnecessary hit to the main thread of the browser. Animations will stutter. Much more noticiable in mobile phones.

By moving the new Renderer declaration to static get singleton() we build the renderer only when the <model-viewer> HTML element is being constructed.

Apart from this, it would be awesome to make the process of creating the renderer much smoother and minimize the jank/stutter in the loading/construction process of the 3D scene. Is there a chance web workers could be used for this task? My knowledge regarding the render process is limited and I may be completely wrong on this assumption.

Thank you!

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[elalish]

Well noticed! This makes a lot of sense and is certainly in keeping with our strategy of only doing CPU work when necessary. Can you sign the CLA? It should be automatic for an individual.

[elalish]

As for web workers, I did look into this some years back. If I recall correctly, the difficulty is that the renderer has to live in one context or the other (worker thread or main thread) and it's quite difficult to synchronize the worker with the DOM updates, unlike on the main thread (that's effectively the purpose of a worker, to be async). We need tight synchronization for things like our poster and hotspots, so we didn't end up going that route.

[Arecsu]

Hey thank you Emmett!!

As for web workers, I did look into this some years back. If I recall correctly, the difficulty is that the renderer has to live in one context or the other (worker thread or main thread) and it's quite difficult to synchronize the worker with the DOM updates, unlike on the main thread (that's effectively the purpose of a worker, to be async). We need tight synchronization for things like our poster and hotspots, so we didn't end up going that route.

And thank for you feedback on this, really appreciated. I see. Hmmm and what about offload the creation of the renderer, load of textures and models to a webWorker and, after it finishes, moving everything to the main thread?

What causes jank and stutter is the loading process. The view of the 3D object itself will stutter no matter if it's offloaded or in the main thread due to heavy objects, textures and/or underpowered CPUs and GPUs anyways. In my experience, the jank and stutter created by the loading process tanks interactivity and animations at the beginning of a webpage, even with lightweight objects for like 2-3 seconds. Then, after loading, everything runs much better.

I didn't think it was possible until I've read this:

• https://stackoverflow.com/questions/48575106/using-webworker-and-indexeddb-for-loader-of-three-js/48620639#48620639
• https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm

Maybe there's a way out by offloading the geometry and texture parsing/loading to a webWorker and bringing them back to the main thread?

[elalish]

Jank for 2-3 seconds? That sounds terrible, but I haven't experienced it. Can you point me to a repro URL and what device you're testing with? I'm curious if something else is going on. Do you experience that on our example pages?

But yes, if you find a good solution, I would be happy to review and merge it. Performance improvement is always welcome!

[Arecsu]

Unfortunately the web is under development and can't share it, but I think it will be live soon. I'm using keen-slider to have a instagram-like horizontal carousel in the page where there's two elements, one of them being the 3D object. The load process,, once it fetches the data from the network, takes ~0.5-1.5s on my Galaxy S21 FE + Chrome. While this is happening, keen-slider becomes unresponsive to touchdown/mousedown events, even though the profiler captures those events correctly. I can see the flame graph showing dropped frames while the loading task is being executed. After then, there's a little bit of stutter for a couple of miliseconds more until everything gets in the right place somehow.

Chances are it is blocking the main thread from executing something else, particularly requestAnimationframe processes. Keen-slider is a JS library and it uses requestAnimationFrame to move the slider using transforms via CSS.

Maybe this is something unnoticeable under normal browser scroll events because they don't depend on javascript functions, the browser handles the whole DOM+painting. But if there's JS involved in updating the DOM while a 3D object is being loaded (and probably this JS function is using requestAnimationFrame), it won't get through if the device is not that powerful.

Hopefully I can report this back once I release the store and have a URL to play around!

[elalish]

We're good at keeping our own JS async enough to not block the main thread, especially model loading, but I think you may be having trouble with the loading of our library itself - just compiling that much JS can take some time. Have you looked through https://modelviewer.dev/examples/lighthouse.html? In any case, let's definitely keep this conversation going - I'm glad to have someone who is looking closely at page load performance.

[Arecsu]

Thank you so much Emmett. I do care about performance a lot to be honest 😊. Hmm it is not really the compiling process that takes that much time. This is the performance graph taken from my phone using the Chrome Android Dev Inspector

It is the initialization of the renderer that takes too much time to execute. I'm using model-viewer-effects, which adds somewhere around ~100ms to the render process. Still, without the model-viewer-effects there's noticeable jank and unresponsiveness with the mouse/touch events if the user happens to interact at the same time the render functions from the performance graphs are running. Which, happens often due to the nature of this website: a user loads the page and wants to scroll left/right to see the 3D model right away, and the page does not respond to the slide touch event due to it, making it look like it freezed the browser.

This is something I can share for now:

The 3D model -> https://cdn.shopify.com/3d/models/o/70cd7e8dd88acdba/McLovin-1024x.glb

And the performance trace: Trace-20240417T150618.zip

[elalish]

Hmm, actually it looks like most of your time is spent uploading textures to the GPU. I see your model has a bunch of solid-color textures - you should remove those and replace them with glTF factors on your materials. Those kinds of textures compress well, so they might seem small, but they get uncompressed on the way to the GPU, so their pixel dimensions are a big deal. You may also want to look into using KTX2 textures, as those can be uploaded in their compressed form to the GPU and save a lot of RAM and upload time.

But yes, moving these uploads off the main thread would be great - if you find a way, that would be excellent!

[Arecsu]

Wow that's actually quite useful. I remember to have experimented with KTX2, but at least the results I got were not good. The textures looked choppy, blocky, lacking finner details. If I wanted those details, the filesize would get huge. But I'll try it again in the upcoming weeks, maybe there's something more I can tweak to compress it in a better way.

Didn't know about those solid-color textures, will search for them. And yes, a good chunk of time is spend decoding the textures, which are in JPG format, and uploading them to the GPU.

Emmet, thank you so much for real. This information is valuable and maybe if I get inspired I'll look for ways to move some render processes out of the main thread. Glad my code has been merged!