Create a Worker + Web Assembly + three.js version to understand if gives enhancements

[nicolocarpignoli]

We have to set a 'definitive' version of the NFT.

Currently, we have found that the best performances version, both on desktop and mobile devices, is the one that uses Web Workers.

We have to check if the version using web workers + WASM (web assembly) gives performances boost, in library loading and/or tracking.

If so, and the performance boost is significant, we can decide to set the 'wasm + workers' as default, NFT version of artoolkit. If not, and worker performances in respect of worker+wasm is pretty much the same, in my opinion, we should not use the WASM one as it adds another variable and complexity.

[nicolocarpignoli]

Both version have an example using them, under the nft_improved_worker folder.

We can start on testing them on both desktop and mobile and see differences.

[franbrennan]

Apologies for failing to keep up but I'm having trouble discerning exactly what content needs to be checked and where to access it (probably just due to me overlooking the obvious!).

If you could provide links, that would be brilliant. I'm happy, willing and able to test across multiple devices and platforms :)

[kalwalt]

@pepperface the two links to test:

1. NFT + Wasm + Worker https://kalwalt.github.io/jsartoolkit5/examples/nft_improved_worker/main_threejs_wasm_worker.html
2. NFT + Worker
   https://kalwalt.github.io/jsartoolkit5/examples/nft_improved_worker/main_threejs_worker.html

Thank you!

[franbrennan]

Great, cheers. I'll give everything a thorough check and report back here :)

[kalwalt]

For now ii did this tests about resources loading time:
no wasm version:

with wasm:

Tested on a WIko View model Android 7.1.2 Smartphone with latest Chrome.
har files:
kalwalt.github.io_wasm.zip
kalwalt.github.io_no_wasm.zip

[kalwalt]

You can see that artoolkit.min.js ( no wasm ! asm code...) load in 1.42s
instead the wasm version:
artoolkit_wasm.js 151 ms (milliseconds!)
artoolkit_wasm.wasm 405 ms.
You can see also a little difference in the NFT marker loading ( a bit faster in wasm) but should be verified...

[franbrennan]

Okay, here's a zip of a series of screen recordings from differing devices, platforms and browsers.

NFT_wasm_worker_tests.zip

Each test used a standard (as opposed to mobile broadband) WIFI connection and an A4-sized printout of the pinball NFT image. None of the screen recordings were edited in any way other than to trim their length shortly after the loads were successful.

The following is a list of the devices tested (the screen recordings are in separate folders for each device):-

1). Huawei Pro20 Lite (running Android 9.1.0 and using Chrome v 79.0.3945.116)

2). Huawei MediaPad (running Android 9.1.0 and using Chrome v 79.0.3945.116)

3). iPad (running iOS 13.3 and using Safari)

4). Desktop iMac i7 (running OS 10.15.2 and using Chrome Canary v 81.0.4026.0)

Old Phone Wildcard!
It's not in the zip as I didn't have the facility to record its screen but I ran a test on a Moto E (2nd gen running Android v 6.0 and using Chrome v 69.0.3497.100). Result: NFT+wasm+worker - load was about the same as the Huawei P20 Lite (the frame rate was good as well). 2). NFT+worker - I gave up waiting for something to happen 😉.

Hope some of this helps 🙂

[kalwalt]

Thank you, @pepperface for your time! i looked at your videos but unfortunately for some of the examples tested it'is difficult to see the fps or the ms. But in your opinion from this point of view which version is better?
Also maybe if i change ms to fps for a better comparison. it would be also nice to develop an output log with the final test (max. fps, min. fps and total time). I admit it's a bit difficult to make comparison in this sense.

[nicolocarpignoli]

Yes at least they should have the same meter of comparison, fps for both will be good

[kalwalt]

@nicolocarpignoli i will change it now!

[kalwalt]

i pushed and gh-pages is updated, everybody can test again. 😄

[kalwalt]

I think also that the two examples should be almost identical, the interpolation for sure add more computation and we need to compare them at the same level. I am of the opinion that it would be better if they had the same code or almost.

[nicolocarpignoli]

Ah yes, didn't see they were different. This changes things maybe. We should align them and add interpolation to the one missing it.

[kalwalt]

yes it's better to align both for this reason. I'm going to do this.

[kalwalt]

Ok i pushed the changes and updated gh-pages, i will do some other tests when i have time.

[nicolocarpignoli]

@pepperface @kalwalt and everyone else who are willing to help: can you please test again? white those, two same links above.
Only mobile counts (don't bother testing with desktop/mac).

And then we should add test entries here: https://docs.google.com/spreadsheets/d/1FykKO1dkPz0zUVuJCOPA2tHIs0Zs3nUybjLJcKihfs4/edit#gid=0

I have run a first test with my mobile phone.

Please remind two things:

• to sum ALL time of loading, of every resource, in ms
• to delete all chronology/files of your browser from one test to another (they are sharing files/assets, so if you don't wipe the memory of the browser, the second test will not be valid as it will use cached files).

Thanks!

[franbrennan]

Yup. I totally agree with all that's been said, above, and it all makes sense. I'll re-test first thing tomorrow morning and report back 🙂👍

[franbrennan]

Apologies for the delayed response... work stuff got in the way.

Latest test results all updated on the spreadsheet as requested 🙂

[nicolocarpignoli]

thank you so much! I see in general Wasm better than no wasm, but also I see you got very fast load in general...ms are like 1/3 than mine in my test. Are you sure there was no cache at all?

Anyway, thank you!

We will await other tests and see. Can't wait.

[franbrennan]

You're most welcome. I'm delighted to be able to help out a little. You dudes leave me for dust, in terms of coding... but testing I can do. To the best of my ability, anyway 🙂

I was scrupulously diligent about clearing all caches. However, like you I was surprised at how swiftly everything was loading; so I re-ran all the tests again (fastidiously clearing caches between each and every load)... twice 😅... before posting up the results.

Overall: 'With WASM on an iPad' was the best experience. Blisteringly fast, plus really smooth and stable 👍

[franbrennan]

@nicolocarpignoli - in case it helps shed some light on the load times I was achieving, I've just run an Internet connection speed check, here, and got the following result:

36.6 Mbps download
9.09 Mbps upload

[nicolocarpignoli]

@pepperface thinking a bit more about this, it's not important the absolute values but they're interesting relatively to each other. So comparing my wasm/nowasm result and your wasm/nowasm results. :)

[kalwalt]

i added also my test to the list. It confirm the general direction.

[nicolocarpignoli]

Ok so actually Wasm is ALWAYS better than no-wasm.

What's the downside to use it always as default version?
Is maybe WASM less readable, when debugging, of the other version?

[kalwalt]

What's the downside to use it always as default version?
Is maybe WASM less readable, when debugging, of the other version?

i didn't tested the artoolkit.debug.js plus WebWorker yet, i have always used the artoolkit.min.js in combination with the WebWorker. You can inspect WASM code but is in WASM, and so yes is less readable... at least to me.

[nicolocarpignoli]

ok. We should create a version with WASM + WORKER + THREEJS + GLTF. If it works ok, can be the default one (to replace current WORKER + THREEJS + GLTF).

[kalwalt]

ok. We should create a version with WASM + WORKER + THREEJS + GLTF. If it works ok, can be the default one (to replace current WORKER + THREEJS + GLTF).

This is a good idea!

[zkSharks]

Was a reference example ever written for WASM + WebWorker?