Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Emscripten 3.1.19 #7056

Merged
merged 18 commits into from
Nov 19, 2022
Merged

Emscripten 3.1.19 #7056

merged 18 commits into from
Nov 19, 2022

Conversation

Jonathhhan
Copy link
Contributor

@Jonathhhan Jonathhhan commented Aug 21, 2022

This is a bit(e) size pull request for Emscripten 3.1.19.
Here are some additional details: https://forum.openframeworks.cc/t/emscripten-3-1-19/40294
Most of the changes in ofxAppEmscriptenWindow.cpp are from @roymacdonald
I left out all the Audioworklet changes, but as soon as they are part of the main Emscripten branch, it would be nice to implement them: emscripten-core/emscripten#16449

@Jonathhhan
Copy link
Contributor Author

Jonathhhan commented Aug 25, 2022

I guess, the checks are not successful because I added -s DYNCALLS=1 which is needed for audio input and output (heap access?).

@ofTheo
Copy link
Member

ofTheo commented Aug 25, 2022

@Jonathhhan does it mean we need to change the emscripten version used by the CI here?

https://github.com/openframeworks/openFrameworks/blob/master/.github/workflows/build-emscripten.yml#L24

The rest of the PR looks good to me.

@Jonathhhan
Copy link
Contributor Author

@ofTheo not sure, because I build the libs directly, without docker.

@ofTheo
Copy link
Member

ofTheo commented Aug 26, 2022

@Jonathhhan I guess I am wondering, what the DYNCALLS=1 means in terms of OF.
Do we need to build all libraries OF uses for emscripten with a certain flag, is it just one library?

Or is it just about passing the flag to emscripten ( right now the flag is not recognized ).
Maybe its just the emscripten version that needs updating?

Could you push to your branch with a change to this line:
https://github.com/openframeworks/openFrameworks/blob/master/.github/workflows/build-emscripten.yml#L24

from:

$PWD:/src emscripten/emsdk:1.40.0 bash

to

$PWD:/src emscripten/emsdk:3.1.19 bash

Then the OF CI will be running the same version as you 🙂

@Jonathhhan
Copy link
Contributor Author

Jonathhhan commented Aug 26, 2022

@ofTheo I made the change in https://github.com/openframeworks/openFrameworks/blob/master/.github/workflows/build-emscripten.yml#L24 I actually compiled the libs with that flag, but it does not seem to make a difference (the mentioned changes are still necessary). And maybe I did not use that file at all. I compiled with something like: -t emscripten update all core libs
DYNCALL is only needed for PLATFORM_LDFLAGS = -Wl --gc-sections --preload-file bin/data@data --emrun --bind --profiling-funcs -s USE_FREETYPE=1 -s DYNCALLS=1, the libs do not need that flag. And I can only confirm that -s DYNCALLS=1 is needed for the audio input and output of ofxPd, but I guess it could be necessary for other audio processing, too.
This is the part in library_html5audio.js where it seems to be needed:

		var inbufferArray = Module.HEAPF32.subarray(inbuffer>>2,(inbuffer>>2)+bufferSize*inputChannels);
		var outbufferArray = Module.HEAPF32.subarray(outbuffer>>2,(outbuffer>>2)+bufferSize*outputChannels);

@ofTheo
Copy link
Member

ofTheo commented Aug 26, 2022

Thanks @Jonathhhan

I think maybe I'll need to build the apothecary stuff against 3.1.19 and get those updated in the apothecary repo and then hopefully this PR will compile fine :)

@dimitre
Copy link
Member

dimitre commented Aug 27, 2022

Not sure if it changes anything but I've noticed 3.1.20 is out

@Jonathhhan
Copy link
Contributor Author

@dimitre yes, it makes sense to compile with the latest version.

@Jonathhhan
Copy link
Contributor Author

I replaced dynCall('viiii',callback, [bufferSize,inputChannels,outputChannels,userData]); with
{{{ makeDynCall('viiii', 'callback') }}}(bufferSize,inputChannels,outputChannels,userData); in library_html5audio.js.
With that DYNCALLS=1 is not needed anymore.

@Jonathhhan
Copy link
Contributor Author

Jonathhhan commented Aug 28, 2022

I made a first attempt to implement the new audio worklet branch (see the end of the Emscripten link in the first post of this thread). Not working yet, but should be doable. Maybe someone more experienced can have a look. It is not part of this pull request.

@Jonathhhan
Copy link
Contributor Author

Jonathhhan commented Sep 8, 2022

I made a first working example with the current audioWorklet branch. It is very hacky and I needed to edit some of the Emscripten files. It may be also possible to implement it much better and without editing the emscripten files. But maybe it is helpful as a first step. https://github.com/Jonathhhan/openFrameworks/tree/emscripten_3.1.9_audioWorklet/addons/ofxEmscripten
It works with this Emscripten branch: https://github.com/juj/emscripten/tree/audio_worklets
audio_worklet.js and library_webaudio.js
need to be copied into emscripten/src and webaudio.h and html5audio.h need to be copied into emscripten/system/include/emscripten
And 3.1.21 is released...

EGLContext ofxAppEmscriptenWindow::getEGLContext(){
return context;
shared_ptr<ofBaseRenderer> & ofxAppEmscriptenWindow::renderer(){
return _renderer;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious if all this stuff needs to be removed?
Do we not have a context anymore?

Copy link
Contributor Author

@Jonathhhan Jonathhhan Nov 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

most of the changes in this file are from @roymacdonald: #6781 (comment)
as far as I can tell it is a web_gl context now:
context = emscripten_webgl_create_context("#canvas", &attrs);

@ofTheo
Copy link
Member

ofTheo commented Nov 17, 2022

Re-running the emscripten job now that Freeimage is updated in apothecary.

@Jonathhhan @themancalledjakob - if this passes let me know if you think its good to merge or if there is any small fixes cleanup that needs to happen.

@ofTheo
Copy link
Member

ofTheo commented Nov 17, 2022

Currently erroring with a linking error:

File "/emsdk/upstream/emscripten/tools/building.py", line 578, in parse_llvm_nm_symbols
    status = line[entry_pos + 11] # Skip address, which is always fixed-length 8 chars.
IndexError: string index out of range

This seems to suggest that maybe apothecary might need to be building libs with the newer SDK?
emscripten-core/emscripten#17826

@Jonathhhan
Copy link
Contributor Author

Jonathhhan commented Nov 18, 2022

Currently erroring with a linking error:

File "/emsdk/upstream/emscripten/tools/building.py", line 578, in parse_llvm_nm_symbols
    status = line[entry_pos + 11] # Skip address, which is always fixed-length 8 chars.
IndexError: string index out of range

This seems to suggest that maybe apothecary might need to be building libs with the newer SDK? emscripten-core/emscripten#17826

@ofTheo thats probably the case, maybe because they updated llvm in 3.1.25: https://github.com/emscripten-core/emscripten/blob/main/ChangeLog.md

attrs.antialias = 1;
attrs.majorVersion = 2;
attrs.minorVersion = 0;
attrs.alpha = 0;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we be passing in any other settings from the settings argument?

Copy link
Contributor Author

@Jonathhhan Jonathhhan Nov 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if the other settings are needed in some cases...

@ofTheo
Copy link
Member

ofTheo commented Nov 18, 2022

Okay now with libs in apothecary being built with 3.1.21 this looks like it all works.
Only thing I am not sure is if we need to provide an option for returning EMSCRIPTEN_WEBGL_CONTEXT_HANDLE similar to how we used to do.

Maybe I'll merge this in for now and we can do some smaller PRs for other things.

@Jonathhhan does this fix the fullscreen issues?

@Jonathhhan
Copy link
Contributor Author

Jonathhhan commented Nov 19, 2022

@ofTheo now it includes the fullscreen fix (also added this change: #7152).

@ofTheo ofTheo merged commit b49bf28 into openframeworks:master Nov 19, 2022
@ofTheo
Copy link
Member

ofTheo commented Nov 19, 2022

Thanks @Jonathhhan !!

@Jonathhhan
Copy link
Contributor Author

Jonathhhan commented Nov 19, 2022

@ofTheo maybe just update it for 3.1.26 (if it is not difficult - but I guess that only the apothecary libs need to be recompiled. I use 3.1.25 at the moment)?

@themancalledjakob
Copy link
Contributor

Out of curiosity I just tried compiling all apothecary libs and this branch with emsdk 1.40.0, and except using emrun to launch a dev server everything seems to just work. running something like php -S 0.0.0.0:8080 in bin was able to launch the project successfully though, so hopefully there is no real big issue with being backwards compatible.

of course otherwise it's better to just compile with 3.1.25 or 3.1.26 (it seems as if there are is a new version almost every other week atm)

@themancalledjakob
Copy link
Contributor

to get a better overview over what works and what doesn't, I thought it would be good to have a gallery of all examples.

I made a little sketch of how this may work, it's super rough. but it might help with getting an overview.
I uploaded it here with the current status. compiled with emsdk 3.1.25 + downloaded libs from apothecary

if you want to try it yourself you need to download this: https://pointer.click/files/testAllExamples.zip and put it in your $OF_ROOT/scripts/emscripten directory. then run testAllExamples.sh
it will create a gallery with all examples in there, a bit threejs style (but much simpler).
you can click on the examples to load them in the preview iframe.
if there is a preview image, it will use it in the list. otherwise it will use a grey placeholder image. but if there was an error compiling, the preview image will be red and the error output from make will be loaded in the iframe instead of the example.

note: some examples also have runtime errors when loading them. these errors are not red, you'd have to open the example to catch them. and of course, some examples cannot work, but some could possibly be fixed.

also I noticed there are some glitches and the whole thing could be more elegant, but what do you think, can this be useful?

@Jonathhhan
Copy link
Contributor Author

Jonathhhan commented Nov 19, 2022

@themancalledjakob thanks. I think this is really useful for finding out what is working (I already recognized some examples that were working before, like the 3DPrimitivesExample).

@Jonathhhan
Copy link
Contributor Author

Jonathhhan commented Nov 19, 2022

The 3DPrimitivesExample works if I comment out this line:

function _html5video_grabber_pixel_format(id) {
    // return allocate(intArrayFromString(VIDEO.grabbers[id].pixelFormat), "i8", ALLOC_STACK)
}

Otherwise this is the error message:
ALLOC_STACK is a library symbol and not included by default; add it to your library.js __deps or to DEFAULT_LIBRARY_FUNCS_TO_INCLUDE on the command line (e.g. -sDEFAULT_LIBRARY_FUNCS_TO_INCLUDE=$ALLOC_STACK)

Uncaught TypeError: allocate is not a function
    at _html5video_grabber_pixel_format (3DPrimitivesExample.js:formatted:8809:12)
    at ofxEmscriptenVideoGrabber::getPixelFormat() const (3DPrimitivesExample.wasm:0x3220c)
    at ofVideoGrabber::setPixelFormat(ofPixelFormat) (3DPrimitivesExample.wasm:0x4ae8a)
    at ofVideoGrabber::setup(int, int, bool) (3DPrimitivesExample.wasm:0x4aa35)
    at ofApp::setup() (3DPrimitivesExample.wasm:0x22f50)
    at ofNode::onParentOrientationChanged(glm::qua<float, (glm::qualifier)0>&) (3DPrimitivesExample.wasm:0x4d5a5)
    at std::__2::__function::__func<std::__2::shared_ptr<of::priv::Function<ofKeyEventArgs, std::__2::recursive_mutex>> ofEvent<ofKeyEventArgs, std::__2::recursive_mutex>::make_function<ofMainLoop>(ofMainLoop*, void (ofMainLoop::*)(ofKeyEventArgs&), int)::'lambda'(void const*, ofKeyEventArgs&), std::__2::allocator<std::__2::shared_ptr<of::priv::Function<ofKeyEventArgs, std::__2::recursive_mutex>> ofEvent<ofKeyEventArgs, std::__2::recursive_mutex>::make_function<ofMainLoop>(ofMainLoop*, void (ofMainLoop::*)(ofKeyEventArgs&), int)::'lambda'(void const*, ofKeyEventArgs&)>, bool (void const*, ofKeyEventArgs&)>::operator()(void const*&&, ofKeyEventArgs&) (3DPrimitivesExample.wasm:0x94f58)
    at ofEvent<ofEventArgs, std::__2::recursive_mutex>::notify(ofEventArgs&) (3DPrimitivesExample.wasm:0x5e4b7)
    at ofCoreEvents::notifySetup() (3DPrimitivesExample.wasm:0x5e375)
    at ofxAppEmscriptenWindow::loop() (3DPrimitivesExample.wasm:0x2f209)
_html5video_grabber_pixel_format @ 3DPrimitivesExample.js:formatted:8809
$ofxEmscriptenVideoGrabber::getPixelFormat() const @ 3DPrimitivesExample.wasm:0x3220c
$ofVideoGrabber::setPixelFormat(ofPixelFormat) @ 3DPrimitivesExample.wasm:0x4ae8a
$ofVideoGrabber::setup(int, int, bool) @ 3DPrimitivesExample.wasm:0x4aa35
$ofApp::setup() @ 3DPrimitivesExample.wasm:0x22f50
$ofNode::onParentOrientationChanged(glm::qua<float, (glm::qualifier)0>&) @ 3DPrimitivesExample.wasm:0x4d5a5
$std::__2::__function::__func<std::__2::shared_ptr<of::priv::Function<ofKeyEventArgs, std::__2::recursive_mutex>> ofEvent<ofKeyEventArgs, std::__2::recursive_mutex>::make_function<ofMainLoop>(ofMainLoop*, void (ofMainLoop::*)(ofKeyEventArgs&), int)::'lambda'(void const*, ofKeyEventArgs&), std::__2::allocator<std::__2::shared_ptr<of::priv::Function<ofKeyEventArgs, std::__2::recursive_mutex>> ofEvent<ofKeyEventArgs, std::__2::recursive_mutex>::make_function<ofMainLoop>(ofMainLoop*, void (ofMainLoop::*)(ofKeyEventArgs&), int)::'lambda'(void const*, ofKeyEventArgs&)>, bool (void const*, ofKeyEventArgs&)>::operator()(void const*&&, ofKeyEventArgs&) @ 3DPrimitivesExample.wasm:0x94f58
$ofEvent<ofEventArgs, std::__2::recursive_mutex>::notify(ofEventArgs&) @ 3DPrimitivesExample.wasm:0x5e4b7
$ofCoreEvents::notifySetup() @ 3DPrimitivesExample.wasm:0x5e375
$ofxAppEmscriptenWindow::loop() @ 3DPrimitivesExample.wasm:0x2f209
$std::__2::__function::__func<void (*)(), std::__2::allocator<void (*)()>, void ()>::operator()() @ 3DPrimitivesExample.wasm:0x94a0e
$main @ 3DPrimitivesExample.wasm:0x21afe
(anonymous) @ 3DPrimitivesExample.js:formatted:975
callMain @ 3DPrimitivesExample.js:formatted:10163
doRun @ 3DPrimitivesExample.js:formatted:10208
(anonymous) @ 3DPrimitivesExample.js:formatted:10217
setTimeout (async)
run @ 3DPrimitivesExample.js:formatted:10213
runCaller @ 3DPrimitivesExample.js:formatted:10151
removeRunDependency @ 3DPrimitivesExample.js:formatted:941
processPackageData @ 3DPrimitivesExample.js:formatted:308
getRequest.onsuccess @ 3DPrimitivesExample.js:formatted:269

@ofTheo
Copy link
Member

ofTheo commented Nov 19, 2022

wow - this is super @themancalledjakob and something that could be great to have hosted on the site ( once the kinks are worked out ).

makes seeing potential issues easier to see.
would be awesome if we could somehow redirect the console error messages to the main console to make it easier to track down.

@themancalledjakob
Copy link
Contributor

cool :) i'm happy this is useful!

would be awesome if we could somehow redirect the console error messages to the main console to make it easier to track down.

which main console do you mean?

@themancalledjakob
Copy link
Contributor

The 3DPrimitivesExample works if I comment out this line:

function _html5video_grabber_pixel_format(id) {
    // return allocate(intArrayFromString(VIDEO.grabbers[id].pixelFormat), "i8", ALLOC_STACK)
}

Otherwise this is the error message: ALLOC_STACK is a library symbol and not included by default; add it to your library.js __deps or to DEFAULT_LIBRARY_FUNCS_TO_INCLUDE on the command line (e.g. -sDEFAULT_LIBRARY_FUNCS_TO_INCLUDE=$ALLOC_STACK)

Uncaught TypeError: allocate is not a function
    at _html5video_grabber_pixel_format (3DPrimitivesExample.js:formatted:8809:12)
    at ofxEmscriptenVideoGrabber::getPixelFormat() const (3DPrimitivesExample.wasm:0x3220c)
    at ofVideoGrabber::setPixelFormat(ofPixelFormat) (3DPrimitivesExample.wasm:0x4ae8a)
    at ofVideoGrabber::setup(int, int, bool) (3DPrimitivesExample.wasm:0x4aa35)
    at ofApp::setup() (3DPrimitivesExample.wasm:0x22f50)
    at ofNode::onParentOrientationChanged(glm::qua<float, (glm::qualifier)0>&) (3DPrimitivesExample.wasm:0x4d5a5)
    at std::__2::__function::__func<std::__2::shared_ptr<of::priv::Function<ofKeyEventArgs, std::__2::recursive_mutex>> ofEvent<ofKeyEventArgs, std::__2::recursive_mutex>::make_function<ofMainLoop>(ofMainLoop*, void (ofMainLoop::*)(ofKeyEventArgs&), int)::'lambda'(void const*, ofKeyEventArgs&), std::__2::allocator<std::__2::shared_ptr<of::priv::Function<ofKeyEventArgs, std::__2::recursive_mutex>> ofEvent<ofKeyEventArgs, std::__2::recursive_mutex>::make_function<ofMainLoop>(ofMainLoop*, void (ofMainLoop::*)(ofKeyEventArgs&), int)::'lambda'(void const*, ofKeyEventArgs&)>, bool (void const*, ofKeyEventArgs&)>::operator()(void const*&&, ofKeyEventArgs&) (3DPrimitivesExample.wasm:0x94f58)
    at ofEvent<ofEventArgs, std::__2::recursive_mutex>::notify(ofEventArgs&) (3DPrimitivesExample.wasm:0x5e4b7)
    at ofCoreEvents::notifySetup() (3DPrimitivesExample.wasm:0x5e375)
    at ofxAppEmscriptenWindow::loop() (3DPrimitivesExample.wasm:0x2f209)
_html5video_grabber_pixel_format @ 3DPrimitivesExample.js:formatted:8809
$ofxEmscriptenVideoGrabber::getPixelFormat() const @ 3DPrimitivesExample.wasm:0x3220c
$ofVideoGrabber::setPixelFormat(ofPixelFormat) @ 3DPrimitivesExample.wasm:0x4ae8a
$ofVideoGrabber::setup(int, int, bool) @ 3DPrimitivesExample.wasm:0x4aa35
$ofApp::setup() @ 3DPrimitivesExample.wasm:0x22f50
$ofNode::onParentOrientationChanged(glm::qua<float, (glm::qualifier)0>&) @ 3DPrimitivesExample.wasm:0x4d5a5
$std::__2::__function::__func<std::__2::shared_ptr<of::priv::Function<ofKeyEventArgs, std::__2::recursive_mutex>> ofEvent<ofKeyEventArgs, std::__2::recursive_mutex>::make_function<ofMainLoop>(ofMainLoop*, void (ofMainLoop::*)(ofKeyEventArgs&), int)::'lambda'(void const*, ofKeyEventArgs&), std::__2::allocator<std::__2::shared_ptr<of::priv::Function<ofKeyEventArgs, std::__2::recursive_mutex>> ofEvent<ofKeyEventArgs, std::__2::recursive_mutex>::make_function<ofMainLoop>(ofMainLoop*, void (ofMainLoop::*)(ofKeyEventArgs&), int)::'lambda'(void const*, ofKeyEventArgs&)>, bool (void const*, ofKeyEventArgs&)>::operator()(void const*&&, ofKeyEventArgs&) @ 3DPrimitivesExample.wasm:0x94f58
$ofEvent<ofEventArgs, std::__2::recursive_mutex>::notify(ofEventArgs&) @ 3DPrimitivesExample.wasm:0x5e4b7
$ofCoreEvents::notifySetup() @ 3DPrimitivesExample.wasm:0x5e375
$ofxAppEmscriptenWindow::loop() @ 3DPrimitivesExample.wasm:0x2f209
$std::__2::__function::__func<void (*)(), std::__2::allocator<void (*)()>, void ()>::operator()() @ 3DPrimitivesExample.wasm:0x94a0e
$main @ 3DPrimitivesExample.wasm:0x21afe
(anonymous) @ 3DPrimitivesExample.js:formatted:975
callMain @ 3DPrimitivesExample.js:formatted:10163
doRun @ 3DPrimitivesExample.js:formatted:10208
(anonymous) @ 3DPrimitivesExample.js:formatted:10217
setTimeout (async)
run @ 3DPrimitivesExample.js:formatted:10213
runCaller @ 3DPrimitivesExample.js:formatted:10151
removeRunDependency @ 3DPrimitivesExample.js:formatted:941
processPackageData @ 3DPrimitivesExample.js:formatted:308
getRequest.onsuccess @ 3DPrimitivesExample.js:formatted:269

oh, cool! good you found that, there is a similar error with quite some other examples

@ofTheo
Copy link
Member

ofTheo commented Nov 22, 2022

which main console do you mean?

The sort of fake terminal that OF outputs messages to below the renderer.
Not sure if that is possible, but making the js errors more visible could help track down the bugs ( maybe even with a toggle which is default off ).

@themancalledjakob
Copy link
Contributor

hmmm not having tested it, but it should be possible to override console.error as seen here: https://stackoverflow.com/questions/69460871/how-to-catch-console-error-so-title

console.errorOriginal = console.error;
console.error = (...messages) => {
    console.errorOriginal(...messages); 
    // add messages to oF terminal
}

could that work?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants