Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Share side modules with worker threads via postMessage #19390

Merged
merged 1 commit into from
May 26, 2023
Merged

Conversation

sbc100
Copy link
Collaborator

@sbc100 sbc100 commented May 18, 2023

Any side modules that are loaded at the time of worker creation are
shared with the worker via the initial postMessage.

As a followup we should extend this to modules that are loaded after
the worker is created but before the pthread runs (for example when
a module is loaded while a worker is unused).

Fixes: #18552

@sbc100 sbc100 force-pushed the shared_modules branch 3 times, most recently from e9d53e4 to 336daa3 Compare May 23, 2023 23:38
sbc100 added a commit that referenced this pull request May 23, 2023
@sbc100 sbc100 force-pushed the shared_modules branch 3 times, most recently from 5bcc37b to 4883bf6 Compare May 24, 2023 00:33
sbc100 added a commit that referenced this pull request May 24, 2023
@sbc100 sbc100 changed the base branch from main to dylink_debug May 24, 2023 00:35
sbc100 added a commit that referenced this pull request May 24, 2023
sbc100 added a commit that referenced this pull request May 24, 2023
Base automatically changed from dylink_debug to main May 24, 2023 15:04
sbc100 added a commit that referenced this pull request May 24, 2023
@sbc100 sbc100 force-pushed the shared_modules branch 2 times, most recently from 35af5a7 to e045661 Compare May 24, 2023 19:13
sbc100 added a commit that referenced this pull request May 24, 2023
Also move the function in question so it is declared alongside its
users (its was previously in a different `#if` block).

Split out from #19390
sbc100 added a commit that referenced this pull request May 24, 2023
Also move the function in question so it is declared alongside its
users (its was previously in a different `#if` block).

Split out from #19390
sbc100 added a commit that referenced this pull request May 24, 2023
Also move the function in question so it is declared alongside its
users (its was previously in a different `#if` block).

Split out from #19390
@sbc100 sbc100 requested a review from kripken May 24, 2023 21:26
@sbc100
Copy link
Collaborator Author

sbc100 commented May 24, 2023

I've already split several PRs out from this one.. and I could split a couple more perhaps. But PTAL and see what you think?

@sbc100 sbc100 requested a review from dschuff May 24, 2023 21:26
src/library_dylink.js Outdated Show resolved Hide resolved
src/library_dylink.js Outdated Show resolved Hide resolved
emcc.py Outdated Show resolved Hide resolved
src/library_pthread.js Outdated Show resolved Hide resolved
src/postamble.js Outdated Show resolved Hide resolved
src/postamble.js Outdated Show resolved Hide resolved
src/postamble.js Outdated Show resolved Hide resolved
test/test_core.py Show resolved Hide resolved
sbc100 added a commit that referenced this pull request May 24, 2023
sbc100 added a commit that referenced this pull request May 25, 2023
@sbc100 sbc100 force-pushed the shared_modules branch 2 times, most recently from 68c4cc8 to f5f159d Compare May 25, 2023 17:43
sbc100 added a commit that referenced this pull request May 25, 2023
Previously we were calling `loadDylibs` in the `run` function but
we can call it earlier as soon as the main module is instantiated.  This
saves a little on code size since it relies on the existing
`runDependencies` mechanism.

Split out from #19390
sbc100 added a commit that referenced this pull request May 25, 2023
Previously we were calling `loadDylibs` in the `run` function but
we can call it earlier as soon as the main module is instantiated.  This
saves a little on code size since it relies on the existing
`runDependencies` mechanism.

Split out from #19390
sbc100 added a commit that referenced this pull request May 25, 2023
Previously we were calling `loadDylibs` in the `run` function but
we can call it earlier as soon as the main module is instantiated.  This
saves a little on code size since it relies on the existing
`runDependencies` mechanism.

Split out from #19390
sbc100 added a commit that referenced this pull request May 25, 2023
Previously we were calling `loadDylibs` in the `run` function but
we can call it earlier as soon as the main module is instantiated.  This
saves a little on code size since it relies on the existing
`runDependencies` mechanism.

Split out from #19390
@sbc100 sbc100 changed the base branch from main to move_loadDylibs May 25, 2023 20:15
@sbc100
Copy link
Collaborator Author

sbc100 commented May 25, 2023

Rebased on top of move_loadDylibs

Copy link
Member

@kripken kripken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much easier to understand now, thanks!

lgtm % question

emcc.py Outdated Show resolved Hide resolved
Base automatically changed from move_loadDylibs to main May 25, 2023 22:37
sbc100 added a commit that referenced this pull request May 25, 2023
Previously we were calling `loadDylibs` in the `run` function but
we can call it earlier as soon as the main module is instantiated.  This
saves a little on code size since it relies on the existing
`runDependencies` mechanism.

Split out from #19390
@sbc100 sbc100 force-pushed the shared_modules branch 2 times, most recently from b0fefa4 to 762624b Compare May 25, 2023 22:49
@sbc100 sbc100 enabled auto-merge (squash) May 25, 2023 23:13
Any side modules that are loaded at the time of worker creation are
shared with the worker via the initial postMessage.

As a followup we should extend this to modules that are loaded after
the worker is created but before the pthread runs (for example when
a module is loaded while a worker is unused).

Fixes: #18552
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New threads should receive DSO module via postMessage
2 participants