Use .zip files instead of .data files for loading WordPress #978
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Playground used the Emscripten's filepacker.py tool to bundle WordPress releases as zip files. This caused a lot of problems: * Filepacker splits WordPress into a `.data` file, with all the bytes concatenated as a single blob, and a `.js` file with the map of offsets and lengths for each file. Sometimes the browsers would invalidate the cache for the `.data` file, but not the `.js` file, causing the playground to fail to load. * The `.js` files had to be patched in the Dockerfile. * The `.js` files used XHR to load the `.data` files, which required custom error handling logic, progress monitoring logic, and even then some of the errors were not handled properly – see #504. * The `.data` loading code required custom loading logic. * The `.data` files were not stream–loaded. This PR doesn't stream-load the `.zip` files either, but that feature may be added in the future using the new `@wp-playground/compression-streams` package. ## Changes * The WordPress build process now produces `.zip` files instead of `.data` + `.js` files. * The worker thread now loads WordPress from zip files, including zips from arbitrary URLs. * Query API and Blueprints now accept URLs via the `?wp=` parameter. ## Testing instructions * Confirm all the E2E tests pass
adamziel
added
[Type] Enhancement
Open
|
The unit test failure seems intermittent. the last step here would be changing the zipFile step option to a Resource instead of adding a new zipBlob option. |
|
this is wonderful! way to go, using common infrastructure to remove a significant chunk of bulk. |
bgrgicak
reviewed
Jan 30, 2024
| } | ||
| } | ||
|
|
||
| // const wordPressFile = fetch(zipFilename); |
| // Start downloading WordPress if needed | ||
| let wordPressRequest = null; | ||
| if (!wordPressAvailableInOPFS) { | ||
| if (requestedWPVersion.startsWith('http')) { |
There was a problem hiding this comment.
If I'm reading this right, we now have support for custom WordPress builds?
Having the ability to prebuild WordPress with additional plugins and themes can be useful for demos like the Woo.com example we explored.
There was a problem hiding this comment.
You are reading this right indeed! :-)
This was referenced Jan 31, 2024
This was referenced Feb 29, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Playground used the Emscripten's filepacker.py tool to bundle WordPress releases as zip files. This caused a lot of problems:
.datafile, with all the bytes concatenated as a single blob, and a.jsfile with the map of offsets and lengths for each file. Sometimes the browsers would invalidate the cache for the.datafile, but not the.jsfile, causing the playground to fail to load..jsfiles had to be patched in the Dockerfile..jsfiles used XHR to load the.datafiles, which required custom error handling logic, progress monitoring logic, and even then some of the errors were not handled properly – see Playground not loading on Android and Chrome #564..dataloading code required custom loading logic..datafiles were not stream–loaded. This PR doesn't stream-load the.zipfiles either, but that feature may be added in the future using the new@wp-playground/compression-streamspackage.Changes
.zipfiles instead of.data.jsfiles.?wp=parameter.Testing instructions
cc @bgrgicak