use latest Jupyterlite with programatic options

[stevejpurves]

The purpose of this PR is to update to using the latest juptyerlite codebase, this involves adjusting to the new organisation of the @juptyerlite/pyodide-kernel and taking steps to specify the location of the wheels to load. This should be a big improvement over the previous implementation once in place, and the objective is to be able to load the latest wheels from CDN (possibly direct from unpkg.com).

[stevejpurves]

we should be able to set the litePluginSettings here? and this would be ahead of the @jupyterlite/pyodide-kernel-extension being activated, so it should be able to pick up on these?

[jtpio]

so it should be able to pick up on these?

Yes I would assume so.

[stevejpurves]

cool - and yes I am seeing some impact of setting this but it's not loading the piplite wheel from unpkg for some reason.

i.e. with setting the litePluginSettings here, on initializing the kernel two of the wheels are loaded from unpkg.com 👏 but it still looks to the /pypi/all.json locally and still loads piplite from there 🤔

[stevejpurves]

ok so this kind of makes sense as litePluginSettings is setting the pipliteUrls i.e. where piplite will load it's wheels from -- we need to also configure something separate to direct where the piplite wheel itself will be loaded from...?

[stevejpurves]

opened jupyterlite/pyodide-kernel#42 to address this

[stevejpurves]

I have also started looking at the remote federated module loading, which I think I'll start in a separate PR on top of this one.

I'm just wondering though, if we put that remote loading in place it will immediately supercede this arrangement of build-time including the server extension while picking up the wheels from online - so it's just better to move to that now. Then AFAIK this will mean that the kernel can be specified by option at runtime, and also be interchangeable (i.e. we can load other xeus kernels too)

[stevejpurves]

currently when trying to run jlite.html in the simple demo, we hit this problem during kernel initialization:

I was expecting that setting the litePluginOptions appropriately would address this but we're not seeing even a network call being made to the all.json.

cc @jtpio does this seem like a reasonable approach?

[github-actions]

PR Preview Action 8eaa763
Preview removed because the pull request was closed.
2023-04-10 20:55 UTC

[jtpio]

The screenshot suggests the name is malformed, but they seem to look good in the npm package: https://www.npmjs.com/package/@jupyterlite/pyodide-kernel?activeTab=code

If you use the @jupyterlite/pyodide-kernel directly then it should be possible to import them from /pypi yes.

Maybe we could also check the Web Worker is correctly getting the wheel info.

[stevejpurves]

@jtpio yes the wheels look good in the package - what I am seeing though is webpack is adding them to my distribution and replacing the names with hashes, so my dist folder contains randomly hashed .whl files.

[jtpio]

so my dist folder contains randomly hashed .whl files.

You might need to use the Asset Modules with a generator like here: https://github.com/jupyterlite/pyodide-kernel/blob/01ef2f5ad3c468205d78dddeffa2990c0bd4eee9/packages/pyodide-kernel-extension/webpack.config.js#L8-L12

[stevejpurves]

ok - so status after that last commit:

• build working and it's simpler than before!
  • service worker gets installed automatically 👏💪
  • I no longer have to change the location of the wheels output by webpack, pypi folder appears in the right place in the dist folder
• I'm specifying lite plugin options in the simple demo, but that is only picking up 2 of the 3 wheels from unpkg (see comment above)

[stevejpurves]

ok I have changed the pageconfig to set the litePluginOptions based on an optional argument in the start server function. As it is this would allow runtime changes to point piplite to look for wheels at a remote location. This means that the iolie wheel itself still needs to be hosted locally but if jupyterlite/pyodide-kernel#42 is merged and released then all wheels can hosted remotely.

[stevejpurves]

merged #612 which addresses a lot of this but we still need some testing to see if that last todo on startup signalling has been achieved.