Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add flit backend #128

Closed
wants to merge 7 commits into from
Closed

Add flit backend #128

wants to merge 7 commits into from

Conversation

blink1073
Copy link
Contributor

@blink1073 blink1073 commented Feb 24, 2022

  • Add build backend based on flit
  • Add pre and post build editable config
  • Add wheel test with data files
  • Add editable test with data files
  • Add an alias for flit install --symlink
  • Add documentation

@blink1073
Copy link
Contributor Author

blink1073 commented Feb 24, 2022

Notes for docs:

When using the flit-based backend, you must specify the following fields in pyproject.toml:

[build-system]
requires = ["jupyter_packaging >=0.12"]
build-backend = "jupyter_packaging.build_flit"

[project]
name = "foo"
version = "0.1.0"
description = "foo package"

The full description of available fields can be found in PEP 621.
If converting from a setup.cfg file, you can use ini2toml

Data files can be specified as:

[tool.jupyter-packaging.external-data]
directory = "data"

Data files are copied at build time in editable mode, per PEP 660. We can support linking by providing an alias to flit install --symlink. # TODO: refer to flit docs here

You can use the same builder and args in editable mode, or use the following specific keys:

[tool.jupyter-packaging.editable-builder]
factory = "foo.main"

[tool.jupyter-packaging.editable-build-args]
fizz = "buzz"

@blink1073
Copy link
Contributor Author

blink1073 commented Mar 2, 2022

To summarize the status:

  • Packages that only ship static config files can use flit directly now that data_files support has landed. This includes kernels and server extensions.
  • Packages that have a build step with static assets, like Jupyter Server's JavaScript assets can use this new flit backend and be fully PEP-compliant.
  • Jupyter LabExtensions are trickier - currently jupyter labextension develop creates a symlink for a directory. This is incompatible with PEP 660 and flit install --symlink. Due to the limitations of PEP 660, pip install -e . results in copied data files. Using flit install --symlink (or an alias provided by jupyter_packaging) uses symlinks for the files in the data directory.

I think the best path forward is for jupyter labextension develop to be an alias for flit install --symlink, and to update the way the extension JavaScript files are built and loaded. The file names should be stable, but the server needs to know that they should not be cached. The extension in jupyterlab_server could be updated to not cache a file if the path is a symlink.

@blink1073
Copy link
Contributor Author

@bollwyvl
Copy link

bollwyvl commented Mar 2, 2022 via email

@fcollonval
Copy link
Contributor

Awesome thanks @blink1073

I think the best path forward is for jupyter labextension develop to be an alias for flit install --symlink, and to update the way the extension JavaScript files are built and loaded. The file names should be stable, but the server needs to know that they should not be cached. The extension in jupyterlab_server could be updated to not cache a file if the path is a symlink.

My concern here is that during development, it is quite common to change the list of source files and the dependencies. If we switch to symlinking at the file level, this will require jupyter labextension build --development and jupyter labextension watch to update the symlinks if the list of files changes, isn't it? And if going that way, this probably means we could keep the hash to avoid caching as we could always update the symlinks.

@blink1073
Copy link
Contributor Author

Yeah that's a fair way to do it. I'll finish this up as planned and leave it to y'all to handle the lab extension migration. Note that this is a new backend, so the cookiecutter would have to be changed to opt in to it.

@blink1073
Copy link
Contributor Author

Alternatively, we may be able to add external_data support to setuptools.

@blink1073
Copy link
Contributor Author

Closing this because it would be too awkward moving forward to support two build backend types in one repo. I think whichever way we move forward it should be with a new repo, cf jupyterlab/frontends-team-compass#146

@blink1073 blink1073 closed this May 18, 2022
@blink1073 blink1073 deleted the flit-backend branch May 18, 2022 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants