Transition to using `flit` #340

pradyunsg · 2020-10-14T20:40:11Z

should we choose to something that uses pyproject.toml like flit?

Originally posted by @brettcannon in #332 (comment)

Yes. :)

The text was updated successfully, but these errors were encountered:

brettcannon · 2020-10-15T17:58:18Z

+1 from me 😄 . (I'm also willing to do the work or ~~con~~/ask someone else like @derek-keeler who has been looking to contribute to do this 😁 ).

pradyunsg · 2020-10-16T00:50:12Z

Oh 100%. PRs very welcome!

Closes pypa#340

* Move from setuptools to flit Closes #340 * Add URLs to the documentation and changelog * Move classifiers above dependency resolution fields * Update copyright for Furio theme

brettcannon · 2021-11-10T23:00:43Z

I'm reopening this as I believe we said we would give folks a year to get build to a place that they could rely on it and I believe that's occurred.

vishwin · 2022-04-10T23:48:18Z

It's probably high time to move back to flit_core, especially since it can bootstrap itself.

henryiii · 2022-05-11T20:59:55Z

Does anyone know how to include multiple license files (other than draft PEP 639)? I made a before and after, and got:

$ unzip -l dist-new/packaging-21.4.dev0-py3-none-any.whl
Archive:  dist/packaging-21.4.dev0-py3-none-any.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
      666  12-20-2021 20:38   packaging/__about__.py
      497  05-19-2021 18:43   packaging/__init__.py
    11488  10-29-2021 15:51   packaging/_manylinux.py
     4378  10-29-2021 15:51   packaging/_musllinux.py
     1431  10-29-2021 15:51   packaging/_structures.py
     9460  05-11-2022 20:37   packaging/markers.py
        0  01-26-2021 19:35   packaging/py.typed
     5123  05-11-2022 20:37   packaging/requirements.py
    30397  05-11-2022 20:37   packaging/specifiers.py
    15697  05-11-2022 20:37   packaging/tags.py
     4355  05-11-2022 20:37   packaging/utils.py
    14665  05-19-2021 18:43   packaging/version.py
      197  05-11-2022 20:56   packaging-21.4.dev0.dist-info/LICENSE
    10174  05-11-2022 20:56   packaging-21.4.dev0.dist-info/LICENSE.APACHE
     1344  05-11-2022 20:56   packaging-21.4.dev0.dist-info/LICENSE.BSD
    15133  05-11-2022 20:56   packaging-21.4.dev0.dist-info/METADATA
       92  05-11-2022 20:56   packaging-21.4.dev0.dist-info/WHEEL
       10  05-11-2022 20:56   packaging-21.4.dev0.dist-info/top_level.txt
     1553  05-11-2022 20:56   packaging-21.4.dev0.dist-info/RECORD
---------                     -------
   126660                     19 files
$ unzip -l dist-new/packaging-21.4.dev0-py3-none-any.whl
Archive:  dist-new/packaging-21.4.dev0-py3-none-any.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
      666  12-20-2021 15:38   packaging/__about__.py
      497  05-19-2021 14:43   packaging/__init__.py
    11488  10-29-2021 11:51   packaging/_manylinux.py
     4378  10-29-2021 11:51   packaging/_musllinux.py
     1431  10-29-2021 11:51   packaging/_structures.py
     9460  05-11-2022 16:37   packaging/markers.py
        0  01-26-2021 14:35   packaging/py.typed
     5123  05-11-2022 16:37   packaging/requirements.py
    30397  05-11-2022 16:37   packaging/specifiers.py
    15697  05-11-2022 16:37   packaging/tags.py
     4355  05-11-2022 16:37   packaging/utils.py
    14665  05-19-2021 14:43   packaging/version.py
      197  01-26-2021 14:35   packaging-21.4.dev0.dist-info/LICENSE
       81  01-01-2016 00:00   packaging-21.4.dev0.dist-info/WHEEL
     2993  01-01-2016 00:00   packaging-21.4.dev0.dist-info/METADATA
     1254  01-01-2016 00:00   packaging-21.4.dev0.dist-info/RECORD
---------                     -------
   102682                     16 files

The other two license files are missing (as is top_level.txt, but I think that one is okay).

brettcannon · 2022-05-11T22:43:47Z

Does anyone know how to include multiple license files (other than draft PEP 639)?

The best I can think of is https://flit.pypa.io/en/latest/pyproject_toml.html#sdist-section .

pradyunsg · 2022-05-11T23:33:42Z

Flit should include files that match the LICENSE* glob, in the repository root, into the dist-info for the distribution: https://github.com/pypa/flit/blob/main/flit_core/flit_core/wheel.py#L185

henryiii · 2022-05-12T03:17:10Z

Ahah! If I go SDist -> wheel, then I only get the single file specified in the license field. If I go directly to wheel, then I get all the license files as shown above. IMO Flit should not be forgetting to put LICENSE* into the SDist.

henryiii · 2022-05-12T03:32:28Z

Is there a reason docs are included in the SDist? I usually don't put docs in SDists. Playing with the include/exclude list currently.

merwok · 2022-05-12T09:25:44Z

See recent long thread about including docs and tests in sdists so that downstream distributors can use them as sources!

dstufft · 2022-07-15T19:55:13Z

Idle question, I don't much care what backend packaging uses, it's simple enough that any of them will do, but setuptools itself supports pyproject.toml now.

Do we gain anything from using flit versus just using setuptools + pyproject.toml? Will downstream distributors have an easier time dealing with this library if it still uses setuptools, but drops the legacy mechanisms for specifying metadata?

henryiii · 2022-07-15T20:53:02Z

The goal is to fix the bootstrapping issue. Flit has no dependencies and recently (just before this PR) can be bootstrapped. Setuptools shouldn’t depend on things that depend on setuptools.

dstufft · 2022-07-15T21:12:07Z

Setuptools vendors packaging IIRC, so it doesn't depend on anything anyways, right?

Which is the same thing that flit_core does, as it vendors tomli (and I think it also vendors pieces of packaging, just not the whole packaging library, which seems wholly worse to me than just vendoring since it makes maintenance into the future harder).

dstufft · 2022-07-15T21:20:31Z

To be clear, It's fine to me if packaging moves to flit, I have no conceptual problem with it.

I just suspect it's going to break more people than any bootstrapping problems it fixes. Any place where setuptools is broken because of circular dependencies, flit_core should be as well unless they've allowed flit_core to keep its vendored items but not setuptools.

pradyunsg · 2022-07-15T21:34:59Z

IIUC, flit has a better bootstrapping story than setuptools, for people building up the ecosystem from scratch -- some of that might be due to this documentation link: https://flit.pypa.io/en/latest/bootstrap.html

layday · 2022-07-15T21:56:13Z

The idea is to move all foundational dependencies to the same (dependency-less) backend to make PEP 517 bootstrapping easier. We are not worried about people debundling setuptools or flit_core. There's a number of reasons that we gravitated towards flit but the main one is that the wheel package is distributed separately and it has a circural dependency on setuptools.

dstufft · 2022-07-15T22:01:30Z

If that's a desired end goal, that all "foundational" dependencies are on flit, that seems like something that would be useful to be documented/recommended somewhere FWIW.

layday · 2022-07-15T23:21:21Z

I mean, this is all being done kind of haphazardly, but my impression is that we think sticking to one backend for bootstrapping is a good thing. pep517 (the package) and installer are already on flit IIRC. There's also an open PR for build to move to flit.

henryiii · 2022-07-15T23:32:02Z

I think this was discussed informally around when flit joined the PyPA. It’s a closed set of libraries, and not something we need to communicate as a “good practice” to others, which is why makes it tricky to formalize. There’s nothing wrong with dependencies, just with circular dependencies in the SDist build and install system. And it’s nice to only require a small set of lightweight libraries to bootstrap a functional environment from source.

abravalheri · 2022-07-15T23:57:49Z

the wheel package is distributed separately and it has a circural dependency on setuptools.

If you guys want, you can also create a PR vendoring wheel in setuptools.

vishwin · 2022-07-16T00:03:48Z

Circular dependencies are completely unacceptable for those of us who maintain operating system-level ports/package repositories. This is why, at least for us in FreeBSD, we cannot just use {,ensure}pip et al to implement PEP-517 into our framework.

nanonyme · 2022-07-30T18:32:03Z

There will be no circular dependencies. I summarized one way to do it in our meta distros issue tracker:

flit_core: flit_core needs to be self-bootstrapped into wheel. it provides a way to unpack itself
pypa/installer: Directly invoke python3 -mflit_core.wheel. Set env PYTHONPATH=src and install with python3 -minstall dist/*.whl
pyparsing: Directly invoke python3 -mflit_core.wheel, install with python3 -minstall dist/*.whl
packaging (needs port to flit_core): Directly invoke python3 -mflit_core.wheel, install with python3 -minstall dist/*.whl
tomli: Directly invoke python3 -mflit_core.wheel, install with python3 -minstall dist/*.whl
pep517: Directly invoke python3 -mflit_core.wheel, install with python3 -minstall dist/*.whl
pypa/build (needs port to flit_core): Directly invoke python3 -mflit_core.wheel, install with python3 -minstall dist/*.whl

After this you have functional pypa/build and pypa/installer. This is minimal working set of tooling for building wheels and installing them. IMHO the sane way to bootstrap setuptools and wheel is to stage the sources together (IMHO these two packages should be merged for good but that's anoher matter) into same build sandbox and use PYTHONPATH to ensure wheel is importable when building wheels for setuptools and wheel using pypa/build and installing them with pypa/installer. Once you have these, you can continue building pip with pypa/build and installing the wheel with pypa/installer.

nanonyme · 2022-07-30T18:43:51Z

@dstufft note there is plan for flit_core to eventually unvendor tomli now that tomllib entered standard library.

pradyunsg mentioned this issue Oct 14, 2020

Metadata validation API #332

Closed

brettcannon added the enhancement label Oct 15, 2020

brettcannon added a commit to brettcannon/packaging that referenced this issue Nov 14, 2020

Move from setuptools to flit

83a51d1

Closes pypa#340

brettcannon mentioned this issue Nov 14, 2020

Move from setuptools to flit #352

Merged

brettcannon closed this as completed in #352 Nov 20, 2020

brettcannon added a commit that referenced this issue Nov 20, 2020

Move from setuptools to flit (#352)

b32f744

* Move from setuptools to flit Closes #340 * Add URLs to the documentation and changelog * Move classifiers above dependency resolution fields * Update copyright for Furio theme

brettcannon mentioned this issue Nov 30, 2020

Reliance on flit makes packaging incompatible with Void, Arch and Alpine Linux package systems #363

Closed

brettcannon reopened this Nov 10, 2021

This was referenced Apr 10, 2022

Switch to flit pypa/build#394

Closed

Revert "move back to setuptools (#367)" #536

Closed

jameshilliard mentioned this issue May 11, 2022

Switched build backend to flit pypa/wheel#436

Closed

henryiii mentioned this issue May 12, 2022

LICENSE* files not included in the SDist (via standard tooling) pypa/flit#549

Closed

henryiii mentioned this issue May 12, 2022

refactor: use flit as a backend #546

Merged

brettcannon closed this as completed in #546 Oct 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Transition to using `flit` #340

Transition to using `flit` #340

pradyunsg commented Oct 14, 2020 •

edited

Loading

brettcannon commented Oct 15, 2020

pradyunsg commented Oct 16, 2020

brettcannon commented Nov 10, 2021

vishwin commented Apr 10, 2022

henryiii commented May 11, 2022 •

edited

Loading

brettcannon commented May 11, 2022

pradyunsg commented May 11, 2022 •

edited

Loading

henryiii commented May 12, 2022

henryiii commented May 12, 2022

merwok commented May 12, 2022

dstufft commented Jul 15, 2022

henryiii commented Jul 15, 2022

dstufft commented Jul 15, 2022

dstufft commented Jul 15, 2022 •

edited

Loading

pradyunsg commented Jul 15, 2022 •

edited

Loading

layday commented Jul 15, 2022

dstufft commented Jul 15, 2022

layday commented Jul 15, 2022

henryiii commented Jul 15, 2022

abravalheri commented Jul 15, 2022 •

edited

Loading

vishwin commented Jul 16, 2022

nanonyme commented Jul 30, 2022 •

edited

Loading

nanonyme commented Jul 30, 2022

Transition to using flit #340

Transition to using flit #340

Comments

pradyunsg commented Oct 14, 2020 • edited Loading

brettcannon commented Oct 15, 2020

pradyunsg commented Oct 16, 2020

brettcannon commented Nov 10, 2021

vishwin commented Apr 10, 2022

henryiii commented May 11, 2022 • edited Loading

brettcannon commented May 11, 2022

pradyunsg commented May 11, 2022 • edited Loading

henryiii commented May 12, 2022

henryiii commented May 12, 2022

merwok commented May 12, 2022

dstufft commented Jul 15, 2022

henryiii commented Jul 15, 2022

dstufft commented Jul 15, 2022

dstufft commented Jul 15, 2022 • edited Loading

pradyunsg commented Jul 15, 2022 • edited Loading

layday commented Jul 15, 2022

dstufft commented Jul 15, 2022

layday commented Jul 15, 2022

henryiii commented Jul 15, 2022

abravalheri commented Jul 15, 2022 • edited Loading

vishwin commented Jul 16, 2022

nanonyme commented Jul 30, 2022 • edited Loading

nanonyme commented Jul 30, 2022

Transition to using `flit` #340

Transition to using `flit` #340

pradyunsg commented Oct 14, 2020 •

edited

Loading

henryiii commented May 11, 2022 •

edited

Loading

pradyunsg commented May 11, 2022 •

edited

Loading

dstufft commented Jul 15, 2022 •

edited

Loading

pradyunsg commented Jul 15, 2022 •

edited

Loading

abravalheri commented Jul 15, 2022 •

edited

Loading

nanonyme commented Jul 30, 2022 •

edited

Loading