Support include/exclude also for wheels

[pombredanne]

Hi @takluyver :
Flit is unique ❤️ because it can build multiple packages from the same tree. Just thrown in another pyproject toml file and you can point filt to it with -f/--ini-file; and you can then build multiple packages easily including small ones, without much ceremony.

It helps when you have larger repos to create smaller reusable libraries and it avoid copy/paste because it is so easy to create and publish a new package.

No other packaging tools does this so far. In this context, I would like to select which file goes in a wheel the same way I can select which files goes in an sdist. I reckon that you may not accept this as a feature based on past issues and your comments on these such as in #636 ... but I am asking just in case you may have changed your mind:

Would you consider a patch that does just this: support a new include/exclude glob in pyproject.toml to build wheels with a bit more control, without having to make major artificial reorganization of the code being built.

One alternative would be to create my own package as a wrapper and using either subclass or monkey patching, but I am not sure you have any minimally API of sorts. A last resort would be a fork.

Thank you for your kind consideration!

[takluyver]

Thanks for the idea!

To be honest, the support for pointing to a different file is a random leftover from the early days of Flit - that's why it's --ini-file even though we no longer use INI. Python packaging has made pyproject.toml a special name since 2016, and there's no obvious way to have >1 of these in a directory, or to describe >1 package in a single pyproject.toml file. You can tell Flit to use a different TOML file by invoking it directly, but tools like pip or build that call Flit as a backend have no way to do this.

If you want several packages made from the same repo, nowadays I'd do this with subdirectories, each one containing its own pyproject.toml file and its own set of source files. That may make some other tasks more cumbersome, but I think that's going to be easier to deal with than trying to create separate packages without that structure.

Maybe there's an opportunity for a tool that lets you carve up a repo into several packages without structuring it as a set of smaller projects in one repo. Feel free to experiment - and to use Flit as a base for experiments - if you want that. But it sounds pretty niche, and at the moment, I don't see it as a likely direction for Flit itself (as in the description "a simple packaging tool for simple packages").

One note if you do want to do something based on Flit: the only part I consider 'public API' is the PEP 517 & PEP 660 build backend interface in flit_core.buildapi. The rest can change with no warning, even if it doesn't change very often.

[pombredanne]

@takluyver Thanks Thomas!
As the saying goes, any observable internal feature and quirk may eventually be relied upon as if it were an API ;) https://www.hyrumslaw.com/ ... and here I shall say that the possibly to use alternative path names for pyproject.toml files comes really handy and is unique among all the packaging tools. My use case is to allow small libraries to be built to foster reuse from another larger repo like https://github.com/nexB/scancode-toolkit that I do not want to break apart further. I guess I am warming up to monorepo-like layouts somehow.

I will try to hack a proof of concept derived from flit and will report back here.

[pombredanne]

@takluyver here is a hack derived from flit, that I called flot!
https://github.com/pombredanne/flot/ and https://pypi.org/project/flot/
And https://github.com/pombredanne/flot/blob/main/doc/rationale.rst for my take on a rationale.

Why use flot?

flot enable the simplest way to build a simple Pythom package, and leaves
everything else, like installation, publishing, and other hard things to other
tools.

Specifically, the easy things are pure Python packages assembled simply from
a bunch of file paths, with no extra build steps. The vast majority of packages
on PyPI are like this and flot makes it dead easy to build these packages.

What, specifically, does flot make easy?

• Build wheels and sdists, nothing else. No command bloat.

• Use only pyproject.toml. No legacy menagerie of setup.cfg, ini, setup.py or
  other manifest.file. There is only one way to do things with flot.

• No magic in files, packages or modules selection. Existing tools are trying to
  be smart about which files to include in a package using magic discovery and
  guesses; this is byzantine and hard to understand.

• Explicit files inclusion is better than implicit. flot assumes little to
  nothing. Just provide a list of paths to include in your package. Files are
  not automatically included or excluded.

• Build easily many packages from one code tree. Just create as many
  pyproject TOML files as you have packages, name them the way you want and you
  can build all of these without fuss.

[pombredanne]

At this stage I started using flot in production, and I am pretty happy with its simplicity and feature set. @takluyver Thank you ++ for flit!

[takluyver]

Thanks!