setup.cfg vs setup.py

[dstufft]

What should you do in setup.cfg vs what should you do in setup.py? This is made even more difficult to figure out by the distutils2 examples floating out there and projects like d2to1.

[hickford]

Why are two files (in different formats) necessary at all? Nodejs has just package.json

[ncoghlan]

Mostly backwards compatibility - setup.py has been around since 1998,
setup.cfg is a newer approach based on static metadata.

[dstufft]

Strictly speaking setup.cfg has been around for as long as setup.py, and it was designed as a way to override build options for stuff. Then distutils2 came along and decided to shove everything in it, then distutils2 failed and now it's kind of half way between that and it's old role depending on if you use something like d2to1 or pbr.

[hickford]

So it's fallout from the turbulent history of packaging middlewares. I understand how these things happen, but it is unfortunate; one would never design a system like this.

That's why I'm enamoured with the Nodejs package manager npm--it's beautifully coherent. Publishing a package is so easy it makes me laugh. https://www.npmjs.org/doc/misc/npm-developers.html

---

ps. Donald, I really like what you're doing here--I feel the discussion in the Python community has been too limited to the technical merits of different packaging libraries, to the neglect of user experience. Of course it was necessary to improve the tools, but that's not the be all and end all. We should aspire to bring the user experience to par with Ruby and Nodejs.

[dstufft]

Well it was designed for both setup.py and setup.cfg to exist side by side, but setup.cfg was mostly an override file. But yea, it's crazy how hard it is to do this stuff, and it'll get better. Until it does we have to put up with some bad (sometimes more bad than before while things get shaken up and settled back down).

[daenney]

How de we account for other tools that have started to shove configuration into setup.cfg, like flake8 does?

[dstufft]

Not sure what you mean by account for, people will be able to include a setup.cfg, even if the tooling no longer uses it.

[daenney]

That's what I meant yes.

[brainwane]

I'm going to move into this discussion this question pypa/twine#258 (comment) from @lrntct :

I just ran into the issue. I had the URL in setup.cfg instead of setup.py. Is it an expected behaviour to ignore the URL when defined in setup.cfg? Maybe related: pypa/setuptools#862

[brainwane]

https://packaging.python.org/tutorials/distributing-packages/#initial-files has some guidance but could have more.

[brainwane]

I would also welcome clarification on how pyproject.toml and PEP 518 figure into this.

[njsmith]

Historically, setup.cfg is "some file that distutils would parse using some code", so the behavior was pretty unspecified. I think in practice it uses the Python stdlib's configparser package, which is a limited and quirky format (so e.g., you can't necessarily parse it from other languages and get the same result, or put anything more interesting than some strings in there), and it ignores unrecognized sections, so other tools like flake8 and tox started optionally sticking their configuration in there too.

pyproject.toml is the new thing, and it has two major differences: (1) it uses a more powerful and better documented language (TOML), (2) it's defined by a general Python-level standard instead of being something specific to setup.py/distutils – and in fact one of its major purposes is to be something pip can look at before invoking setup.py (as in PEP 518), or can look at and then decide not to invoke setup.py at all (PEP 517). (In fact, part of the motivation here is to give us an escape hatch from the decade+ of messy stuff that has accumulated around distutils.) And since it's going to be mandatory for lots of python projects and has a nice format, PEP 518 also provides a standard way for tools to start sticking their configuration in there, if they want to. I'm not aware of any tools that are doing this yet besides towncrier, probably more will start once pip 10 ships and pyproject.toml starts being a real thing you can use instead of just a cool idea in some standards docs.

[Ivoz]

A major problem is that setup.cfg has in the past been used to configure how to build a project, from both a user and maintainer perspective, but more recently in history has been cooped by other tools to be used as a way to configure / describe a project purely from a maintainer perspective (e.g pbr). So it has been pulled in two directions, which once every so often collide and create confusion for new users / packagers. A single file being used for multiple purposes, for different things, with documentation coming from different places... is not a great situation to be in.

Hence I think it should be moved away from in general whereever possible, in favour of other solutions without double-uses, like the new toml standard.

[ncoghlan]

Yeah, that's exactly the direction we're going. While this confusion will likely always exist for setuptools/distutils users for historical reasons, for future users we should reach a point where they only need pyproject.toml plus their preferred packaging tool's input format (e.g. "flit.ini" for folks using flit)