How would you feel about pytoml becoming a dependency of pip?

[njsmith]

Hello!

There's a discussion happening on distutils-sig about choosing a new standard configuration file format for Python source trees, to be read by tools like pip when they want to build a package. There are multiple threads, but maybe this one is the most relevant.

Currently sentiment seems to be shifting towards TOML, and attention is focused on pytoml because in some initial testing we found that pytoml seems to handle unicode correctly on py2 and the other toml parser we could find doesn't.

So... I wanted to reach out to you and see what you thought about this. I'm super excited about the possibility of having something like TOML become a standard part of our package building toolkit. But... this would also mean a sudden huge influx of users and attention on your little project here, which I know can be a very mixed blessing. And there'd be issues that come up, like, pip needs support for python 2.6 and 3.3 -- I don't know if you're actually interested in supporting those going forward? (I actually have a patch for this that I'll submit as a PR in a moment, but there are also ongoing costs to keeping old python versions working and a PR doesn't solve that.) And we'd probably need to fix the error message situation. And so forth.

What do you think?

[avakar]

Hi,

thank you for letting me know what's happening! Unfortunately, I don't think I'll make you happy with my answer.

TOML is a bad file format. It looks good at first glance, and for really really trivial things it is probably good. But once I started using it and the configuration schema became more complex, I found the syntax ugly and hard to read.

I personally abandoned TOML and as such, I'm not planning on adding any new features (there are in fact issues open due to changes in the datetime format, which btw TOML people made without bumping the specs version number, so there are probably incompatible implementations of 0.4.0 out there. In any case, I'm not planning on working on those issues).

That being said, I do accept merge requests and I do fix bugs (that are not due to specs change). The support for 2.6 and 3.3 is trivial and I don't foresee it as a maintenance problem. I'm willing to fix the error message issue if you decide to adopt TOML (please don't). But that's pretty much all you can expect of me. I'd be happy to release the toml name on pypi if someone else wants to fork and continue this project.

If you're still reading, I'd like to put forth CSON as a format of choice. I happen to have a Python implementation too: https://github.com/avakar/pycson. It's missing in your comparison but I think it scores "yes" in all rows of your summary table (except maybe "well-defined", but that's a stretch for TOML too). It's quite popular in the Javascript world so there is some tooling for it available.

[njsmith]

Hi,

Thanks for the thoughtful reply!

But once I started using it and the configuration schema became more complex, I found the syntax ugly and hard to read

For context, can you say a bit more about what kinds of things you were using it for? One thing I've realized through these discussions is that people's use cases for configuration files vary quite a bit (much more than I'd expected), and the same format might be great for one user and terrible for another even though they both want "a configuration file".

For this packaging thing, if we don't go with TOML then we'll probably end up with something like ConfigParser/INI, not CSON. So that might help calibrate the complexity of the configuration schema we have in mind... :-)

That being said, I do accept merge requests and I do fix bugs (that are not due to specs change). The support for 2.6 and 3.3 is trivial and I don't foresee it as a maintenance problem. I'm willing to fix the error message issue if you decide to adopt TOML (please don't). But that's pretty much all you can expect of me. I'd be happy to release the toml name on pypi if someone else wants to fork and continue this project.

Noted, and thanks for being clear.

[nchammas]

@njsmith - Just as a Python packaging end-user, I'm curious to know: Where do we stand with this issue? Will pip, after all, add a dependency on this package? Is someone from the PyPA going to fork this project and begin maintaining it?

[njsmith]

@nchammas: pip will indeed be gaining TOML-file reading capabilities -- the spec (at least for basic capabilities) is finalized, and I think @takluyver is working on implementing it (the progress tracking bug is: pypa/pip#3691). I suspect that for the first cut he'll use this package, and in the longer run we'll see if switching to a fork or something else makes sense.

(My hope is that once people start using pyproject.toml then it'll provide motivation for someone to improve TOML file handling in Python in general, whatever that looks like. The "PyPA" is just an ad hoc group of overworked volunteers – notice that PEP 518 was accepted 6 months ago and we still don't have an implementation! – so no promises there.)

[nchammas]

OK, sounds good. Thanks for the update and links.

[takluyver]

Yep, pypa/pip#4144 is my WIP implementation of PEP 518.