Proposal: Roadmap to Sphinx-2.0

[tk0miya]

Goal of 2.0

• Make APIs and configuration items stable and public.
  • Remove deprecated APIs and configuration variables so far and make our code simple
• Drop old environment supports
  • Python 2
  • Ubuntu 14.04 (EOL)
    • Start to use 16.04 as a standard version
  • docutils-0.12
  • TeXLive 2014 (or 2015?)
• Split unmaintained or less used builders and extensions to external package
  • For a while, Sphinx depends on them to keep compatiblity
  • It does not mean deprecation on 2.0. Deprecation will go in future version.
• Switch versioning rule to semantic versioning
• Use HTML5 builder by default

Schedule

• Sphinx-1.8: 2018/Autumn
• Sphinx-2.0: 2019/Spring

This schedule is not fixed. We can be adjusted depending on the situation.
I propose you not to release Sphinx-1.9.

About 1.8

• Same as so far, schedule based release. Basically, it will shipped 6months after 1.7.
• Enable DeprecationWarning for 2.0
• Start to split some builders and extensions to external package
• Refactoring

Note.

This does not mention what new feature is added. This doesn't mean not to add new feature. It only means we don't have scheduled feature any more. So please go ahead same as so far.

In addition, this is a proposal from me (not fixed!). So please let me know your idea and update this.

[shimizukawa]

Cool!

[stephenfin]

• Drop old environment supports
  • Ubuntu 14.04 (EOL)
    • Start to use 16.04 as a standard version

Surely 18.04? That will be released in a few months (2018-04).

• Split unmaintained or less used builders and extensions to external package
  • For a while, Sphinx depends on them to keep compatibility
  • It does not mean deprecation on 2.0. Deprecation will go in future version.

I'd be a little nervous about this one. I agree that some builders could/should definitely be moved, but extensions are a whole other game. We should definitely discuss this at length. It will be controversial, but I think the biggest thing we could and should split out is sphinx.ext.autodoc and related extensions/utilities.

---

The rest of this looks a-OK to me 👍 I would also like to slot in the work on a unified sphinx binary here, along with the sphinx-build simplification effort that I've been working on, which will overlap quite nicely into the single sphinx binary. We should discuss this idea in detail post 1.7 (in person would be easier, but I doubt you're ever at European conferences, @tk0miya?).

[tk0miya]

Thank you for comment!

Surely 18.04? That will be released in a few months (2018-04).

Basically, we use active Ubuntu LTS to determine what version of tooling we have to support (ex, TeXLive).
At 2019-04, Ubuntu 14.04 will go EOL. So I meant moving 16.04 for our base version of Ubuntu.

I'd be a little nervous about this one. I agree that some builders could/should definitely be moved, but extensions are a whole other game. We should definitely discuss this at length. It will be controversial, but I think the biggest thing we could and should split out is sphinx.ext.autodoc and related extensions/utilities.

Okay, let's discuss it. This is only my proposal, so it is not determined yet.

I would also like to slot in the work on a unified sphinx binary here, along with the sphinx-build simplification effort that I've been working on, which will overlap quite nicely into the single sphinx binary.

Sounds good. +1!!

We should discuss this idea in detail post 1.7 (in person would be easier, but I doubt you're ever at European conferences, @tk0miya?).

Sorry, I don't have plan for visiting for now.
In any case, I can't speak English not at all. I only can discuss with texts (chat is also. but it is very slow...).

[Suzumizaki]

Hello, I'm back from breaking about several years.

How do you think about perfect (not PARTIAL!) Japanese language support?
I made some extension against Sphinx 1.3, listed below URL:
http://h12u.com/sphinx/index.html

That includes:

• User defined sort order to list in the index pages
• Yomigana based sort order feature
• (HTML5 output, with filenames and anchors written in unicode characters directly)

And yesterday, I found:

• with MS Windows, Japanese Edition for me, apidoc tools cannot treat non-ASCII filenames.

Would you mind include/resolve these features/issues into Sphinx? If not, Why not?

[tk0miya]

@Suzumizaki Thank you for proposal. That sounds good.
But I'd like not to promise the target release of them. Because we don't have enough time to work for Sphinx.
I know your proposal is enough good and worthy. So I can promise you to work in my best.

Please send us PRs for them. I'll review them step by step. Always welcome :-)

[Suzumizaki]

Thanks @tk0miya, I have a question.

How to send you "PRs" as you say? For example, do you mean the normal pull request is NOT enough?

[stephenfin]

PR = pull request 🙂

[tk0miya]

Yes, I meant normal pull requests 😄

[Suzumizaki]

@stephenfin @tk0miya Hahaha... Thanks, and sorry. I understand.

I also cannot start suddenly now, but I will try to. later!

[stephenfin]

Surely 18.04? That will be released in a few months (2018-04).

Basically, we use active Ubuntu LTS to determine what version of tooling we have to support (ex, TeXLive).
At 2019-04, Ubuntu 14.04 will go EOL. So I meant moving 16.04 for our base version of Ubuntu.

Yes, but we will not be starting 2.0 development for another few months. Why not wait until 18.04 is out instead of 16.04?

[tk0miya]

What I called "standard version" is used to determine what version of tooling we should support. It also uses for testing. We've chosen oldest ubuntu LTS for it. At present, the standard version is 14.04.

So moving to 18.04 means relatively new toolings are not supported.
For example, 18.04 provides python-3.6.3. therefore py35 support is dropped.
https://packages.ubuntu.com/ja/bionic/python3

[stephenfin]

Oh, where is this actually used? I imagine we'll still be using Travis for our CI tests so there's not reason we have to drop py35. FWIW, I develop and test on Fedora (27 at the moment) which comes with multiple Python versions 😄

[tk0miya]

It is used both testing and decision of support tooling. I don't understand why do you want to move "standard version" to 18.04.

Note: I don't know "standard version" is good wording. criteria? base version? support platform? Anyway, I wrote its purpose on last message.

[stephenfin]

It is used both testing and decision of support tooling. I don't understand why do you want to move "standard version" to 18.04.

Oh, just so that we're using the latest and greatest LTS distro. It seemed silly to use an older version when a newer one exists. Perhaps I am missing something. It is not a big deal either.

[stephenfin]

I've been thinking a little more on this, and I don't think we should release 2.0 until spring 2020. The reason for this is autodoc. That currently supports Python 2.7 and Python 2.7 is not deprecated until 2020. If we drop Python 2.7 support before then, we will leave anyone who's still using Python 2.7 high and dry despite the fact that it is indeed supported. If we wait until 2.7 is final EOLd, we can happily drop this knowing that no users are affected. Suggesting that users use the older version of Sphinx isn't really an option, since we don't support those.

An alternative approach would be to retain Python 2.7 support until 3.0. However, that might not be something you want to do?

[tk0miya]

Oh, just so that we're using the latest and greatest LTS distro. It seemed silly to use an older version when a newer one exists. Perhaps I am missing something. It is not a big deal either.

If we'll use latest LTS for testing, how do you confirm Sphinx works fine with old version?
IMO, we should use it for testing to confirm working.
Of course, I'm okay to use latest one partially (or use older one partially). But we have to use it for testing at least one if we support it.

If we drop Python 2.7 support before then, we will leave anyone who's still using Python 2.7 high and dry despite the fact that it is indeed supported.

Yes, it's difficult question. But, TBH, I can't imagine who uses py27 and latest Sphinx.
I understand there are still many py27 users for some reason. I guess they don't need to use latest Sphinx.

So I prefer to extend support term of Sphinx-1.8 for a while (if somebody really wants).

An alternative approach would be to retain Python 2.7 support until 3.0. However, that might not be something you want to do?

If my opinion; the extension of support term is rejected, I'd like to vote this.
I feel postponing 2.0 to 2020 is almost same as releasing 3.0 in 2020.

[stephenfin]

If we'll use latest LTS for testing, how do you confirm Sphinx works fine with old version?
IMO, we should use it for testing to confirm working.

OK, that's a good point. If it's only for testing, then 16.04 works. I personally use Fedora so it's good to have a spread of base OSes. The main issue I do see with 16.04 is my comment in tox.ini about waiting for 18.04. That means we'll have to wait another two years for this 😞

So I prefer to extend support term of Sphinx-1.8 for a while (if somebody really wants).

If my opinion; the extension of support term is rejected, I'd like to vote this.
I feel postponing 2.0 to 2020 is almost same as releasing 3.0 in 2020.

Either of these would be suitable for me. I just want to make sure we don't leave Python 2.7 users in the lurch with broken and unsupported Sphinx.

[tk0miya]

The main issue I do see with 16.04 is my comment in tox.ini about waiting for 18.04. That means we'll have to wait another two years for this 😞

Please wait for two years. It is too aggressive to request upgrading to 18.04 for all developers.

[tk0miya]

Is there any comment for supporting py27?

[stephenfin]

Is there any comment for supporting py27?

From my perspective, I would like to support this until 2020. This would probably mean we should wait to drop it until 3.0?

[Zac-HD]

Is there any comment for supporting py27?

I would really like to drop it, like Django. (but please use the python_requires keyword in setup.py; see django/django#9413).

From my perspective, I would like to support [Python 2] until 2020. This would probably mean we should wait to drop it until [Sphinx] 3.0?

I think it would be better to make Sphinx 2.0 require Python 3, but continue bugfixes for the last 1.x release until 2020 for Python 2. Again, Django is doing something very similar with their LTS plan.

[tk0miya]

From my perspective, I would like to support this until 2020. This would probably mean we should wait to drop it until 3.0?

I'd like to drop them all as soon as possible. So I prefer to separate codebase between 1.x and 2.x series on v2.0.0 release.
And 1.8.x will be supported until EOL of py2 (if maintainer exists).

Note: EOL of py2 is Jan 1, 2020 (see https://www.python.org/dev/peps/pep-0373/#maintenance-releases).

[tk0miya]

And 1.8.x will be supported until EOL of py2 (if maintainer exists).

I have to mention what "if maintainer exists" means.

Personally, the extension of support term is not important to me. The EOL of python2 was announced 10 years ago (https://www.python.org/dev/peps/pep-0373/ ). And there is still time about one year for release of Sphinx-2.0. I think it is enough time to migrate.
As you know, the maintainer of this project is very few. So we have to choice what we should do. It is "choice and concentration" for saving our time. And I don't want to slow down the development of Sphinx.

But it does not means I opposite to support it. I can agree to continue our supports if anybody want to do that. In other words, does anyone want to become a maintainer of 1.8 branch?
If none, py2 support will be stopped at the same time as 2.0 release (or will be updated only if critical bugs are found).

I think dropping py2 support does not harm anybody. Sphinx-1.8 will not vanish even if 2.0 released. And users can use it with pinning the version.
In addition, Sphinx-1.8 seriese will become enough stable at the 2.0 release. I suppose there are no critical bugs (or very less).

[shimizukawa]

I agree that Sphinx-2.x series only supports Python 3.

[jfbu]

I am wondering if there will be some Sphinx 1.9 still supporting Python 2.7. I am asking this thinking about some LaTeX aspects which will be released with 2.0, but not with 1.8.x because they are not 100% backwards compatible (for example #5992, which slightly modifies the behaviour of image inclusion). Depending on whether the LaTeX changes involve also some changes in writers/latex.py it may be easy or not to backport from 2.0 branch to some 1.9.x series. If it is easy to backport then it could benefit some users (I expect some latency in moving all projects to Python3).

[tk0miya]

There are no plan to release 1.9 series. 1.8.x will be final version of 1.x series.
And we'll stop to maintain them after 2.0 release. So no reason to backport them to me.

[stephenfin]

@jfbu Even if we wanted to, we couldn't support a 1.9 now. 2.0 is going to be Python 3 only and releasing a 1.9 release would mean reverting all the "drop Python 3" patches we have, along with needing to verify the features since work with Python 2.7. That's a lot of work :)

[tk0miya]

I just released 2.0.0b1. And 2.0.0 final will be within a month.
Thank you for your support.
I think this roadmap is no longer needed. Closing now.