Consider releasing Fortran standard every 3 years instead of 5

[certik]

Currently there is some pushback from the community that if some feature does not get into 202x (current plans around 2023 and the feature list is already fixed), the next opportunity is 202y (2028), which is almost 10 years from now, and then another few years for the compilers to catch up, and so in practice this is too slow to be usable before many people retire.

Another related issue is that the committee currently does not seem to consider 202y features too much, it is concentrating on 2020x features. That means that discussing and working on new features, such as those proposed in this repository, is very slow.

A more efficient way would be to release a standard every 3 years on a set schedule. What gets in gets in, what is not ready will go to the next standard. Furthermore, the committee should consider all good proposals at all times, and push them forward, because good proposals take years of refining, and it's not enough to just start in a very short window (1 year) between Fortran standards, and then stop working on new features until another short window in 5 years.

[FortranFan]

:-)))

This will require being open-minded and taking inspiration from that "other" ISO standard!! "Draft FAQ: Why does the C++ standard ship every three years?"

[milancurcic]

I am strongly in favor! Shorter feedback cycles -> more frequent minor updates. This may even percolate to faster implementation in compilers. I like it. Of course, this will come down to whether the committee can agree to do this, and how can we help to make it happen.

@FortranFan and others have raised (many times in the past) the issue of the how many years it takes for proposals to reach the committee, be considered, accepted, published, and then for the features to be actually implemented by vendors who may or may not prioritize features based on customer requests. I strongly agree and wonder what can we do about it. On one side, it seems to be the reality of things:

• Committees are slow
• Businesses follow money
• Compiler developers are scarce

On the other side, this is the Fortran development model we've grown up with and know, but there may be another path forward (complementary to the existing, not replacing it!).

I'm rambling now and going a bit off topic, but how feasible do you all think it is to plan for the long term to have a open source reference implementation of Fortran? This would include something like:

1. An open source compiler + interpreter (remember that compiled/interpreted is not a property of the language but of the implementation!) that mirrors the development of the language;
2. Allow implementing and testing experimental features and proposals, which would make it easier to evaluate in practice (both ease of use and performance), and to argue for to the committee;
3. Have a "master" branch of the language that evolves fast and in short cycles; The committee reviews proposals as usual, and releases "tags" or 3/5/whatever - yearly standard releases of the language.

I think Rust follows a similar model. The language is open source and has a reference implementation. If you check out the master, you get whatever is latest. There are also versioned stable releases (1.38.0 current). And then there are editions -- Rust 2015, Rust 2018 (analogous to Fortran standard releases). Developers who need the rock solid stuff use editions; those who want to experiment or want the bleeding edge use nightly or latest versioned build. I think it's an elegant model.

This warrants a separate issue (again, sorry for hijacking), and I'd be happy to write one soon, but for now just probing the temperature of people here.

[aradi]

I absolutely agree on the 3 years term, as it could/should speed up the language feature implementation process in compilers. It would also kind of send the message: "yes the language is still alive..."

As for the reference implementation: We have already GFortran based on GCC suite and flang/f18 based on LLVM. Both open source. As for the licensing policy, f18/llvm would be a better option for a community driven reference implementation, since compiler vendors could incorporate the reference implementation directly into their code, provided their compiler is also based on llvm. (Actually, the flang/f18 is provided by NVIDIA/PGI and I heard some rumours that Intel may also come up with an llvm-compiler backend at some point).

However, I think the main bottleneck is lack of human resources...

[milancurcic]

I agree, building from an existing implementation like f18 or maybe even LFortran rather than starting from zero.

[aradi]

Oh, I was not aware of LFortran, but indeed looking at it, that may be also an option.

[certik]

Our goal should be to have two independent compiler implementations of every proposed feature into Fortran. If those compilers can be open source, even better.

(Yes, I absolutely want LFortran to be able to eventually compile any Fortran code, and to allow implement new features on a branch.)

[tclune]

While the current F202x plan is not exactly 3 years, the concept was to make it a shorter cycle for the various reasons that have been identified in this thread. I defer to those with more experience, but it is possible that we will have x=2 rather than x=3.

There are consequences to this though. In particular the feature list is smaller per revision, and disproportionately so as there is some constant overhead of committee review for each revision. One noticeable consequence: even though generic programming is the top priority in our approved features, it is not part of F202x.

[certik]

@tskeith, thank you for being involved here. I would be interested in learning what in the proposal you disagree with and why. I'll be happy to discuss.

[tskeith]

I disagree that releasing a standard every 3 years is more efficient than every 5 years. Given the overhead associated with the release, it's probably less efficient.

If you can figure out a way to get the committee to work faster or more efficiently then it might make sense. But that should come first.

[milancurcic]

The standard is released every 5 years or less often because of how the committee works.

Of course that a more frequent release model would require the committee to work more efficiently. I think that is implied here, and in part addressed by this repository.

For comparison, consider the Rust development model where specific and narrow RFCs are continuously implemented and the language is updated in short iterations, several times per year.

[certik]

@tskeith thanks for the feedback. Yes, just releasing every 3 years while keeping the committee workflow the same would not work. I 100% agree. That is not what I am proposing.

The big part of the above proposal is to switch to a set release schedule, so we ship a new standard on a given date, what goes in goes in, what doesn't gets to the next standard. There are other approaches, as @milancurcic mentioned. I agree with you that the hardest part is to switch how the committee operates and that it must come first. And then lowering 5 years to 3 years will be much easier.

[tclune]

The optimum pace for rolling out new standards is difficult to assess. As indicated in this thread, there is some minimum overhead to get new features "right" and review the entire standard for consistency for those new features. That sets a lower bound of about 3 years.

But having a slower process (with more features) can delay when any given feature is commonly available across implementations. Each vendor makes its own choice for the order of feature implementation.

And yes, the sweet spot for frequency of releases would likely be significantly different if there were reference implementations prior to standardization.

[tclune]

With regard to pinning the date and letting incomplete features slip into the next standard: while this works well in some software projects, it is less simple in the case of a standard. Features are often not isolated and lots of little bits of the document must be edited and reviewed. Pulling those edits back out when a problem is discovered late in the day is generally worse than delaying the release and fixing the problem. But when the problem is severe or the edits are more isolated, features have been dropped rather than slowing a release.

[certik]

Part of the solution in my opinion is to also "crowdsource" the editing of the Standard itself, i.e. have the Standard latex sources as a GitHub repository (perhaps a private) and allow multiple people to collaborate on it, just like any other software project. That might be a lot more efficient than our current process of "writing the patch by hand" in a proposal, and then have a single person do the edits.

(Btw, no incomplete feature should ever slip into the standard.)

[tclune]

Quite possibly, though "efficient" is probably not the right word. It would undoubtedly be much less efficient in terms of person hours but would be distributed across many more individuals.

[sblionel]

The standard publishing process itself adds about a year to the timeline.