Concentrate formatter changes in next release

[straight-shoota]

We're frequently merging formatter changes, but usually it's tiny bug fixes for edge cases that don't affect much code.

With #12951 we have a formatter change in the development branch that will affect quite a lot of code bases.
I think it would be a good chance to introduce other impactful formatter changes in the same release. This would concentrate these changes in a single release and users don't need to constantly reformat their code bases. It's easier to do all changes A, B, C all in one go for release 1.8 instead of A in 1.8, B in 1.9 C in 1.10 etc.

These are some formatter changes pending implementation that I could find in the issue tracker:

• Formatter should normalize symbol literals and similar fragments #12938
• Reformat unescaped Unicode bi-directional control characters (Warn on unescaped Unicode bi-directional control characters #12923 (comment))
• [RFC] Formatter: remove constants alignment #11949
• Priority of unary - relative to ** #12999
• operator precedence enforcement  #5074
• Formatter: String interpolation with macro expansion #9154
• Single colon delimiter works in type names #11209
• Control characters in Char and String literals #11478 (experimental)
• Format the value after # =>  #7272 (experimental)
• Formatter should insert a new line between method invocations with do ... end #9804 (experimental)
• Allow short for loops in macros #12168
• Formatter: Allow empty line in case expression (between when blocks) #4360
• Formatter removes \ from ternary expressions spanning multiple lines #10226

Ready for enabling

• Formatter: add & to yielding methods without a block parameter #12951 (-Dmethod_signature_yield)
• Trailing commas for multiline def / macro / fun parameter lists #13169 (-Ddef_trailing_comma)
• Formatter: Add more whitespace around ProcLiterals #14209 (-Dproc_literal_whitespace)

We probably won't be able to address all for the next release, but we should try to focus at least on some of them.

[beta-ziliani]

An alternative could be to add each as opt-in, and at some point turn them all by default

[straight-shoota]

Considering that the 1.8 release is approaching soon and we have had no progress on any other changes, it's probably best to delay the release of formatter changes.

We should disable the changes from #12951 by default for 1.8.
They can be available behind a flag.

[beta-ziliani]

Let's add a preview flag to stuff in there all the stuff coming in a future release

[HertzDevil]

The formatter does not actually support -D flags right now. If we want to hide things behind a flag without "implementing" flags themselves for 1.8 it seems our only option is an environment variable

[straight-shoota]

Yeah, I was planning to first implement it with an internal flag only. This allows us to keep the specs running. We can still consider exposing it in the CLI, but that would be a separate step.

But let's continue this thread in a separate discussion, this issue should be about coordinating formatter changes, not implementation details.

[HertzDevil]

If we have multiple formatter changes but they are still not enough, do we have just a single flag for all of them, or individual flags? If it's the former it could be -Dpreview_formatter

[beta-ziliani]

I was thinking in just one flag

[straight-shoota]

Specific flags have some use internally as it clearly denotes what parts of the code belong to that feature. That's how #13215 works.
If we expose preview mode in the CLI, I would only use a single flag to enable all available preview features.

[straight-shoota]

In #14158 (comment) @Blacksmoke16 brought up that we probably shouldn't wait for all these issues to be fixed.
And yeah, this list is just meant to collect formatter changes for the purpose of grouping them.

I'm not sure what should drive the decision to enable all pending formatter changes.
The idea of this to avoid the invonvenience of formatter changes in every single release. But it's not meant to hold up improvements indefinitely. The initial expectation was these would be implemented much faster. 😅

#12951 for example has been merged for a year now. I think it outta be time for it to actually roll out.
#13169 is also ready. So we can take those two for now.

I wouldn't want to do this for 1.11 though because we're already in the freeze period.
But let's schedule their activation for 1.12. And perhaps we can get a couple more PRs going til then.