[Feature Request] Allow command line option to only reformat specific lines

[GadgetSteve]

Operating system: All
Python version: All Supported
Black version: Future
Does also happen on master: Yes

Currently if black is used on an existing project base it will likely create a huge diff as many lines will be changed. Some projects discourage this or only allow it on a major version change.

One way to address this would be to allow per file line ranges to specify that only those lines be considered for re-formatting in a similar manner to yapf --lines option. This would allow black reformatting of changed lines only.

This ticket raised in response to python/steering-council#9 (comment) but of course it could be considered a compromise.

[zsol]

This has come up several times by now, and I still don't think it's going to happen in Black for the reasons outlined in microsoft/vscode-python#134

[gpshead]

Practicality beats purity. :)

We've been allowing incremental formatting (aka "changed areas only") for years on hundreds of millions of lines of code (across a few languages including Python) at Google. We'd be unlikely consider major language formatters that do not support this at this point.

Why? It adds overhead and cognitive burden of distraction to the code review process to have areas wildly unrelated to the change at hand be altered.

Perhaps counter intuitively, this cannot always be prevented by simply formatting the entire file once at the start of formatter adoption. formatters change behavior over time as they are improved and evolve. So formatting changes to unrelated things in other areas of the code are inevitable even after a file has been fully formatted earlier in its history. Typically infrequent, but it happens.

[gpshead]

btw, if you do choose to implement this, --lines might be lower level than you want; integration with diff from the users version control system to determine the ranges of lines to focus on is what makes it easy for the end user to use such a feature.

[Carreau]

I've hacked this together in the last couple hours:

darken --since <commitish>

It's ugly, but it:

• parses git diff for changed files/lines numbers.
• insert fmt : on/off comments,
• apply black,
• removed insert fmt-comments

It has some edge cases mostly due to microsoft/vscode-python#560 but it works.
A baked-in options would be good as black tends to insert white lines around comments in imports, so inserting/removing fmt is not exactly the same as having a --line option.

[Starwort]

This but only so that VSCode integration works properly!

whenever you paste it complains that 'black does not support format selection' and it bugs me

[OddBloke]

I'd like to add my voice to this request: I am involved in projects which are happy (keen, even) to adopt a specific coding style, but are emphatically not happy for that coding style to be applied to the project in one fell swoop which muddies up git blame (and makes backporting changes to older releases harder). So, for now, black is off the table for us.

[mrgnw]

re: @Starwort

whenever you paste it complains that 'black does not support format selection' and it bugs me

This setting should take care of that. https://github.com/Microsoft/vscode-python/issues/1883#issuecomment-395216906

"[python]": { "editor.formatOnPaste": false }

That said, I do think partial reformatting (particularly via VSCode, etc.) would help convince my team to use Black by making it easier to gradually make changes before selling everybody on the whole thing.

(Checking out darken, maybe we can hack an editor extension for that in the meantime.)

[stefan-truework]

which muddies up git blame

@OddBloke Checkout recent versions of Git, which learned to ignore specific versions of commits as either specified on the command line in a file or its configuration, https://git-scm.com/docs/git-blame#Documentation/git-blame.txt---ignore-revltrevgt (since https://github.com/git/git/blob/master/Documentation/RelNotes/2.23.0.txt)

[DylanYoung]

I'm going to second this one as well (for all the reasons listed above), but will note that while a --diff option is nice for cli usage, it's not ideal for editor usage (the editor has to create a diff, then pass it into black instead of just passing the lines directly).

Moreover, flake8 has this option and it doesn't play nice with VCS when doing a bunch of commits on already completed work. This is because while it analyzes the diff (in this case a git diff --cached) to get line numbers, it then uses the real file to calculate the errors and warnings: since the real file has unstaged changes not present in the stash, you can end up with improperly formatted commits. It seems to me that getting it to play nice with VCS isn't possible without more context (another diff of unapplied changes perhaps? Or some other method to get the contents of the file before the diff, but without any other changes). Though I supposes --lines has the same problem.. Just something to keep in mind.

[wbolster]

fyi, i use partial formatting from my editor whenever i cannot reformat files. reformatting everything is always my preferred option, but not always possible.

black-macchiato works fantastically well for me, especially with the emacs integration to reformat the current (potentially multiline) statement.

https://github.com/wbolster/black-macchiato

[bytearchive]

This seems like a blocker for us adopting also

[h]

My team is also blocked from adopting black because of this issue.

[akaihola]

The darker tool applies Black formatting only to lines which have changed in the Git working tree since the last commit.

[DylanYoung]

@akaihola I'm now getting that same error message with vscode using darker. It was working at first though, so maybe a bug in vscode...

Black does not support the "Format Selection" command

[akaihola]

Maybe Black's documentation could point to darker. It can be used as a wrapper to Black, and may be the solution for some people to the problem of not wanting to reformat a complete code base in one go.

[wasdee]

Maybe Black's documentation could point to darker. It can be used as a wrapper to Black, and may be the solution for some people to the problem of not wanting to reformat a complete code base in one go.

I wonder if we could use darker with vscode.

Oh, there is the way to config. 🎇

[Carreau]

Oh, there is the way to config. 🎇

I suggest you also follow this issue for a per repo config file and look at the --revision option for CI integration and make sure all new node is covered.

[felix-hilden]

Some updates: this was recently discussed in another thread: microsoft/vscode-python#1352. It started as vim-specific but has since broadened. I won't close this one though, since it has so many reactions. Looks like the tide is turning for formatting ranges.

[fearedbliss]

Adding my support in favor of this. Even though I personally like running black on the whole file, sometimes in work environments it's necessary to reduce the amount of changes made, especially if the team hasn't fully transitioned to blackening their code base. In the intermediate transition period, black can still be beneficial by allowing people to continue producing new code in those code bases, but only blackened the new parts of the code. This allow both, nicely formatted code with black, and reducing mental overhead on the team.

PS: If someone really wanted to, they could have a copy of the old file (or keep it open in a non-automatically refreshed IDE buffer), run black, copy the newly formatted code, and then Ctrl-Z (or restore the old file), and place the code back in... or they could just copy/paste their new code into a temporary/small new file, black that, and move the code back in place. This is just annoying though and not a productive workflow. But it would technically be the only workaround possible.

[dwt]

Finally! :-) Any idea when this will be released? Sorry I am not very knowledgeable about the black release cycle.

[JelleZijlstra]

Likely later this week.

[fearedbliss]

Thanks for implementing this :).

[GadgetSteve]

Many thanks to @yilei for implementing this!