Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC 0101] Nix formatting #101

Closed
wants to merge 7 commits into from

Conversation

happysalada
Copy link

@happysalada happysalada commented Aug 17, 2021

automated formatting of nix files in nixpkgs

rendered

@edolstra edolstra changed the title [RFC 101] nix formatting [RFC 0101] nix formatting Aug 17, 2021
@edolstra edolstra changed the title [RFC 0101] nix formatting [RFC 0101] Nix formatting Aug 17, 2021
@gytis-ivaskevicius
Copy link

gytis-ivaskevicius commented Aug 17, 2021

May I note that we can not reformat the whole codebase just yet because it will just result in conflicts between 21.05 and unstable branches? I believe the preferable time to do so is right before 21.11 branch off.

Food for thought:

  • Is lack of linter an actual issue in nixpkgs? (I don't think so??)
  • If *.nix file formatting gets approved - shall we get some formatting rules going for sh/json/py/perl/etc?
  • If we are being this anal, I'd say it's about time that we add some sort of devshell to nixpkgs (Maybe we could commit in a stripped-down version of https://github.com/numtide/devshell to nixpkgs? )
  • I think we should go over nixpkgs-fmt rules and tweak them a little to encourage a little bit more dense formatting.

[alternatives]: #alternatives

- Alternative formatter [nixfmt](https://github.com/serokell/nixfmt)
- Keep the status quo of not having an official formatter. The danger is that this creates discord within the nix community. On top of fragmented the community it can generate lengthy discussions (which do not advance the eco-system).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is also an alternative of explicitly discouraging any comments on formatting not directly based on coding style recommendations spelled out in the Nixpkgs manual.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say that it would be step in wrong direction. As far as I remember, nothing in nixpkgs manual says that your expression should contain newlines at all.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, discouraging is not the same as strictly forbidding, if you in good faith consider something an exceptionally unfortunate formatting decision, you could comment about that… with a link to a PR to the manual, obviously.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where would you draw the line? It should be explicit, otherwise one reviewer thinks one part is exceptionally unfortunate and the next thinks another part is exceptionally unfortunate.

Copy link
Member

@sternenseemann sternenseemann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit torn on this one. On the one hand, I feel like having a formatter is just something we'll have to get over with, but at the same time I'm sceptical it'll solve the problems we are having and may have some risks:

  • Formatting is not the only point of contention. There's also endless discussion about other aspects of nix expressions that have little to no actual consequence, e. g. meta = { maintainers = [ lib.maintainers.xyz ]; } vs. meta = with lib; { maintainers = with maintainers; [ xyz ]; }. To fix these issues once and for all we would need to have a full blown coding style guide.
  • Depending on the implementation it may become more cumbersome to contribute (for the first time), especially if you have to install extra software. A PoC of a GtiHub action that does formatting would probably be useful here.

I also want to echo @7c6f434c here: We should also consider establishing an explicitly liberal formatting and code style policy: nixpkgs is a project of many occasional contributors who we can't realistically expect to familiarize themselves with a lengthy contributing guideline (already first time contributors often miss the commit format section).

[unresolved]: #unresolved-questions

- Not sure how much work there is left on nixpkgs-fmt before most people would consider it ok to use. Not even sure how much it is actually used.
- Are there situation where automated formatting is worse than manual formatting ? Do we want to have exceptions to automated formatting ?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One major question mark for this RFC is the question of automatically generated source files. We have multiple tools currently used to automatically generate files in nixpkgs: hackage2nix, node2nix, quicklisp-to-nix, go2nix, bundix to name a few.

I think none of these generate code that any formatter is happy with at the moment. If they were to be reformatted, the changes would be rolled back the next time the files are regenerated.

I think the RFC needs to commit to one of the following solutions:

  • Update code generators to produce conforming code. Then the question becomes: Who is gonna do that? Is it feasible? Has it any advantage? The burden to update the code generators may be significant as you can cut quite a lot of corners when generating nix because whitespace is not syntactically significant.
  • Make exceptions for generated files. The question is how this could be implemented in a hassle-free way since some generated files (think *deps.nix) don't have a well-known location and may be moved around as well.
  • Forbid code generation. This could solve this problem as well, but would require a separate RFC which proves that it's doable. On additional incentive for such a change would be that it likely improves performance: We could use JSON files to encode the same information as before and Nix's JSON parser is must faster than the Nix parser itself.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have to admit that updating code generation by putting the generator call inside a script and calling nixpkgs-fmt afterwards sounds like a nonzero but manageable amount of work.

On the other hand, maintaining generators to always be formatter-clean would create unncessary pushback to fixing some weird corner cases.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO just fixup the generated files after they have been generated. Either the tool itself can do that, your pre-commit hooks, or the update script. Either of these works. If we enforce formatting of files then almost no file should be excluded. Especially not generated files as being able to read them is important. There is no excuse to treat those a black boxes that shouldn't be looked at.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original tool should be fixed to always comply, to avoid churn. Or we marke the file "no format". Or we do #109


[drawbacks]: #drawbacks

- Having a commit that changes the formatting, can make git blame harder to use. It will need `--ignore-rev` flag.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore-rev can be set up convenient-ish-ly by contributors by running one to two git config commands. However it needs to be stated that GitHub doesn't support this feature at all. I'm guessing the web blame feature is quite frequently used.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made a pr for a .git-blame-ignore-revs file
NixOS/nixpkgs#162430

According to a github employee they're currently working on the web blame

Update: implementation of this feature is in progress
6 days ago

community/community#5033 (comment)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation of this RFC should include several parts

- Agree on an automated formatter. [nixpkgs-fmt](https://github.com/nix-community/nixpkgs-fmt) is proposed here.
- If adopted, agree on a schedule to format nixpkgs codebase. Should it be done at once, or on a per package basis.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One good point made by Domen in NixOS/nixpkgs#121490 is that it should be done once, on a treewide basis, just before branch-off of a stable release: Otherwise backporting changes from unstable will almost always run into merge conflicts which have to be manually resolved.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If nixpkgs-fmt changes the content of values, I would consider this as a bug. Unfortunately due to how the internal data structures that rnix-parser is using, deal with multi-line strings and comments has been particularly challenging.

If somebody could extract the repro and post an issue to nixpkgs-fmt that would be great.

[drawbacks]: #drawbacks

- Having a commit that changes the formatting, can make git blame harder to use. It will need `--ignore-rev` flag.
- Every formatter will have bugs. An example of a bug for nixpkgs-fmt can be found [here](https://github.com/NixOS/nixpkgs/pull/129392)
Copy link
Member

@sternenseemann sternenseemann Aug 17, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another drawback is that, as it stands nixpkgs-fmt is unable to reformat indented strings in a way that doesn't change their content. As a result, a reformat of nixpkgs will trigger a mass rebuild (see NixOS/nixpkgs#121490). Sending the reformat through staging will be quite painful as we'll have to deal with constant merge conflicts in the staging-next phase.

This implies that the “ideal” solution I've outlined in my other comment may not actually be doable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alejandra gets multi-line strings done perfectly, plus only 94 out of 34079 derivations would need rebuilding (and for reasons that can be happily ignored, see footnote on the readme)


The implementation of this RFC should include several parts

- Agree on an automated formatter. [nixpkgs-fmt](https://github.com/nix-community/nixpkgs-fmt) is proposed here.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That formatter would need to become a channel blocker for the nixpkgs-* channels (like e. g. nixpkgs-review is).

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to propose Alejandra to be taken into consideration as well.

@7c6f434c
Copy link
Member

7c6f434c commented Aug 17, 2021

I actually think that style guide with «please fix any violations in freshly-changed parts if asked, please do not ask to change formatting without a guideline» could be efficient at shutting down the improperly placed discussions about style regardless of whether the guide is short or excessively long or in-between.

(I probably prefer a short one).

One thing that has a high risk of relying on semantically described guidelines even with a formatter is vertical whitespace — I think there is a preference to group things like version/pname/source — build parameters — meta, with empty lines in between, and I have some suspicion some prefer this tradition to stay but an automated procedure won't enforce ensure this well enough.

@sternenseemann
Copy link
Member

For nixpkgs-fmt, preserving empty lines is at least a design goal:

Respect the developer's expressivity. Empty lines can be useful as a way to separate blocks of code.

@7c6f434c
Copy link
Member

Yep, nixpkgs-fmt takes a reasonable approach, I just meant some formatting guidelines still makes sense even with a formatter.


[alternatives]: #alternatives

- Alternative formatter [nixfmt](https://github.com/serokell/nixfmt)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enforces short line lengths which is not ideal with src in fetchers.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If single string literal is too long, it is kept, so I see little problem with it. Can you provide example when it generates outright ugly results?

Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
@bhipple
Copy link

bhipple commented Aug 22, 2021

+1 from me on just applying nixpkgs-fmt and calling it a day. Contributing to codebases with automatic formatters is generally much more pleasant, and nixpkgs-fmt's design goals seem ideally suited to nixpkgs.

Formatting is not the only point of contention. There's also endless discussion about other aspects of nix expressions that have little to no actual consequence ...

Agreed, but best not to let the best be the enemy of the good.

Copy link
Member

@zimbatm zimbatm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm glad somebody is trying to take nixpkgs-fmt to the finish line. Thanks!


- Agree on an automated formatter. [nixpkgs-fmt](https://github.com/nix-community/nixpkgs-fmt) is proposed here.
- If adopted, agree on a schedule to format nixpkgs codebase. Should it be done at once, or on a per package basis.
- Potentially agree on a hook to enforce formatting
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be implemented as a github action pretty easily. $(nix-build -A nixpgks-fmt)/bin/nixpkgs-fmt .

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is also treefmt, which might be used to exclude specific path to address problematic files.
Here is a test run of treefmt on nixpkgs.

List to consider for exclusion:

The implementation of this RFC should include several parts

- Agree on an automated formatter. [nixpkgs-fmt](https://github.com/nix-community/nixpkgs-fmt) is proposed here.
- If adopted, agree on a schedule to format nixpkgs codebase. Should it be done at once, or on a per package basis.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If nixpkgs-fmt changes the content of values, I would consider this as a bug. Unfortunately due to how the internal data structures that rnix-parser is using, deal with multi-line strings and comments has been particularly challenging.

If somebody could extract the repro and post an issue to nixpkgs-fmt that would be great.

@Mic92 Mic92 added status: open for nominations Open for shepherding team nominations and removed status: new labels Sep 2, 2021
@Mic92
Copy link
Member

Mic92 commented Sep 2, 2021

Who wants to shepherd for this RFC?

@zimbatm
Copy link
Member

zimbatm commented Sep 8, 2021

I would be up for it. I am biased as one of the nixpkgs-fmt author of course.

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixpkgs-fmt-bot/15002/4

@nrdxp
Copy link

nrdxp commented Sep 13, 2021

I can shepherd as well if you still need a few more.

@Mic92
Copy link
Member

Mic92 commented Sep 14, 2021

Ok. We still need at least one more shepherd for this.

@0x4A6F
Copy link
Member

0x4A6F commented Sep 14, 2021

I nominate myself as a shepherd to fill in the blank.

@legendofmiracles
Copy link

While I support a standard formatter in nixpkgs, I have some gripes with nixpkgs-fmt. It does not print syntax errors when formatting, instead it goes on and removes any syntax after the line where the error is made, making it very confusing and frustrating to use.
Sure, you can run the formatter again after you fixed the issue, but the syntax errors that it didn't show will break everything again.

@Mic92
Copy link
Member

Mic92 commented Sep 23, 2021

The sheperds for this RFC are zimbatm, nrdxp, 0x4a6f, with @zimbatm as the shepherd leader.

0101-nix formatting.md Outdated Show resolved Hide resolved
@rhendric
Copy link
Member

rhendric commented Sep 6, 2023

I'm aware, but

  • Rewrite/Update the RFC to reflect the current status. Document all the current formatting decisions and justify them.

suggests that it is still in flux.

My concern isn't ‘nobody will ever invest person-hours into working on rules’. My concern is if the team can't put together a rule set and say, ’Here it is, ready for your review; it isn't perfect and we expect to evolve it in the future but here's a complete specification of what correctly-formatted means,’ then we may be granting authority to a team that is more invested in the journey than the destination.

And no, I don't consider ‘whatever the current implementation happens to do right now’ an adequate specification for this.

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixcon-governance-workshop/32705/3

@adamcstephens
Copy link

adamcstephens commented Sep 7, 2023

My support of this RFC hinges on whether the people to whom the RFC assigns authority over formatting are on team ‘any rules are better than no rules’, as a number of recent commenters seem to be.

I find this gatekeeping mildly offensive, and I am not even someone who has spent a lot of effort working on this. It's pretty clear that those working on the rules do not share this view, or they would have pushed for the use of one of the existing formatters instead of dedicating a non-trivial amount of their time updating and implementing rules into nixfmt.

Personally, I am of the opinion that any formatter is better than no formatter. nixpkgs maintainers are already stretched too thin, and the status quo requires extra effort to maintain. Files must be manually formatted, pull requests go through extra reviews because of formatting, and there is no consistency across the code base. In short, we are wasting people's time needlessly without one.

@rhendric I'm sure there would be plenty of space for you to help contribute to this effort, and I'd encourage you to do so if you feel so strongly about this topic. If you're not interested, or don't have the time to do so, instead I'd ask for you to make space for those who will be working on the implementation to make the decisions. This process, and the resulting formatter, as you point out will never be perfect, but putting up roadblocks is no more productive than rubber stamping a formatter.

@NotAShelf
Copy link
Member

Files must be manually formatted, pull requests go through extra reviews because of formatting, and there is no consistency across the code base. In short, we are wasting people's time needlessly without one.

couldn't have said it better

@rhendric
Copy link
Member

rhendric commented Sep 7, 2023

I apologize for any offense I caused.

I thought the RFC process was intended to gather the opinions of the community. As a community member, I didn't think that expressing my opinion on an RFC would constitute ‘gatekeeping’.

Also, I find it pretty odd to be accused of gatekeeping in a message that then proceeds to tell me to leave the conversation to the more active members.


To your argument, bad code formatting standards impacts readability, which would also waste people's time when they are trying to understand what is going on in a large and complex project like Nixpkgs. The two time wastes, along with alternate mitigations for each, should be weighed against each other—I disagree that there is a no-brainer argument for implementing a formatter, any formatter. I'd go into more detail but you seem pretty upset that I would dare to share my thoughts here so I'll leave it at that.

@adamcstephens
Copy link

So you think the status quo, of inconsistent formatting across the codebase (along with the other negatives I listed above) is potentially lower than the cognitive load of an imperfect formatting? Seems like a good research track for an academic to study, but where I sit as a maintainer I fail to see how that helps here. I know that I spend time manually aligning and rearranging code, when a computer could do this much more efficiently. I've also reviewed and been subject to review of changes that alter formatting, requiring extra cycles through the review process.

We have a lot of smart people who have worked on this, and when I encouraged you to join in I was hoping you could lend your expertise. But if you don't want to lend anything but negative roadblocks to what I consider a crucial need in the Nix ecosystem, I'm going to push back on that negativity.

My support of this RFC hinges on whether the people to whom the RFC assigns authority over formatting are on team ‘any rules are better than no rules’, as a number of recent commenters seem to be.

We need positive contributions to make this happen, not dismissive statements about those with different opinions. If we can't move this RFC forward because of all the bike shedding over specific rules, then yes I want to delegate this responsibility (with accountability!) to a smaller more focused group. I definitely didn't tell you to leave it to others, but it's clear we're at odds here so maybe you missed my invitation for you to be more involved.

@rhendric
Copy link
Member

rhendric commented Sep 7, 2023

I didn't intend to be dismissive of anyone. I'm sincerely sorry that I caused you to feel that way.

Can we start over?

piegames said:

If people are willing to throw out all actual style discussion from the RFC text, and delegating everything to the implementation and the team, then a lot of work could be saved. This would require a lot of trust from the community into the people behind this though.

I took that as an implicit question: is the community willing to let the implementation team cut this corner?

If that question is addressed, in part, to me, as a community member, then my answer is, respectfully, no. That doesn't mean I think the people who say ‘yes’ are to be dismissed. Nor does it mean that I feel so strongly about this that I would throw myself into the machine to stop the gears of progress. I was asked a question and I gave an honest answer that happens to disagree with yours. That's a ‘positive contribution’, surely?

I probably wouldn't have spoken up at all if this hadn't been prefaced in my inbox with a small flood of messages from people expressing the sentiment that any formatter is better than no formatter. I intended to communicate that I don't want the team to adopt that sentiment themselves. I wouldn't have expected them to prior to this flood of messages, but given the context—hey community, should we be less diligent so we can finish faster?—I thought that if all of the responses were ‘yes’, there was a good chance that they would take that response seriously and declare mission accomplished prematurely, even though prior to now that isn't what I would have expected from them.

If you still think that's a ‘negative roadblock’, then I really don't understand how you expect community feedback processes to work. The team asks the community more or less a yes-no question, but only one answer is permitted because the other answer constitutes being negative? I don't get it.

As for the object-level question, we might have to agree to disagree. I would certainly prefer a world in which a good-enough formatter is applied consistently to all of Nixpkgs to the world in which committer-maintainers have free reign within their domains. I only expect a small number of things out of a formatter for it to qualify as good-enough. I'm skeptical, though, that if you took the union of every maintainer's small-number-of-things, you would still have a small number of things—or even a consistent set of things. The best way, IMO, to demonstrate that this is possible is with an existence proof, and I would like to see that proof before I agree that it would be good for the project to apply a uniform format to Nixpkgs. I respect your right to disagree and voice your disagreement, as I would hope you do mine.

@emilazy
Copy link
Member

emilazy commented Sep 7, 2023

(I do want to clarify that although you can probably fairly reasonably gloss my bottom-line opinion as "any formatter is better than none", I wasn't intending to express that with any of my comments here, as opposed to more tightly-scoped sentiments like "formatting standards that are technologically possible to use with the Nix of today are preferable to 5 more years of upstream prerequisite work" and "community consensus is important for high-level matters like whether to format at all and the principles of the formatting rules, but the minutiae of how to apply those to every single detailed case is better handled by a focused group of experts than indefinite public bikeshedding of a necessarily-incomplete natural-language specification". That said, I expect we do still have fundamental disagreements here all the same.)

@rhendric
Copy link
Member

rhendric commented Sep 7, 2023

"formatting standards that are technologically possible to use with the Nix of today are preferable to 5 more years of upstream prerequisite work"
"community consensus is important for high-level matters like whether to format at all and the principles of the formatting rules, but the minutiae of how to apply those to every single detailed case is better handled by a focused group of experts than indefinite public bikeshedding of a necessarily-incomplete natural-language specification"

And for the record, I fully cosign the first, and mostly cosign the second with my only caveat being that I think the experts should come back to the public with those minutiae—but only so that the public can see the principles in action, ask questions that might not have occurred to them absent the minutiae, and give a general approval vote, not to start a new round of bikeshedding.

@justinlovinger
Copy link

I am going to throw my weight in on the side of "formatting rules don't matter". The biggest issue with Nixpkgs today is too many unmerged pull-requests. Automated formatting will streamline pull-requests, regardless of specific formatting rules. To that end, our focus when selecting a formatting tool should be stability and compatibility, not specific formatting rules.

From my personal investigations, the most compatible and stable formatter is Alejandra. nixfmt and nixpkgs-fmt have broken LSP integration, resulting in full-document replacement and lost cursor positions when automatically formatting, helix-editor/helix#2655.

@rhendric
Copy link
Member

rhendric commented Sep 12, 2023

Nah, the most compatible and stable formatter is cat. If you literally don't care what set of rules the formatter implements, cat has some unbeatable advantages:

  • No need to adopt the project into NixOS!
  • Literally zero bugs!
  • It comes pre-integrated into the pull request pipeline and every code editor—no plugin required!
  • It is guaranteed to 100% accurately reflect the original code author's intent!

Using cat would dramatically reduce the time the RFC team needs to spend on implementing formatter technology. All we would need to do is find consensus—get the committer-maintainers to abide by the principle that anything our chosen formatter outputs is acceptable formatting and not to hold up pull requests on any nitpicks our formatter doesn't catch. (This is the same consensus any formatter would need for actually solving helping to solve the slow-PR problem.)

If you think this is a good idea, then I think you are probably in favor of a different RFC than this one, unless this one basically gets a lobotomy. If you think this is a bad idea, then you do care (or, perhaps more likely, you're concerned that certain others care) about the set of formatting rules after all, because that's the only way cat could be considered inferior to any other formatter.

@justinlovinger
Copy link

justinlovinger commented Sep 12, 2023

Nah, the most compatible and stable formatter is cat.

If cat could streamline to pull-request process, I would be in favor of it. Unfortunately, it cannot.

We should evaluate options in context of the purpose of this RFC. As of writing, the motivation section says:

  • We want to prevent debate around how things should be formatted.
  • We want to make it as easy as possible for contributors (especially new ones) to make changes without having to worry about formatting.
  • Conversely, reviewers should not be bothered with ill-formatted contributions.
  • We want a unified Nix code style that's consistent with itself, easily readable, accessible and results in readable diffs.

Only the last bullet-point touches on specifics of formatting rules.

I suggest, to move this RFC along, we split the specifics of formatting rules into a separate RFC for the creation of an official Nix style-guide. We can debate details of code-style there.

This RFC can focus on automated formatting for Nixpkgs. When the official style-guide is ready, the formatter we choose will either implement it, or we will migrate to a formatter that does.

@rhendric
Copy link
Member

If cat could streamline to pull-request process, I would be in favor of it. Unfortunately, it cannot.

How do you think any other formatter would streamline the pull request process? What do you think makes cat different?

As I see it, the only thing that contributes to streamlining the pull request process is this step:

get the committer-maintainers to abide by the principle that anything our chosen formatter outputs is acceptable formatting and not to hold up pull requests on any nitpicks our formatter doesn't catch.

If the formatter is cat, this means committer-maintainers agreeing not to ever hold up a pull request on a formatting nitpick. That'll streamline things just as much as if we adopted nixfmt or anything else.

It might be a harder sell, you might object? You might have a harder time getting committers to agree not to argue with cat than to agree not to argue with nixfmt? Why would that possibly be? Could it be because the rules enforced by the formatter affect how willing people are to defer to automated formatting?

@7c6f434c
Copy link
Member

Well, there are people like me who agree that cat and autodeletion of formatting related comments would be the best answer, there are people who want kind-of uniformity more than any specific details (stuff like black-formatting NixOS tests could not come from any other motivation), then there are people who have specific opinions on formatting.

In the meantime, Nixpkgs has some formatting conventions that are bad for reading the code (the goal being to improve reading the diffs).

Most formatters can probably be sold to uniformists (sadly cat cannot), people with actual opinions won't all be happy at once anyway, and damage minimisation is useful to reduce the risk of people like me caring enough to try deadlocking things.

@infinisil
Copy link
Member

infinisil commented Sep 12, 2023

2023-09-12 Meeting

Attendees: @infinisil, @piegamesde, @0x4A6F (lurking)

  • The discussions on this PR have diverged from the current state within the shepherd team. This is partially due to the fact that the PR is marked as draft and work on the text is happening in a separate branch.
    • The shepherd team will keep working on this, and plans to continue the discussion in a new pull request soon.
    • We think that this pull request can be closed now, CC @happysalada
  • Pinged the nixfmt repo owners here for updates
  • Action item: Write a test for nixfmt to enforce the sub-expressions-on-lines rule
  • Continued working together on the RFC text: updates

@happysalada happysalada deleted the nix_formatting branch September 13, 2023 13:20
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/proposal-require-non-committer-review-for-prs/33307/6

@infinisil
Copy link
Member

2023-09-26 Meeting

Attendees: @infinisil @piegamesde @0x4A6F @tomberek Sergey (Serokell) Roman (Serokell)

Agenda:

  • Discuss Future of the repository nixfmt#129 with Roman and Sergey from Serokell

    • Where to put the repo?
      • Wait for the RFC, then move to Nix organization
    • Sergey would also want to be in the formatting team
  • Talk about the release

    • Last release before the RFC style
    • Hackage permissions, add infinisil to the maintainers of nixfmt
    • nixfmt.serokell.io: Not necessary to transfer/copy
    • @infinisil: Manage the release?
    • Bump GHC and dependency version
    • @infinisil will also get permission to nixfmt
    • Switch CI to GitHub Actions (from buildkite)
      • Needs some secret adjustments
      • Use Serokell's Self-hosted CI runners for now
    • Remove the requirement for signed commits on the repository
    • @Lucus16 has been doing the previous releases, @infinisil will talk to him
    • No need to do anything in Nixpkgs, Haskell updates happen regularly
  • Discuss RFC 101/166 style nixfmt#118

    • Agreement that there's no need to split it into smaller PR's, it's a big change
    • Most importantly, make sure the tests are proper
    • Merge it only after the next Nixfmt release
      • Then can close issues which don't apply anymore
  • Semicolon style:

    • Current rule: In (non-inherit) binders, only expressions where the last line doesn't have unbounded indentation can have the semicolon on the same line:
    let
      # If statements have unbounded
      x =
        if cond then
          foo
        else
          bar
      ;
    
      # Not unbounded
      y =
        "foo bar baz"
        + "foo bar baz";
    
      # Bounded, but it makes more sense like this
      inherit (foo)
        bar
        baz
      ;
    
      # Nested operations are unbounded
      z =
        if foo then
          bar
        else
          "foo bar baz"
          +
            3
            * 5
      ;
    
      # Bounded
      y = {
        
      };
      
      # Bounded
      zsdfjklsdkjflskdfj.dsjfklsdf =
        f:
        {
          a = 10;
        };
    in
    x
    • @infinisil (and probably other people) is not entirely happy about the unintuitive rule, but might make sense
    • Let's try it out and see how many people complain about it
  • Merged Rework a lot of things nix-rfc-101/rfcs#1

  • Updated the RFC draft text: Update 0101-nix-formatting.md nix-rfc-101/rfcs#2

@infinisil
Copy link
Member

infinisil commented Oct 10, 2023

2023-10-10 Meeting

Attendees: @infinisil @0x4A6F @Lucus16 Sergey @tomberek @piegamesde

@infinisil
Copy link
Member

2023-10-31 Meeting

Attendees: @piegamesde @infinisil Sergey

@infinisil
Copy link
Member

infinisil commented Nov 14, 2023

2023-11-14 Meeting

Attendees: @infinisil @piegames @tomberek @0x4A6F Sergey

  • When to merge RFC 101/166 style nixfmt#118
    • Now is not great, if the RFC is rejected we wouldn't want to maintain nixfmt
    • But later is also not great because master is diverging
  • Open a new RFC or pseudo-continue the previous one in a new PR?
    • We can probably go a bit faster since we have a shepherd team already and there's been continuous meetings
    • But need to honor the RFC process
    • @piegames wants to keep the RFC number
  • Line length limit (Column width experiments piegamesde/nixfmt#3 and https://github.com/piegamesde/nixpkgs/commits/nixfmt-3)
    • @piegamesde's recommendation is 100-char ignoring indentation, 120-char with
    • Agreement: 80-char width is legacy and too small
    • Do we need a line length limit at all?
      • Yes, without a limit the current code would format it all on a single line
      • Formatter works by making lines have a reasonable amount of information
      • How about never contracting multi-line code?
        • Ideal if the format is a function based on the AST only, why?
          • Two people writing the same code should get the same formatting result
          • Is that worth it?
        • What can even get contracted? Lists, attribute sets, inherits can only be on a single line if they have not too many elements
        • Even if we never contract, still need a line length limit to expand huge single-lines
          • Could still work, trivial example of inserting newlines between every token
          • But needs a lot more rules
      • Agreement: Line length limit is fine to have
    • Agreement:
      • 100-char line length limit ignoring indentation, optional hard line limit
      • Optional special cases for string-like values like strings, paths, comments and urls
    • @piegamesde will implement the 100-char limit in RFC 101/166 style nixfmt#118
    • What should we write down in the RFC?
  • How to make it easy for people to try out the formatter?
    • Nixpkgs channels are too slow, we don't want people to use old versions
    • Fetch directory from git, e.g. with the Flakes
      • Dependencies are cached in cache.nixos.org, only nixfmt needs compilation
        • Consider turning off profiling, speeds up the build
      • nix build github:piegamesde/nixfmt/rfc101-style
      • flake.nix
        {
          inputs.nixfmt-rfc101-style = {
            url = "github:piegamesde/nixfmt/rfc101-style";
          };
        
          outputs =
            { nixfmt-rfc101-style, ... }:
            {
              formatter.x86_64-linux = nixfmt-rfc101-style.packages.x86_64-linux.default;
            };
        }
        allows nix fmt to format all Nix files
      • niv add piegamsde/nixfmt --branch=rfc101-style
      • npins add github piegamesde nixfmt --branch rfc101-style
  • Review of the RFC text, updates: Update after meeting nix-rfc-101/rfcs#7
    • Added some more examples and explanation for the function declaration formatting
    • Updated examples to match the latest style
  • @infinisil will do some final polishing on the RFC text and write a description for the new RFC, pass it off to the others for final review before opening the new one later this week.

@piegamesde
Copy link
Member

Reopened as #166

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/enforcing-nix-formatting-in-nixpkgs/49506/3

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/satisfaction-survey-from-the-new-rfc-166-formatting/49758/54

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-formatting-team-treewide-nixpkgs-formatting/56666/1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.