Commit Signature Verification #8848

flandweber · 2023-08-18T13:11:55Z

I don't have a lot of experience developing C++ and am completly new to this project so I'd be really grateful for feedback on this.

Motivation

When fetching a derivation from a git repository it is desirable to verify that the commit was made by the code owner and not altered by the git server.
Commit signatures are a git-native way of verifying this.

This can be useful for

administering systems through untrusted git servers (by verifying with your own public key)
having a second check when building a program from source (by verifying with the developers key)

Context

Mainly #2849, although this implementation is only restricted to the git fetcher.
Since this can also be used for source code signature verification: #613 and https://discourse.nixos.org/t/any-interest-in-checkings-signatures-while-building-packages/8918

This is much simpler and more limited than what NixOS/rfcs#100 suggests since this PR is only about verifying the current commit with a provided public key.

Implementation

To implement this change I added verifyCommit, keytype, publicKey and publicKeys as an additional git input attributes.
To avoid having unverified fetches cached and substituted for verified fetches, I added the attributes to unlockedAttributes as well as lockedAttributes. I'm still not certain on how the caching exactly works and it could be improved slightly by having verified fetches store a cache entry for unverified fetches as well, but I'm not sure it is worth the trouble.
publicKeys, as its supposted to unambiguously define a list of public keys, is a parsed as a json string. I did not implement it as a list since those can't be given directly as part of a flake uri. The json can, provided it is percent-encoded. The json string is also used for cache lookups.
It can be given to fetchGit and as flake input attribute in the form of a normal list of attributes, which is then turned into a json string by the flake or fetchTree parser. This is immediately reparsed to std::vector<publicKey> by the fetch-method in git.cc, which feels hacky to me, but I didn't find a better way to do this while still keeping the current seperation of concerns between flake.cc and git.cc.

To keep this PR small I consider the following out-of-scope

gpg signatures
fetchers other than git
tag signature verification (though it should be easy to implement this once verifyCommit is implemented)

Checklist for maintainers

Maintainers: tick if completed or explain if not relevant

agreed on idea
agreed on implementation strategy
tests, as appropriate
- functional tests - tests/**.sh
- unit tests - src/*/tests
- integration tests - tests/nixos/*
documentation in the manual
documentation in the internal API docs
code and comments are self-explanatory
commit message explains why the change was made
new feature or incompatible change: updated release notes

Priorities

Add 👍 to pull requests you find important.

tomberek · 2023-08-19T00:56:58Z

Thank you for the PR. This provides an avenue to have verified updates. As you noted, it might need to support multiple and other kinds of keys in practice.

flandweber · 2023-08-19T20:48:27Z

@tomberek Thanks!

it might need to support multiple and other kinds of keys in practice.

Do you have thoughts on how that should look like?

I'm not sure gpg-signatures are equally easily implementable so I'd rather stick to ssh keys for now and regard other fetchers and gpg signature verification (which in my opinion is undeniably desierable, esspecially for prebuild binaries) as out-of-scope.

fricklerhandwerk · 2023-10-09T12:42:54Z

Triaged in Nix team meeting:

We generally want to move in that direction
We should merge this as an experimental feature before committing

fricklerhandwerk

Thanks a lot @VwCSXg for this high-quality PR. A few superficial suggestions, but in essence this is solid. Global change requests:

Please wrap the new behavior in a new experimental feature (e.g. git-verify-signatures, feel free to find a suitable name) to leave freedom to iterate on the design before committing to it in the long term. Link to the experimental feature from the documentation of the new fetchGit parameters (check the manual for samples).
Please add a release note in rl-next.md linking to the builtin and the experimental feature.

Approving already, please ping maintainers when you're done with the fixups.

src/libexpr/flake/flake.cc

src/libexpr/primops/fetchTree.cc

src/libexpr/flake/flake.cc

src/libexpr/primops/fetchTree.cc

src/libfetchers/git.cc

tests/functional/fetchGitVerification.sh

fricklerhandwerk · 2023-10-10T14:43:05Z

src/libexpr/flake/flake.cc

+                            std::vector<nix::fetchers::PublicKey> publicKeys;
+                            parsePublicKeysList(state, attr, publicKeys);
+                            attrs.emplace(state.symbols[attr.name], publicKeys_to_string(publicKeys));


@VwCSXg and me are both not sure if this is a sustainable design to add parameters to the input this way, but there seems no obvious other way. @edolstra @roberth do you think it makes sense or do you have other ideas? Since we're guarding all of this by an experimental feature we can still adjust in the future, so no pressure to answer it now.

flandweber · 2023-10-15T08:21:57Z

I think this is ready now.

Approving already, please ping maintainers when you're done with the fixups.

@fricklerhandwerk
@Ericson2314
@roberth

fricklerhandwerk · 2023-10-20T01:05:09Z

I'll need another few days to get back to it due to other priorities. If I don't react by end of next week and no one else does the final pass, please ping me @bootrhetoric.

flandweber · 2023-11-02T19:04:55Z

If I don't react by end of next week and no one else does the final pass, please ping me @bootrhetoric.

@fricklerhandwerk

fricklerhandwerk

Committed the documentation nits myself. Very good, many thanks for the collaboration!

Ugh, there are merge conflicts now. Sorry, you'll have to take another close look since one upstream change interferes with yours.

src/libfetchers/git.cc

src/libexpr/primops/fetchTree.cc

This implements the git input attributes `verifyCommit`, `keytype`, `publicKey` and `publicKeys` as experimental feature `verified-fetches`. `publicKeys` should be a json string. This representation was chosen because all attributes must be of type bool, int or string so they can be included in flake uris (see definition of fetchers::Attr).

This adds publicKeys as an optional fetcher input attribute to flakes and builtins.fetchGit to provide a nix interface for the json-encoded `publicKeys` attribute of the git fetcher. Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>

This adds simple tests of the commit signature verification mechanism of fetchGit and its flake input wrapper. OpenSSH is added to the build dependencies since it's needed to create a key when testing the functionality. It is neither a built- nor a runtime dependency.

edolstra · 2023-11-06T09:39:09Z

Doesn't this block migrating to libgit2? In that case, I would be in favor of reverting this since using libgit2 is more of a priority than signature verification.

flandweber · 2023-11-06T10:17:30Z

Oh, I wasn't aware of your work on this, but I don't see why it would be a block:

nixos-discourse · 2024-01-29T17:50:40Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/git-verify-in-band-commit-verification/38991/1

nixos-discourse · 2024-08-17T10:13:39Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/bachelors-thesis-on-expression-update-security/50738/1

flandweber requested a review from edolstra as a code owner August 18, 2023 13:11

github-actions bot added with-tests Issues related to testing. PRs with tests have some priority fetching Networking with the outside (non-Nix) world, input locking labels Aug 18, 2023

flandweber marked this pull request as draft August 18, 2023 13:21

flandweber changed the title ~~WIP: Commit Signature Verification~~ WIP: SSH Commit Signature Verification Aug 19, 2023

flandweber force-pushed the flake-authentication branch 10 times, most recently from 4120542 to ab36171 Compare August 24, 2023 23:05

flandweber changed the title ~~WIP: SSH Commit Signature Verification~~ Commit Signature Verification Aug 24, 2023

flandweber marked this pull request as ready for review August 24, 2023 23:08

flandweber marked this pull request as draft August 24, 2023 23:14

flandweber force-pushed the flake-authentication branch from ab36171 to a26e77d Compare August 24, 2023 23:25

flandweber marked this pull request as ready for review August 24, 2023 23:26

roberth mentioned this pull request Aug 18, 2023

Diffing build outputs is not verification #8846

Open

2 tasks

fricklerhandwerk added feature Feature request or proposal idea approved The given proposal has been discussed and approved by the Nix team. An implementation is welcome. labels Oct 9, 2023

flandweber force-pushed the flake-authentication branch 2 times, most recently from fdaf84d to 9d8163d Compare October 10, 2023 10:00

fricklerhandwerk self-assigned this Oct 10, 2023

fricklerhandwerk approved these changes Oct 10, 2023

View reviewed changes

flandweber force-pushed the flake-authentication branch 5 times, most recently from b1c0e79 to 6c509e1 Compare October 12, 2023 19:55

flandweber requested a review from thufschmitt as a code owner October 12, 2023 19:55

flandweber force-pushed the flake-authentication branch 3 times, most recently from 3979a64 to 41cf52d Compare October 15, 2023 08:16

flandweber force-pushed the flake-authentication branch from 41cf52d to 53efc65 Compare October 20, 2023 19:18

fricklerhandwerk approved these changes Nov 2, 2023

View reviewed changes

flandweber and others added 3 commits November 3, 2023 20:15

flandweber force-pushed the flake-authentication branch from 34f9d8e to 2719327 Compare November 3, 2023 19:24

fricklerhandwerk enabled auto-merge November 3, 2023 22:49

fricklerhandwerk merged commit 8e222fb into NixOS:master Nov 3, 2023
8 checks passed

flandweber deleted the flake-authentication branch November 4, 2023 11:04

Ericson2314 mentioned this pull request Nov 10, 2023

Content addressing and adding to store cleanup #9325

Merged

flandweber mentioned this pull request Jul 27, 2024

fetchgit: add git signature verification NixOS/nixpkgs#330457

Open

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Commit Signature Verification #8848

Commit Signature Verification #8848

flandweber commented Aug 18, 2023 •

edited

Loading

tomberek commented Aug 19, 2023

flandweber commented Aug 19, 2023 •

edited

Loading

fricklerhandwerk commented Oct 9, 2023

fricklerhandwerk left a comment •

edited

Loading

fricklerhandwerk Oct 10, 2023

flandweber commented Oct 15, 2023

fricklerhandwerk commented Oct 20, 2023

flandweber commented Nov 2, 2023

fricklerhandwerk left a comment •

edited

Loading

edolstra commented Nov 6, 2023

flandweber commented Nov 6, 2023 •

edited

Loading

nixos-discourse commented Jan 29, 2024

nixos-discourse commented Aug 17, 2024

Commit Signature Verification #8848

Commit Signature Verification #8848

Conversation

flandweber commented Aug 18, 2023 • edited Loading

Motivation

Context

Implementation

Checklist for maintainers

Priorities

tomberek commented Aug 19, 2023

flandweber commented Aug 19, 2023 • edited Loading

fricklerhandwerk commented Oct 9, 2023

fricklerhandwerk left a comment • edited Loading

Choose a reason for hiding this comment

fricklerhandwerk Oct 10, 2023

Choose a reason for hiding this comment

flandweber commented Oct 15, 2023

fricklerhandwerk commented Oct 20, 2023

flandweber commented Nov 2, 2023

fricklerhandwerk left a comment • edited Loading

Choose a reason for hiding this comment

edolstra commented Nov 6, 2023

flandweber commented Nov 6, 2023 • edited Loading

nixos-discourse commented Jan 29, 2024

nixos-discourse commented Aug 17, 2024

flandweber commented Aug 18, 2023 •

edited

Loading

flandweber commented Aug 19, 2023 •

edited

Loading

fricklerhandwerk left a comment •

edited

Loading

fricklerhandwerk left a comment •

edited

Loading

flandweber commented Nov 6, 2023 •

edited

Loading