-
-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Derivations can output "text-hashed" data #3959
Conversation
I'd like to learn what are the possible use-cases once this feature becomes truly useful and complete (meaning that the drawback you mentioned earlier is solved). This feels very confusing to me as a casual Nix user |
@kisik21 the use-case is instead of using IFD, we can build derivations produced by derivations. For example, we could do |
Oh, I see! So the extra step of running node2nix or whatever gets used to package up node packages, or something else, is not needed with derivation-ception that's being made possible here. Or should we call it Nix-ception? 😂 |
In particular, this means that derivations can output derivations. But that ramification isn't (yet!) useful as we would want, since there is no way to have a dependent derivation that is itself a dependent derivation.
2c3be0b
to
a4e5de1
Compare
This should pass, and is rebased to now only contain one other PR. |
c17f78e
to
90d76fa
Compare
Need to get experiment features to daemon like with the other tests.
Good to round out the library interface.
In other words, use a plain `ContentAddress` not `ContentAddressWithReferences` for `DerivationOutput::CAFixed`. Supporting fixed output derivations with (fixed) references would be a cool feature, but it is out of scope at this moment.
In this one, we don't just output an existing derivation as is, but modify it first.
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
Thanks so much! Co-authored-by: Adam Joseph <54836058+amjoseph-nixpkgs@users.noreply.github.com>
nix path-info $drv --derivation --json | jq | ||
nix path-info $out1 --derivation --json | jq | ||
|
||
test $out1 == $drv |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't suppose we can do nix build $drvProducingDrv^out^out
yet?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
:) not yet, but soon!
ead1411
to
2a37fef
Compare
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hardly anything significant it seems. 🚀
Didn't mean to use the private name that shouldn't be exposed.
It appears we were checking a variable in the process of definining it.
Thanks! Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
The `ContentAddressWithReferences` method is made total, with error handling now squarely the caller's job. This is better.
This causes a warning with gcc:
Any idea what causes that? |
Motivation
We're finally getting to RFC 92 features proper!
Derivations are currently serialized to store objects (and also thus given drv paths, even when we just stick with in-memory derivations) using "text hashing" / "text file ingestion method", a type of content addressing. That means, if we want to compute derivations we need to use
The latter part is what we are adding.
Context
A few things needed to make this go.
A little cleanup of
ContentAddressMethod
building on IntroduceStoreReferences
andContentAddressWithReferences
#3746. The two variants are nowTextIngestionMethod
andFileIngestionMethod
, which together are tantamount to a 3-way enum:This data type no longer species any hash type, but is just the way of ingestion data. A nice single purpose now.
TextHashMethod
->TextIngestionMethod
, since it no longer implies SHA-256 or any other hash type (or hashing at all!), though out of an abundance of caution we disallow trying to use any other hash type with it. (We're separating concerns but not supporting all the options yet!)FixedOutputHashMethod
is deleted. It was a type for a (hash type, flat | recursive) pair, but we simply don't need that anymore. Everything supports text hashing too now.The new stuff is gated under a new
dynamic-derivations
feature. I imagine I'll just put all the constituent features that make up dynamic derivations under this, as they are meant to be used together.Checklist for maintainers
Maintainers: tick if completed or explain if not relevant
tests/**.sh
src/*/tests
tests/nixos/*
Priorities
Add 👍 to pull requests you find important.
In particular, this means that derivations can output derivations. But that ramification isn't (yet!) useful as we would want, since there is no way to have a dependent derivation that is itself a dependent derivation.
depends on #3746