Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add nix query-substituters command #4733

Closed
wants to merge 12 commits into from

Conversation

matthewbauer
Copy link
Member

@matthewbauer matthewbauer commented Apr 22, 2021

nix query-substituters is used to figure out how much of a flake's packages are
in a substituter. This allows us to get an idea for how much we'll
have to build locally. It can be used with Nixpkgs, but also
individual projects to determine objectively how much is missing from
the cache.

Example usage is:

$ nix query-substituters nixpkgs --substituters https://cache.nixos.org

Note that this is using derivations instead of output paths to
determine what depends on each path. This is necessary since we don't
know what dependencies a given store path will have unless it's built.

/cc @edolstra @domenkozar @vcunat @masaeedu

Fixes #3752

When no attr path is specified (like when using -f ... arg),
findAlongAttrPath will not autocall the expression. This is needed
when you have a .nix file that has arguments. This should make
searches like:

$ nix search -f ...

work correctly.
nix weather is used to figure out how much of a flake's packages are
in a substituter. This allows us to get an idea for how much we'll
have to build locally. It can be used with Nixpkgs, but also
individual projects to determine objectively how much is missing from
the cache.

Example usage is:

$ nix weather nixpkgs --substituters https://cache.nixos.org

Note that this is using derivations instead of output paths to
determine what depends on each path. This is necessary since we don't
know what dependencies a given store path will have unless it's built.

/cc @edolstra @domenkozar @vcunat @masaeedu

Fixes NixOS#3752
@grahamc
Copy link
Member

grahamc commented Apr 22, 2021

I wonder if this should respect WantMassQuery?

@matthewbauer
Copy link
Member Author

I wonder if this should respect WantMassQuery?

Yeah that sounds good. I've updated with a check for that.

@edolstra
Copy link
Member

What does "weather" mean in this context? I have trouble connecting it with what the command does.

uint64_t narSize = 0;
std::optional<uint64_t> downloadSize = 0;

for (auto & path : validPaths) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks very sequential so it will be latency-bound. Maybe you can use queryMissing instead? It should be a lot faster. Or use the async version of queryPathInfo.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

queryValidPaths should populate the cache up front. These queryPathInfo should be just cache lookups. I supposed some caches may not have cached backing them up? In that case, it would be very slow.

src/nix/weather.cc Outdated Show resolved Hide resolved
@domenkozar
Copy link
Member

The naming weather is clever, but not clear to the user.

How about nix query-substituters or something similar?

@andir
Copy link
Member

andir commented Apr 23, 2021

Could this also get a machine readable output so we can script around it?

@matthewbauer matthewbauer changed the title Add nix weather command Add nix query-substituters command Apr 23, 2021
@stale
Copy link

stale bot commented Oct 22, 2021

I marked this as stale due to inactivity. → More info

@stale stale bot added the stale label Oct 22, 2021
@Ericson2314 Ericson2314 added the UX The way in which users interact with Nix. Higher level than UI. label Jun 14, 2023
@stale stale bot removed the stale label Jun 14, 2023
@Ericson2314 Ericson2314 added new-cli Relating to the "nix" command stale labels Jun 14, 2023
@stale stale bot removed the stale label Jun 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new-cli Relating to the "nix" command UX The way in which users interact with Nix. Higher level than UI.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add "nix weather" command that is equivalent to "guix weather"
6 participants