Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Factor out MemorySourceAccessor, implement missing features #9283

Merged
merged 1 commit into from
Nov 6, 2023

Conversation

Ericson2314
Copy link
Member

Motivation

The new MemorySourceAccessor rather than being a slightly lossy flat map is a complete in-memory model of file system objects.

Context

I need this to write more unit tests for #8918

Priorities

Add 👍 to pull requests you find important.

@github-actions github-actions bot added the fetching Networking with the outside (non-Nix) world, input locking label Nov 3, 2023

namespace nix {

struct MemoryInputAccessorImpl : MemoryInputAccessor
{
std::map<CanonPath, std::string> files;
MemorySourceAccessor a;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we just make MemoryInputAccessorImpl a subclass of MemorySourceAccessor and InputAccessor? Then we don't need all these forwarding methods.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@edolstra I we'll need to make the base SourceAccessors virtual but then yes we can.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK it's done!

src/libutil/memory-source-accessor.cc Outdated Show resolved Hide resolved
The new `MemorySourceAccessor` rather than being a slightly lossy flat
map is a complete in-memory model of file system objects.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
@Ericson2314 Ericson2314 marked this pull request as ready for review November 4, 2023 19:30
Copy link
Member

@roberth roberth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs symlink support for this to be a proper accessor. If that's not a priority, the flaw can be documented while this is moved into tests where you want to use it.

return r->contents;
else
throw Error("file '%s' is not a regular file", path);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No symlink support.

Maybe the source accessor virtual methods should not even be responsible for that behavior. Or at least, symlink resolution should be implemented once, as a mix-in or an adapter.

(Also applies to other methods)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I actually think this is right and its PosixSourceAccessor that needs to be fixed.

Copy link
Member Author

@Ericson2314 Ericson2314 Nov 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#9306 partially fixes PosixSourceAccesor and documents this in the methods.

@fricklerhandwerk
Copy link
Contributor

fricklerhandwerk commented Nov 6, 2023

Triaged in Nix team meeting:

Copy link
Member

@roberth roberth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A suggestion; otherwise lgtm.

* `MemorySourceAccessor`, this has a side benefit of nicely
* defining what a "file system object" is in Nix.
*/
struct File {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps:

Suggested change
struct File {
struct FileSystemObject {

I don't like the idea that a directory would be a file.

@roberth roberth merged commit 1fa87c6 into NixOS:master Nov 6, 2023
8 checks passed
@Ericson2314 Ericson2314 deleted the mememory-source-accessor branch November 6, 2023 18:51
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-11-06-nix-team-meeting-minutes-101/35247/1

@Ericson2314 Ericson2314 added this to the git-hashing stabilisation milestone Mar 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fetching Networking with the outside (non-Nix) world, input locking
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

5 participants