[yshui]

Motivation

Avoid unnecessary store -> store copies

Context

See #11228

I checked the PR introduced this change, and didn't find an explanation why this == "source" check is necessary.

IMO all store -> store copies are unnecessary. And even if there is indeed a valid reason this is only done for store paths named "source", it's not like the user can't just name their derivation "source".

Priorities and Process

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

[edolstra]

I think this would violate the postcondition that the resulting store path is named source. The store path should only depend on the contents of the input path. E.g. path:/foo and path:/nix/store/<hash>-xyzzy should result in the same store path, if /foo and /nix/store/blabla have the same contents. But this change would cause the first to return a store path like /nix/store/<hash'>-source and the latter /nix/store/<hash>-xyzzy.

[yshui]

hmm, i see your point. but nix store paths aren't content addressable to begin with (unless ca-derivation is used), is there really a benefit to treat input store paths differently? (or is there anything in nix/nixpkgs currently depends on this behavior?)

(and besides, i can still bypass all this by naming my derivation "source")

[Ericson2314]

They are content addressed to begin with without that feature, just not ones produced by derivations.

[yshui]

@Ericson2314 is there a real way to tell them apart? is it possible to tell apart store paths that are actually content-addressable, vs store paths that just happen to be named "source"?

[yshui]

I have observed assertion failure from nix if I name my derivation "source" (without this patch). I think this is poorly thought out.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-08-07-nix-team-meeting-minutes-167/50287/1

[roberth]

Based on today's meeting, returning a path named source may become more relevant as part of a different solution that avoids copying in other cases or potentially more cases, so we're holding off on a decision on this PR for now.

[yshui]

would naming a derivation "source" become prohibited in the future?

[Ericson2314]

@yshui No, I don't think we would need to prohibit that.

[roberth]

The reason to hold off is that we may need to revert this change in behavior soon, so that we don't have to track the custom name in the lazy path value returned by fetchTree.

would naming a derivation "source" become prohibited in the future?

No, I don't think we would need to prohibit that.

I agree and in fact it is desirable when performing file set filtering in a content-addressed derivation. If that derivation is also named source, and the contents are the same, it means that a flake-based derivation and a Nixpkgs derivation could potentially produce derivations with identical output hashes or identical realisations, avoiding a duplicate build.

• Make output paths/hashes indifferent to possible CA/FOD origin of sources #9259
• lib.fileset: Function for transforming store path contents nixpkgs#264541

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/confused-about-nix-copying-nixpkgs/53737/4

[roberth]

We've released a fix for fetches

• create cache entry for paths already in the nix store #12911 (comment)

We also see opportunities to remove copying for inputs that originate from store paths, as we are growing the ability to internally "mount" lazily loaded file system contents.

• To illustrate: Source path error improvements #12915

Changing the semantics of fetching as proposed in this PR does not seem like a viable solution though, so I'll close this one now.
Nonetheless, thank you for engaging with the problem.