New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Nix assumes file size reported by fstat is accurate when copying files to the Nix store #10667
Comments
This is hard to support because the NAR file format puts the size of the file before its contents. So if we can't rely on the file size reported by Related: #10019 |
I see. Here's a few thoughts I have on how this could theoretically be solved, note that I don't really know how feasible these are given that I'm not familiar with how Nix works internally:
Some things to consider:
|
Triaged in Nix team meeting:
|
This issue has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/2024-05-15-nix-team-meeting-minutes-146/45491/1 |
Describe the bug
When a file is copied to the Nix store from a local path, and an
fstat
on that file returns a size of 0, an empty file will end up in the Nix store even though the file may yield more data when read.This mismatch between actual file size and reported file size is considered absolutely legal under Linux (citation needed). Nix should handle it correctly.
Steps To Reproduce
The filesystem I have used to reproduce this is my own FUSE3-based filesystem, confgenfs (which behaves like this because another implementation is not possible - files may be different every time they're read and are generated on-demand). I also recall observing this behavior with other (typically FUSE-based) filesystems, but did not find another one while writing this issue.
Expected behavior
The file is read until the
read
(or whatever syscall) returns a length of 0 instead of stopping at the length returned byfstat
, causing the file to be correctly copied to the store.nix-env --version
outputnix-env (Nix) 2.22.0
Additional context
Nix will actually
open
the file as reported by the filesystem in this case, but will not read all data from it.Priorities
Add 👍 to issues you find important.
The text was updated successfully, but these errors were encountered: