Copying a file onto ceph-fuse fails when the source is read-only

[Varstahl]

• Version: 1.44.2
• Platform: Linux 5.10.0-21-cloud-amd64 SMP Debian 5.10.162-1 (2023-01-21) x86_64 GNU/Linux
• Subsystem: fs

Description

Trying to copy a read-only file onto a ceph-fuse filesystem fails, currently unknown if it affects CephFS itself. This is an outstanding issue introduced with 278cfa0 in August 2020, and has been afflicting releases since Node v12.19.0.

Replication

Using Node the simplest way to reproduce the error is the following one-liner, where /tmp/foo has 0444 mode:

require("fs").copyFileSync("/tmp/foo", "/ceph-mount/tmp/bar");

Details

When uv__fs_copyfile() is called, while opening the destination file it creates it with the mode of the source file. Since it can't be opened exclusively, after some checks it attempts to truncate it:

libuv/src/unix/fs.c

Lines 1306 to 1310 in 9581e3d

	/* Truncate the file in case the destination already existed. */
	if (ftruncate(dstfd, 0) != 0) {
	err = UV__ERR(errno);
	goto out;
	}

When ftruncate() is called on a read-only file on CephFS though, it returns an EACCES error, which then stops the file copying. This can be solved in 3 ways:

1. add +w while opening the file to ensure that the error doesn't occur:

     /* Open the destination file. */
     dstfd = uv_fs_open(NULL,
                        &fs_req,
                        req->new_path,
                        dst_flags,
                        src_statsbuf.st_mode | 0222,
                        NULL);

   After the truncate the correct mode is re-set by fchmod() in the next instruction:
   

   libuv/src/unix/fs.c

   Line 1313 in 9581e3d

   if (fchmod(dstfd, src_statsbuf.st_mode) == -1) {

2. Verify if the file is new, in which case don't truncate:

       /* Truncate the file in case the destination already existed. */
       if ((dst_statsbuf.st_size > 0) && (ftruncate(dstfd, 0) != 0)) {

3. Manage the exception, if it happens.

I assumed that ftruncate() was always called, even on new files, to reduce the potential for races when opened non exclusively, so I chose option 3 and wrote a tiny exception handler in the style of the surrounding code to make sure to adhere to the standard. PR incoming in a few moments.

I'm also opening an issue on the Ceph tracker, but I think that using the check to guard against those errors would prove beneficial, and hopefully be readily available as soon as Node v20.

References

• fs.copyFile fails with permission denied when source is read-only nodejs/node#37284
• fs.copyFile fails with permission denied when source is read-only #3117
• fs.copyFile fails with permission denied when source is read-only #3322
• https://tracker.ceph.com/issues/58962

[vtjnash]

This sounds like a bug in the ceph kernel driver. I am not sure it is up to libuv to address driver bugs? EACCES is not specified as a valid error to be returned from that syscall (https://pubs.opengroup.org/onlinepubs/000095399/functions/ftruncate.html)

[Varstahl]

Not sure myself, I provided the fix as we originally believed it was an error with the file_copy_range and then discovered the actual problem while debugging. Also I seem to recall libuv has worked around bugged Ceph implementations in the past, so that's another reason why I provided a fix, although I might be mistaking.

I'm also bringing this up to the Ceph tracker as soon as my account gets activated, which if resolved properly could also fix other problems that the PR i provided can't work around, such as a file manually created as read-only and then truncated with fs.truncateSync(). Since that calls directly ftruncate() it would require changes that are really out of scope, I think.

[vtjnash]

If it was created read-only already, then the required open permissions (O_WRONLY | O_CREAT) should have prevented it from getting overwritten anyways?

[Varstahl]

I think I didn't explain myself properly, sorry, I meant manual opening and manually trimming in sequence, like:

var fd = fs.openSync('file', 'w', 0o400);
fs.truncateSync(fd, 0);

Although this code doesn't make sense, it would incur in the same issue. That's all I meant.