Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fs: add recursive watch for linux #45098

Merged
merged 38 commits into from Oct 31, 2022
Merged
Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
cdc9349
fs: add recursive watch to linux
anonrig Oct 19, 2022
296646b
fs: replace traverse with readdir for performance
anonrig Oct 21, 2022
77cb9a0
fs: fix linter issues
anonrig Oct 22, 2022
5b72449
fs: move linux watcher to internal/fs/watch
anonrig Oct 22, 2022
87866c3
test: add missing clear interval for linux
anonrig Oct 22, 2022
00ab12d
fs: add promise support to linux watcher
anonrig Oct 22, 2022
cb4b455
test: simplify start and update tests
anonrig Oct 22, 2022
4144949
fs: avoid prototype pollution
anonrig Oct 22, 2022
9a86410
test: improve fs.watch tests
anonrig Oct 23, 2022
8b36541
test: handle edge cases
anonrig Oct 23, 2022
89abb18
fs: handle more linux edge cases
anonrig Oct 23, 2022
f15251f
fs: update requested changes
anonrig Oct 23, 2022
cd240a6
fs: fix circular dependency
anonrig Oct 24, 2022
6311cf5
fs: improve tests
anonrig Oct 24, 2022
d8d2a0c
test: add url as parameter for fs.watch
anonrig Oct 24, 2022
632a4bc
test: update lint errors
anonrig Oct 25, 2022
df610ef
test: fix url error
anonrig Oct 25, 2022
8243dde
test: improve test-fs-watch-recursive.js
anonrig Oct 25, 2022
37a0839
fs: use arrays instead of sets
anonrig Oct 25, 2022
c7512ef
fs: remove lazy loading assert
anonrig Oct 25, 2022
6eee878
fs: revert certain changes
anonrig Oct 26, 2022
a6906f2
fs: add recursive validation to linux watcher
anonrig Oct 26, 2022
6061330
test: improve tests
anonrig Oct 26, 2022
9d596e8
fs: do not throw abort errors
anonrig Oct 26, 2022
d94f365
fs: rename recursive watch
anonrig Oct 26, 2022
b5161e5
fs: adjust implementation
anonrig Oct 26, 2022
6e4299d
fs: add ref and unref to fs.watch
anonrig Oct 26, 2022
2e5d4b3
fs: update async iterator implementation
anonrig Oct 27, 2022
cd3199b
fs: improve watcher
anonrig Oct 28, 2022
d69a565
fs: rename linux watcher
anonrig Oct 29, 2022
28ee387
fs: add support for symlinks
anonrig Oct 29, 2022
8e4e3dd
test: remove redundant test
anonrig Oct 29, 2022
e6019be
fs: rename to nonNativeWatcher
anonrig Oct 29, 2022
1e903a2
fs: update comments
anonrig Oct 30, 2022
b8b87f6
test: update symlink error message
anonrig Oct 30, 2022
ab4f1d2
fs: update implementation
anonrig Oct 30, 2022
c11bb62
test: make fs.watch errors more strict
anonrig Oct 31, 2022
bddb83f
fs: add documentation
anonrig Oct 31, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 0 additions & 4 deletions doc/api/fs.md
Expand Up @@ -4377,10 +4377,6 @@ the returned {fs.FSWatcher}.
The `fs.watch` API is not 100% consistent across platforms, and is
unavailable in some situations.
Comment on lines 4380 to 4381
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it still true that it is unavailable in some situations?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IBMi and AIX does not fully support fs.watch

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a task for later (follow on PR) it would be good to explain a bit more what "situations" it is unavailable.


The recursive option is only supported on macOS and Windows.
An `ERR_FEATURE_UNAVAILABLE_ON_PLATFORM` exception will be thrown
when the option is used on a platform that does not support it.

anonrig marked this conversation as resolved.
Show resolved Hide resolved
On Windows, no events will be emitted if the watched directory is moved or
renamed. An `EPERM` error is reported when the watched directory is deleted.

Expand Down
26 changes: 17 additions & 9 deletions lib/fs.js
Expand Up @@ -57,6 +57,7 @@ const {

const pathModule = require('path');
const { isArrayBufferView } = require('internal/util/types');
const nonNativeWatcher = require('internal/fs/recursive_watch');

// We need to get the statValues from the binding at the callsite since
// it's re-initialized after deserialization.
Expand All @@ -68,7 +69,6 @@ const {
codes: {
ERR_FS_FILE_TOO_LARGE,
ERR_INVALID_ARG_VALUE,
ERR_FEATURE_UNAVAILABLE_ON_PLATFORM,
},
AbortError,
uvErrmapGet,
Expand Down Expand Up @@ -163,7 +163,6 @@ let FileWriteStream;
const isWindows = process.platform === 'win32';
const isOSX = process.platform === 'darwin';
anonrig marked this conversation as resolved.
Show resolved Hide resolved


function showTruncateDeprecation() {
if (truncateWarn) {
process.emitWarning(
Expand Down Expand Up @@ -2297,13 +2296,22 @@ function watch(filename, options, listener) {

if (options.persistent === undefined) options.persistent = true;
if (options.recursive === undefined) options.recursive = false;
if (options.recursive && !(isOSX || isWindows))
throw new ERR_FEATURE_UNAVAILABLE_ON_PLATFORM('watch recursively');
const watcher = new watchers.FSWatcher();
watcher[watchers.kFSWatchStart](filename,
options.persistent,
options.recursive,
options.encoding);

let watcher;

// TODO(anonrig): Remove this when/if libuv supports it.
anonrig marked this conversation as resolved.
Show resolved Hide resolved
// libuv does not support recursive file watch on all platforms, e.g. Linux due to
// the limitations of inotify.
anonrig marked this conversation as resolved.
Show resolved Hide resolved
if (options.recursive && !isOSX && !isWindows) {
watcher = new nonNativeWatcher.FSWatcher(options);
watcher[watchers.kFSWatchStart](filename);
} else {
watcher = new watchers.FSWatcher();
watcher[watchers.kFSWatchStart](filename,
options.persistent,
options.recursive,
options.encoding);
}

if (listener) {
watcher.addListener('change', listener);
Expand Down
26 changes: 24 additions & 2 deletions lib/internal/fs/promises.js
Expand Up @@ -88,7 +88,8 @@ const {
} = require('internal/util');
const { EventEmitterMixin } = require('internal/event_target');
const { StringDecoder } = require('string_decoder');
const { watch } = require('internal/fs/watchers');
const { kFSWatchStart, watch } = require('internal/fs/watchers');
const nonNativeWatcher = require('internal/fs/recursive_watch');
const { isIterable } = require('internal/streams/utils');
const assert = require('internal/assert');

Expand Down Expand Up @@ -120,6 +121,7 @@ const getDirectoryEntriesPromise = promisify(getDirents);
const validateRmOptionsPromise = promisify(validateRmOptions);

const isWindows = process.platform === 'win32';
const isOSX = process.platform === 'darwin';

let cpPromises;
function lazyLoadCpPromises() {
Expand Down Expand Up @@ -917,6 +919,26 @@ async function readFile(path, options) {
return handleFdClose(readFileHandle(fd, options), fd.close);
}

async function* _watch(filename, options = kEmptyObject) {
validateObject(options, 'options');

if (options.recursive != null) {
validateBoolean(options.recursive, 'options.recursive');

// TODO(anonrig): Remove this when/if libuv supports it.
// libuv does not support recursive file watch on all platforms, e.g. Linux due to
// the limitations of inotify.
anonrig marked this conversation as resolved.
Show resolved Hide resolved
if (options.recursive && !isOSX && !isWindows) {
const watcher = new nonNativeWatcher.FSWatcher(options);
await watcher[kFSWatchStart](filename);
yield* watcher;
return;
}
}

yield* watch(filename, options);
}

module.exports = {
exports: {
access,
Expand Down Expand Up @@ -947,7 +969,7 @@ module.exports = {
writeFile,
appendFile,
readFile,
watch,
watch: !isOSX && !isWindows ? _watch : watch,
constants,
},

Expand Down