Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

inotify recursive scan is not supported with event_filter that does not include IN_CREATE #990

Open
MPalarya opened this issue May 22, 2023 · 1 comment

Comments

@MPalarya
Copy link

MPalarya commented May 22, 2023

We're using the latest master which is 3.0.0 release with the addition of 48c49a1

We're only interested in move and delete events, specifically of files, but we'd like to watch for files inside subdirs as well, so we're defining it like this:

observer.schedule(
    event_handler=our_handler,
    path=our_path,
    recursive=True,
    event_filter=[FileMovedEvent, FileDeletedEvent],
)

I understand that inotify does not support recursive scanning,
and based on the implementation here:

        def _recursive_simulate(src_path):
            events = []
            for root, dirnames, filenames in os.walk(src_path):
                for dirname in dirnames:
                    try:
                        full_path = os.path.join(root, dirname)
                        wd_dir = self._add_watch(full_path, self._event_mask)
                        e = InotifyEvent(
                            wd_dir,
                            InotifyConstants.IN_CREATE | InotifyConstants.IN_ISDIR,
                            0,
                            dirname,
                            full_path,
                        )
                        events.append(e)
                    except OSError:
                        pass
                for filename in filenames:
                    full_path = os.path.join(root, filename)
                    wd_parent_dir = self._wd_for_path[os.path.dirname(full_path)]
                    e = InotifyEvent(
                        wd_parent_dir,
                        InotifyConstants.IN_CREATE,
                        0,
                        filename,
                        full_path,
                    )
                    events.append(e)
            return events

the approach seems to be to add the WDs when either a file is CREATEd or a directory fires its own event (create, modify, whatever)

so in order to achieve what we're looking for, we need to add DirCreatedEvent in addition to the two events that we're after.
which means more CPU and RAM, slower performance and noisy debug logs

how can this be improved?
I would expect that when selecting recursive=True internally, we will implicitly add IN_ISDIR without IN_CREATE to reduce the noise of all file creations, and still identify new directories created to add watches

@MPalarya
Copy link
Author

cc @marioga & @BoboTiG

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant