Emacs load times may be slow due to explosion in load-path (subdirs.el)

[knazarov]

I've been investigating why my Emacs loads slower when installed through emacs-overlay, compared to a regular .emacs.d. And I've figured out that there's an issue with how some packages play together with site-lisp/.

Under normal circumstances, people don't install third-party packages to site-lisp/, but instead use something like use-package which installs packages to their home directory. With system-wide site-lisp/ there's a peculiar problem when Emacs would scan for all subdirectories of every package and add them to load-path. Some packages contain a ton of subdirs (example - evil-collection, for which I've opened an issue here).

When a lot of directories get added to load-path, it slows down Emacs startup, as each (require ...) call now needs an O(n) scan through everything in load-path to find the library. In my case it adds on the order of 400 milliseconds for just evil-collection package alone.

The fix is usually to add an empty file called .nosearch to the subdirs of the module. For the curious, you can read elisp reference (search for subdirs.el)

I'm opening this ticket in order for people to be aware that there may be other cases when a combination of how emacs-overlay works and the packages in elpa/melpa can interact. If you find such cases -- go ahead and submit a PR to the upstream package.