[WIP] Add options to improve development with linked modules #3753
Conversation
Creates a symlink for each file/directory inside a package instead of the directory itself. This has the effect of excluding node_modules (and other files excluded by filters).
This will automatically link dependencies to the globally linked copy when installing. Avoids having to manually setup links when setting up a project locally.
|
One current issue is with the way the node require resolution algorithm works. Dependencies of symlinked modules are resolved relative to the realpath, not the symlink. This means that you'll still get duplicates if both modules depend on a third dependency, or errors if that dependency is not installed in either place. One option to solve this is to use hard links instead of soft links. Hard links are not without their problems though. One is that you can't hard link directories on some systems I think (works on OS X, but I don't think on Linux?). Another is the recently added runtime option for node --preserve-symlinks, which skips getting the realpath when resolving modules. Something similar would need to be added to browserify/webpack to solve this there as well. Not sure that's super great as it relies on users enabling an option for linked modules to work correctly. |
|
Made a pull to add support for |
|
I think you might be interested by the workspace (two other parts can be found in the same repository) proposal, that is currently implemented behind a flag. |
|
@arcanis yep, workspaces are cool, but serve a different purpose I think. See my note about it in the PR description. |
|
Hey, @devongovett, great job at looking into this! My concern is that yarn may have too many ways to link projects and we need to make an effort to consolidate a few. Do you want maybe to take over |
|
@bestander sounds like a good idea. I'll take a look next week when I'm back from vacation! |
|
Updated the knit RFC here: yarnpkg/rfcs#73 |
# Conflicts: # src/cli/commands/link.js
|
@devongovett what is the current status of your work and this PR? Shall we close it or try to help you get it merged? |
|
Hi! I'm really grateful for the work you put in this PR, but unfortunately the project moved forward since you submitted it and I fear it can be merged in this current state. I'm going to close it, but please feel free to check if the problem still exists, and to rebase your work on top of the current master and open a new PR if it's the case. Again, sorry we didn't got around merging it sooner! |
Summary
Currently, setting up and maintaining links between many modules in local development is very painful. Not only do you have to manually link modules together, but once you do you can easily end up with many duplicate dependencies. This leads to errors e.g. when multiple copies of react are loaded if several linked dependencies also depend on react. A workaround is to install react globally and link to that from every place that uses it, but doing this everywhere is painful.
This pull introduces two options to help with these problems:
yarn link --deep- Instead of linking the global copy to the current directory itself, this creates a link for every file/directory inside the current directory excluding ignored files (most importantlynode_modules). This means that when you install a linked module, the dependencies are not duplicated (see 2). It does mean that if you add a file in the root of the linked module, you must runyarn link --deepagain to add a link to it globally, however. This is kinda like the knit proposal.yarn install --link- This automatically sets up links to all dependencies that are linked globally and installs their dependencies. This means that once you've setup global links for all of the dependencies in your project withyarn link(e.g. after cloning from git), you can just run this one command to setup all of the links and install all normal dependencies in one go. If those links are produced with the--deepoption, dependencies will be fully deduped.As mentioned above, this is similar to the
knitproposal. (1) is pretty much the same idea as plainyarn knit. (2) is different in that it automatically installs all linked dependencies rather than needing to manually callyarn knit depfor each one. I think this is important to enable easy setup of projects using many linked modules. However, I should note that you can also callyarn add dep --linkas well, which is more similar toyarn knit dep.I know workspaces are also being worked on at the moment, however I think this solves a different problem. Where workspaces are great for a tree of related modules (e.g. in a monorepo), the options in this pull are for linking together modules in separate trees, e.g. something that might be shared between multiple workspaces.
Test plan
If this is something the yarn community is interested in including, I'll clean up this PR, add tests and such. So far I'm just playing around, but it seems to work for my needs.