Proposal for loading plugins relative to the configs that depend on them

[not-an-aardvark]

(Previous discussions include: #3458, #10125)

Background/Problem description

Currently, ESLint plugins and shareable configs are loaded from the location of the ESLint package, rather than the location of the config file where the plugins and configs are referenced. This leads to several problems:

• The current behavior requires end users to manually install any plugins required by a shareable config. As a result, a shareable config can't add a plugin without requiring new manual installation steps from the end user. This greatly degrades the ergonomics of using custom rules from plugins in shareable configs, and results in increased pressure to add new rules and options to ESLint core.
• The current behavior assumes that if a user installs a config/plugin and ESLint in the same project, then ESLint will be able to load that config/plugin. This relies on an implementation detail of how npm works rather than a specified behavior, which leads to problems when using other package management strategies, e.g. with lerna. (More details about this problem can be found in #10125.)
• The current behavior leads to a large amount of confusion from users where ESLint behaves differently depending on whether it's installed "globally" with npm.

To address these issues, many users have proposed loading all configs and plugins from the location of the config that depends on them (e.g. see #3458). For example, if a config has a field like plugins: react, ESLint would load eslint-plugin-react from the location of the config file itself. This would be beneficial because it would allow the author of a shareable config to control its dependencies, rather than the end user. It would also have a side-effect of removing the behavior change between local and global ESLint installations, which is a frequent source of confusion for users.

Unfortunately, implementing this scheme as-is would cause naming ambiguity. There could be two shareable configs which have dependencies on two different plugins that both happen to be called react (or they depend on two different versions of a plugin called react). If the end user depended on these two shareable configs and also configured a rule like react/some-rule in their top-level config, the end user's config would be ambiguous because it wouldn't be clear which react plugin they were referring to. Since the configurations for a given rule might be incompatible across different versions of a plugin, this could make it impossible to set the configuration for a particular rule or to override a configuration which was set by a shareable config. This would be an unacceptably poor user experience.

Another way to state the problem is that ESLint's mechanism for naming rules in a config file (plugin-name/rule-name) is fundamentally unable to disambiguate two plugins that have the same name. Currently, there are no naming conflicts because all plugins are loaded from the location of the eslint package, so plugins effectively live in a global namespace. If plugins could be loaded as dependencies of shareable configs, then naming conflicts would start to become a problem. As a result, we need to be able to support having two plugins with the same name before we can support having plugins as dependencies of configs.

Design goals of solution

• A config author should be able to add or upgrade any plugin in their config, without requiring additional installation steps from end users that extend that config.
• The end user should maintain the ability to override any configuration setting inherited from an extended config.
• A config author should be able to extend any two other configs at the same time, and have ESLint lint their code successfully. (The two configs might advocate mutually-incompatible code styles, but that issue is out of scope for this proposal.)
• ESLint's config-loading behavior should be compatible with the use of any package manager that follows the de-facto package.json spec, without relying on the implementation details of any particular package manager.
• Standalone config files should continue to be usable as shareable configs, and vice versa, without any changes.
• Shareable configs which currently have plugins as peerDependencies should be able to transition to the new solution without requiring changes to the configs of their users (or at least the vast majority of their users).
• The vast majority of existing configs in local-installation setups should continue to work with the new solution.

Summary of proposed solution

Since the problem stems from a limitation of ESLint's naming scheme, I think the most straightforward solution to the problem would involve changing the way that rules are named in a config file. Specifically, this solution proposes using hierarchical naming for rules, where rules would be configured with something that looks like a path. (In other words, a user could refer to a rule like foo::react/some-rule for the version of eslint-plugin-react used by eslint-config-foo, and this would be a different rule than bar::react/some-rule, which would refer to the version of eslint-plugin-react used by eslint-config-bar.)

This is similar to a few other proposals discussed in #3458, but I've done a significant amount of work since then on precisely describing the behavior, figuring out the edge cases, and thinking through alternatives.

Details of proposed solution

When describing how rule name resolution works in this proposal, it's useful to think of a "config tree" representing the dependencies between shareable configs and plugins. The root node is the end user's config, and each node has a set of named children representing the shareable configs that it extends and plugins that it depends on. Here's an example tree:

In this example, the end user's config extends eslint-config-foo and eslint-config-bar. eslint-config-bar extends eslint-config-baz. eslint-config-foo and eslint-config-baz both depend on versions of eslint-plugin-react (perhaps different versions, although this doesn't matter as far as resolution is concerned). eslint-config-baz also depends on eslint-plugin-import.

Rule name resolution

• Each reference to a plugin rule in a config consists of three parts: a config scope (i.e. a list of configs), a plugin name, and a rule name.
  • For example, in an existing rule configuration like react/no-typos, the config scope is an empty list, the plugin name is react, and the rule name is no-typos. (In existing rule configurations, the config scope is always an empty list.)
  • In a rule configuration like foo::bar::react/no-typos, the config scope is ['foo', 'bar'], the plugin name is react, and the rule name is no-typos.
  • The syntax shown here for writing a config scope (which uses :: as a separator) is up for bikeshedding. For now, I would recommend focusing on the abstract idea of a (configScope, pluginName, ruleName) triple; the question of how best to syntactically represent that can be decided independently of the rest of the proposal.
• Each reference to a plugin rule is also implicitly associated with a config in the config tree. References that appear in a config file are associated with that config file. References outside of a config file (e.g. from the command line or inline config comments) are associated with the root of the config tree.

To resolve a (configScope, pluginName, ruleName) triple to a loaded rule, which is referenced in a config baseConfig:

• If configScope is non-empty, find the child config of baseConfig in the config tree which has a name of configScope[0], and recursively resolve the rule (configScope.slice(1), pluginName, ruleName) from that config.
  • (If there is no such child config, the rule reference is invalid. ESLint should exit with a useful error message.)
• Otherwise, if configScope is empty:
  • If baseConfig has a direct child plugin with the name pluginName, or baseConfig is a plugin config from a plugin called pluginName, return the rule called ruleName from that plugin.
  • Otherwise, search for all plugins that are descendants of baseConfig in the config tree and have a name of pluginName.
    • If there is exactly one such plugin, return the rule called ruleName from that plugin.
    • If there are no such plugins, the rule reference is invalid. ESLint should exit with a useful error message.
    • If there is more than one such plugin, the rule reference is ambiguous. ESLint should exit with useful error message.
      • For example, this error message could include all of the matching plugins that were found, and provide a replacement rule reference that would disambiguate each of them. The user could the choose one of the replacements and copy-paste it into their config. This would make it simple for the user to resolve an ambiguity.

A few examples of this config resolution strategy, with the config tree given above (reproduced below for convenience):

Example config tree (same as above)

• If the end user's config references the rule react/no-typos, the config scope is empty. Since the root node of the tree has multiple descendants called eslint-plugin-react, the rule reference is ambiguous.
• If the end user's config references the rule bar::react/no-typos, the config scope is non-empty, so the resolution strategy then tries to resolve the rule react/no-typos from the eslint-config-bar node in the tree. Since there is only one descendent of that node called eslint-plugin-react, the rule would successfully resolve to the no-typos rule of that plugin.

Notable advantages and disadvantages of this strategy

• This strategy allows shareable configs to specify plugins and other shareable configs as direct dependencies, without manual installation steps by the user. It also ensures that the end user can always override any extended configuration.
• Any ambiguity in a rule reference in a given config file will be immediately apparent to the author of that config file, since the presence of an ambiguity only depends on the descendants of that config in the config tree. In other words, there is no situation where a particular user's configuration would be broken and they would need to lobby the author of their shareable config to make a change (because in that case, the shareable config would be broken for all of its users and likely would have been fixed before publishing).
• The strategy is mostly backwards-compatible with existing setups, because in existing setups there is always at most one version of a plugin reachable from anywhere in a config tree. The exceptions are cases where an existing config unnecessarily uses a plugins array to override rule configurations from a shareable config's plugin; with this change, the shareable config and the end user's config would end up configuring two independent versions of the same plugin. To fix this in both versions, the end user's config could simply remove that plugin from their plugins array. There are also probably setups now where a shareable config overrides the plugin rules configured by a sibling shareable config; these could be fixed in both versions by making one shareable config a child of the other.
• Adding a plugin to a shareable config is a breaking change for the shareable config, because it creates the possibility of an ambiguity in ancestor configs. I consider this to be acceptable because:
  • Adding a plugin to a shareable config would usually be a breaking change anyway, because the shareable config would be enabling new rules from that plugin, causing more errors to be reported to the end user.
  • With the current status quo, adding a plugin to a shareable config is always a breaking change because the end user needs to install it.
• With this strategy, an end user would be exposed to some details of the layout of shareable configs that they depend on. For example, if a shareable config eslint-config-foo has two descendant plugins with the same name, then the config scope that is needed to refer to those rules is "contagious". In other words, if eslint-config-foo needs to use a reference like bar::react/no-typos to avoid ambiguities, then a config that extends eslint-config-foo needs to use something like foo::bar::react/no-typos to configure that rule.
  • I consider this to be acceptable because end users are already exposed to many of the details of their shareable configs, in that changes to shareable configs will lead to different linting errors/warnings being reported on the end user's code. I think it's very important to protect configs from caring about details of sibling shareable configs, since this would create implicit dependencies between the sibling configs where no dependency otherwise existed. However, I think it's fine if configs are exposed to some details of descendent shareable configs, since there is already a dependency relationship between them anyway.
  • I believe that this problem (exposure to the details of dependency configs) is inherent to any solution that both (a) gives users the power to arbitrarily override third-party configuration, and (b) allows third-party configuration to pull in custom rules from multiple external sources. To illustrate this point, I'll examine two other proposals that attempted to solve the problem, and argue that the problem still exists even with those alternate proposals.

    • Support having plugins as dependencies in shareable config #3458 (comment) proposed adding an exportedPlugins field for shareable configs. This has an advantage that it makes the exposed API of a shareable config explicit. However, with this alternate proposal, ESLint would have to require each shareable config to export all of its dependency configs (otherwise the end user would be unable to reference rules from the dependency configs in order to override them). As a result, the dependency chain of a shareable config would still end up in an end-user config.

    • Support having plugins as dependencies in shareable config #3458 (comment) proposed solving the problem by using plugins that depend on other plugins and reexport the rules of their dependencies, without any changes to ESLint core. It suggests two possible ways of re-exporting the rules: either a plugin could export them directly with the same name, or it could give the names a common prefix. Unfortunately, both of these strategies have problems:

      • If plugins re-export rules using the same names as the ones provided by their dependencies, then they will encounter a conflict when two of their dependencies export a rule with the same name. In effect, I think this strategy just changes the initial problem of having plugins with the same name to a new problem of having rules with the same name, and shifts the burden of solving it onto plugin authors. Even if a plugin created an ad-hoc fix for this by using a different name for one of the rules, it would still have the possibility of breaking again if one of its dependency plugins added a rule with the same name as one of its other dependency plugins.
      • If plugins put rules behind namespaces, then when a namespacing plugin depends on another namespacing plugin, the namespaces would end up getting compounded. In order to use one of the rules, an end user would need to repeat back multiple levels of namespaces, effectively restating the plugin's dependency chain again. This would result in the same problem as with the current proposal.

      If these problems with alternative proposals seem like they would only occur in unlikely scenarios, I think it's worth noting that the problem in this proposal of having deep conflicting rule names in this proposal would also be somewhat unlikely. We should certainly make sure users have the ability to handle those scenarios, and give straightforward advice about how to do so (namely, that shareable configs need to bump the major version when adding plugins or adding dependencies that add plugins), but it's also important to make sure we make fair comparisons between proposals, and not compare the common cases of one proposal to the inconvenient edge cases of another.

---

I've spent a lot of time on this proposal, and would appreciate any feedback (particularly feedback that identifies problems with it). I think the current state of how plugins and configs are loaded is causing a large number of problems in the ecosystem, and I'm hoping we can work on this to finally pin down a solution.

[transitive-bullshit]

@not-an-aardvark this is an excellent breakdown of the core issue and proposed solution 👍

I've been following this issue for some time, and I can't see any problems with your proposed solution.

This is such a ubiquitous issue that affects thousands of developers, it'd be great to gather consensus around this solution and move the ecosystem forwards soon.

[calebeby]

@not-an-aardvark This seems like a really solid proposal, thanks for thinking this through. I don't see any problems atm. Is there anything we can do to get more visibility on this PR so we can get more feedback and move forward with this?

[ilyavolodin]

@not-an-aardvark What do we need to move forward with this proposal? Should we mark it for discussion with the rest of the TSC? I also have a few concerns about it below:

As a result, we need to be able to support having two plugins with the same name before we can support having plugins as dependencies of configs.

I'm not at all convinced that this is a real problem. Currently we require plugins to be valid NPM packages. Because of that, it's impossible to have two plugins with the same name. Now it is possible to have a config and a plugin with the same short-form name, but that shouldn't be an issue, since you can't really configure rules from the config. I know this requirement has been around for a very long time and was create by @nzakas but I honestly could never understand why we are trying to solve a non-existing problem.
Now having multiple shareable configs that rely on different versions of the same plugin might actually be a problem. However, I'm also not convinced that this wouldn't be a extremely rare edge case, and would justify added complexity from both configuration and coding perspectives. I, personally, would be perfectly fine with throwing a blocking error at the config resolution time in cases like that.

[ljharb]

@ilyavolodin you could easily have two versions of the same plugin package.

[not-an-aardvark]

What do we need to move forward with this proposal? Should we mark it for discussion with the rest of the TSC?

I'd be happy for more people to take a look at it. For now, if people have objections/concerns I think it might be more productive to discuss them on GitHub rather than at a TSC meeting. The complexity of this proposal in combination with the limited-time setting of a TSC meeting might make it difficult to address objections to the technical details.

I think it would also be beneficial to have an implementation of this ready before we officially accept it at a TSC meeting; this would give a chance for users and TSC members to try it out and see how it works, and we could be aware of any problems that would pop up before officially making a decision on it. (I'd like to implement this at some point, but I haven't started yet.)

I'm not at all convinced that this is a real problem. Currently we require plugins to be valid NPM packages. Because of that, it's impossible to have two plugins with the same name.

Thanks for the feedback and for voicing your concerns.

It's true that we currently require plugins to be located in node_modules. However, there are a few cases where plugins can have the same name:

• Different configs could depend on different versions of a plugin from npm, as you noted.
• Future enhancements to ESLint might allow plugins to be loaded from outside of node_modules. For example, I think Ability to load plugins by path, not just name #6237 (adding the ability to load plugins by path) would provide a huge benefit for users. However, that issue is blocked on the ability to handle plugins with the same name, because the loaded plugin from a path would need to be given a name, which could be the same as the name of another loaded plugin.
• Packages in node_modules could come from anywhere; they don't necessarily need to come from the npm registry. As a result, there could be two entirely unrelated plugins in node_modules with the same name. (However, most users do install their packages from the npm registry, so I'm not designing for this case.)

However, I'm also not convinced that this wouldn't be a extremely rare edge case, and would justify added complexity from both configuration and coding perspectives. I, personally, would be perfectly fine with throwing a blocking error at the config resolution time in cases like that.

It should be noted that for users who don't install plugins with the same name, the config complexity would be the same as it is currently. In other words, the new syntax is only necessary for resolving conflicts, so it doesn't need to be used if there are no conflicts. I agree that this would add implementation complexity in either case.

I also think that this case wouldn't really be that rare, because many more shareable configs would include plugins after this change, increasing the chance of a conflict. (Currently, adding a plugin to a shareable config creates an installation burden for the user, so shareable configs often don't include them.) To me, it seems important that users can combine shareable configs without encountering fatal errors.

[ilyavolodin]

In other words, the new syntax is only necessary for resolving conflicts, so it doesn't need to be used if there are no conflicts.

I missed that part of the design proposal. Thanks for pointing that out. That does lower my concern about increased complexity of configuration. But still doesn't remove the increased complexity of the code on our side.

I also think that this case wouldn't really be that rare, because many more shareable configs would include plugins after this change, increasing the chance of a conflict.

Problem is, it feels like we are trying to design a complex system without having any info/data on how it's going to be used. I think we are just guessing here about how often something like this is going to happen and how many people will run into this problem. I'm not sure how we can get more data on this subject without releasing the enhancement, but I also don't want to spend a whole lot of effort designing a fall-back strategy for something that might be very rare behavior with unexpected results.

@ilyavolodin you could easily have two versions of the same plugin package.

Sure, I agree that might happen. I'm not sure, however, that's actually a desired behavior and makes sense from the user's perspective. At least right now, I can't come up with the scenario where something like this would make sense.

[mightyiam]

I've been lurking. I care about StandardJS and I'm not finding any comments from the main people here so my apologies for the mentions in advance, but you probably should take a look at this, since eslint-config-standard is a popular shareable config that does depend on several plugins.
@feross, @devjin0617, @Flet, @Asoul, @falmar

[not-an-aardvark]

Problem is, it feels like we are trying to design a complex system without having any info/data on how it's going to be used. I think we are just guessing here about how often something like this is going to happen and how many people will run into this problem. ... I also don't want to spend a whole lot of effort designing a fall-back strategy for something that might be very rare behavior with unexpected results.

Generally speaking, I want ESLint to be robust enough that it doesn't create failures for reasons outside of the user's control. I think throwing a fatal error for duplicate plugin names would be a poor solution because it would it would create a "dependency hell" scenario -- a shareable config would face pressure to not add a particular plugin as a dependency, because it would be incompatible with a user's other configs and the user would have no recourse except to stop using that shareable config.

We have enough users that "rare" setups will still occur fairly often in terms of absolute numbers. Even if this only happened for 1% of config setups, we would still be creating problems for thousands of users. I think it's important that we give the user a way to resolve conflicts rather than just deciding not to handle the edge case.

If this proposal fails, then I'd be willing to consider throwing a fatal error on conflicts as a backup, since it does have some significant advantages (and some disadvantages) over the current state of affairs. But I think this is a case where it's worth the implementation complexity to implement a more complete solution.

[ilyavolodin]

We have enough users that "rare" setups will still occur fairly often in terms of absolute numbers. Even if this only happened for 1% of config setups, we would still be creating problems for thousands of users. I think it's important that we give the user a way to resolve conflicts rather than just deciding not to handle the edge case.

That's a good argument. I'm just concern about added complexity for the end user when they run into an issue like that. Problem is, dependency on multiple plugin version will most likely never occur in the shareable config. It will always happen in the end-users config (at least that's my guess). And also on the maintenance burden on us for this extra feature. But I guess I can live with both, considering we've had this problem for number of years, and the is the first complete proposal that we have for it.

[ljharb]

@ilyavolodin if something like this proposal goes through, then eslint-config-airbnb very well could depend on another config, which could bring in its own plugins and config, which could create a conflict - and that level of composability is exactly what's already possible and awesome with npm, and it'd be great if shared configs could do the same.

[arcanis]

Note that this issue bites us in the context of Plug'n'Play, where we strictly enforce boundaries between packages. We do have a top-level fallback to account for this kind of plugin systems (if a package is required by a dependency that doesn't own it, we check whether the top-level dependencies list it and use it if it's the case), but it breaks apart in the case of shared configs (since in this case the top-level package usually doesn't list each individual plugin of the shared configs).

The two solutions to this issue are either:

• For the shared configs to be able to resolve their plugins themselves using require.resolve (instead of delegating the resolution to Eslint), but it unfortunately doesn't work because Eslint doesn't accept absolute paths.

• For Eslint to call require.resolve by specifying the configuration path as issuer (using the paths option of require.resolve). This doesn't work either for the same reason: the returned path would be absolute.

[ljharb]

That seems like a flaw in PNP if it can’t account for normal resolution and dep nesting :-/

[arcanis]

That seems like a flaw in PNP if it can’t account for normal resolution and dep nesting :-/

@ljharb It cannot because it's invalid, even right now. Let's take the following tree (nodes are packages rather than folders). Let's take the case of a package Foo depending on Bar, and Bar depending on Baz.

In typical Yarn installs, those three packages all get hoisted to the top-level in order to optimize the dependency tree. Everything's fine: Foo can require Bar, and Bar can require Baz. But is it really fine? Hoisting those packages had an important side effect: Foo can now also require Baz, which was impossible before (because it was hidden inside Bar).

Hoisting in as optimization. It may happen, but there's nothing that guarantees it. Even worse, relying on such a behavior means that your code will behave in unpredictable ways based on the other packages in your dependency tree - something that you don't control! For example, let's say that our top-level also depends on Qux, which depends on Baz@2.0 (an entirely API-incompatible version of Baz). What will happen when Foo requires Baz? Will it load Baz@1.0, or Baz@2.0? Sad answer is that you don't know: it will depend on which package gets hoisted up to the top-level (assuming one of them even gets hoisted!). Meaning that, as the author of Foo, you actually have no idea whether your package will work or not for your users.

Plug'n'Play guarantees strict boundaries. We could decide not to do it, but I have to emphasis it: it does so by design. It prevents you from relying on Undefined Behaviors (in C/C++ terms), and as such makes your development more stable.

Of course, tools relying on it to load plugins have a slightly hardest time, but the fallback coupled with require.resolve make it relatively easy to workaround and then fix. Eslint troubles are quite specific to the way your configuration is setup, and the conflicts that might arise when multiple plugins with the same names are used together.

[not-an-aardvark]

Due to the issues in #10125, it's unsurprising to me that ESLint is currently failing with certain package management strategies, but I'm a bit confused about why this is happening with PNP. You mentioned that:

it breaks apart in the case of shared configs (since in this case the top-level package usually doesn't list each individual plugin of the shared configs)

In fact, I think the current convention is for top-level packages to explicitly list the plugins for their shareable configs. Does the issue still occur when running ESLint from the top-level package, as opposed to from nested packages like react-scripts?

[arcanis]

In fact, I think the current convention is for top-level packages to explicitly list the plugins for their shareable configs.

If the top-level application lists all of the plugins, they will fall into the fallback, yep. It goes against the spirit of create-react-app & similar other scaffolding tools, though (cc @gaearon)

[gaearon]

Yeah, in Create React App we're pretty intentional about ESLint being an implementation detail of our toolchain. We don't want our users to be exposed to a fact that there is a preset behind it until they "eject".

[ljharb]

I don't see the conflict; CRA can list all the preset's peer deps as deps, and that satisfies the peer dep requirements without hoisting anything up to the user's app itself.

[not-an-aardvark]

If the top-level application lists all of the plugins, they will fall into the fallback, yep. It goes against the spirit of create-react-app & similar other scaffolding tools, though

Understood. I was just trying to determine how much of the ecosystem is currently affected by this issue.

@ljharb The issue is that ESLint's plugin-loading depends on eslint being able to load a package that eslint doesn't specify anywhere in package.json. The resulting system works in some situations due to package manager implementation details, but is unsound.

Can we move this discussion to #10125? I think that issue has a better description of the problem, whereas this issue is more focused on a particular proposal.

[nzakas]

Hi everyone,

The ESLint team has just created an RFC process for complicated changes that require designs. It seems like this discussion is now split amongst at least three different issues and would benefit from being consolidated into an RFC:

https://github.com/eslint/rfcs/

I'd like to suggest that whomever is most interested (@not-an-aardvark?) please consolidate everything into an RFC proposal and close the outstanding issues so we can focus the conversation.

Thanks!

[not-an-aardvark]

I've opened an RFC at eslint/rfcs#5. Closing this issue so that discussion can move there instead.