Should extras behavior be non-recursive?

[yanok]

Problem

Right now extras is recursive: meaning that if there is a complicated (non-token) extra rule, other extras (including the rule itself!) could appear anywhere inside. While this is sometimes desirable, for example, one could get a nested /* */ comments just by adding an extra rule

comment: $ => seq('/*', repeat(/./), '*/')

but this could also be confusing:

1. What if my language doesn't actually support nesting and this was not intended?
2. Even if I want to support nested comments, I most likely don't want to show them as separate syntax nodes, just the top-level one, and there doesn't seem to be a way to achieve this.
3. Worse than that, if I also have a single line comments //, defined as

   sl_comment: $ => seq('//', /[^\n]*/, '\n')

   as another extra rule, that can also appear inside comment, potentially breaking the /* balancing, such that

   /* /* // */
   */
   

   is invalid. Again, this might be what you want, but there is no control over it.

Expected behavior

I think the easiest thing is to just make extras non-recursive: rules listed in extras can appear anywhere expect for the rules listed in extras themselves. I guess there are not many extras usually, so there nesting is desired, it could be added explicitly.

Alternatively, there could be an additional configuration for extras, specifying for every pair of extra rules, if one could appear inside another, but that's probably an overkill.

[ObserverOfTime]

If you can implement this and it doesn't break, say comments with spaces inside, then feel free to submit a PR.