getTokenBefore dereferences a null, when processing "await import" templated string

[hwinkler]

Tell us about your environment

• ESLint Version:
  7.0.0-alpha.2
• Node Version:
  12.16.1
• npm Version:
  6.13.4

What parser (default, Babel-ESLint, etc.) are you using?
babel-eslint
Please show your full configuration:

Configuration

{
  "parser": "babel-eslint",
  "plugins": [
    "babel"
  ],

  "globals": {
    "Buffer": false,
    "clearTimeout": false,
    "console": false,
    "__dirname": false,
    "document": false,
    "EventSource": false,
    "expect": false,
    "File": false,
    "FileReader": false,
    "Float32Array": false,
    "global": false,
    "module": false,
    "navigator": false,
    "process": false,
    "Promise": false,
    "Reflect": false,
    "require": false,
    "setTimeout": false,
    "sinon": false,
    "Symbol": false,
    "TextDecoder": false,
    "window": false,
    "Worker": false
  },

  "rules": {
    //
    //Possible Errors
    //
    // The following rules point out areas where you might have made mistakes.
    //
    "comma-dangle": 2, // disallow or enforce trailing commas (recommended)
    "no-cond-assign": 2, // disallow assignment in conditional expressions (recommended)
    "no-console": 0, // disallow use of console (recommended) (temporarily suppressing until I do Winston logging)
    "no-constant-condition": 2, // disallow use of constant expressions in conditions (recommended)
    "no-control-regex": 2, // disallow control characters in regular expressions (recommended)
    "no-debugger": 2, // disallow use of debugger (recommended)
    "no-dupe-args": 2, // disallow duplicate arguments in functions (recommended)
    "no-dupe-keys": 2, // disallow duplicate keys when creating object literals (recommended)
    "no-duplicate-case": 2, // disallow a duplicate case label. (recommended)
    "no-empty": 2, // disallow empty block statements (recommended)
    "no-empty-character-class": 2, // disallow the use of empty character classes in regular expressions (recommended)
    "no-ex-assign": 2, // disallow assigning to the exception in a catch block (recommended)
    "no-extra-boolean-cast": 2, // disallow double-negation boolean casts in a boolean context (recommended)
    "no-extra-parens": 0, // disallow unnecessary parentheses
    "no-extra-semi": 2, // disallow unnecessary semicolons (recommended) (fixable)
    "no-func-assign": 2, // disallow overwriting functions written as function declarations (recommended)
    "no-inner-declarations": 2, // disallow function or variable declarations in nested blocks (recommended)
    "no-invalid-regexp": 2, // disallow invalid regular expression strings in the RegExp constructor (recommended)
    "no-irregular-whitespace": 2, // disallow irregular whitespace outside of strings and comments (recommended)
    "no-negated-in-lhs": 2, // disallow negation of the left operand of an in expression (recommended)
    "no-obj-calls": 2, // disallow the use of object properties of the global object (Math and JSON) as functions (recommended)
    "no-regex-spaces": 2, // disallow multiple spaces in a regular expression literal (recommended)
    "no-sparse-arrays": 2, // disallow sparse arrays (recommended)
    "no-unexpected-multiline": 2, // Avoid code that looks like two expressions but is actually one (recommended)
    "no-unreachable": 2, // disallow unreachable statements after a return, throw, continue, or break statement (recommended)
    "use-isnan": 2, // disallow comparisons with the value NaN (recommended)
    "valid-jsdoc": 2, // Ensure JSDoc comments are valid
    "valid-typeof": 2, // Ensure that the results of typeof are compared against a valid string (recommended)


    //
    // Best Practices
    //
    // These are rules designed to prevent you from making mistakes.
    // They either prescribe a better way of doing something or help you avoid footguns.
    //
    "accessor-pairs": 2, // Enforces getter/setter pairs in objects
    "array-callback-return": 2, // Enforces return statements in callbacks of array's methods
    "block-scoped-var": 2, // treat var statements as if they were block scoped
    "complexity": 0, // specify the maximum cyclomatic complexity allowed in a program
    "consistent-return": 2, // require return statements to either always or never specify values
    "curly": 2, // specify curly brace conventions for all control statements
    "default-case": 2, // require default case in switch statements
    "dot-location": [2, "property"], // enforces consistent newlines before or after dots
    "dot-notation": 2, // encourages use of dot notation whenever possible
    "eqeqeq": 2, // require the use of === and !==
    "guard-for-in": 2, // make sure for-in loops have an if statement
    "no-alert": 2, // disallow the use of alert, confirm, and prompt
    "no-caller": 2, // disallow use of arguments.caller or arguments.callee
    "no-case-declarations": 2, // disallow lexical declarations in case clauses (recommended)
    "no-div-regex": 2, // disallow division operators explicitly at beginning of regular expression
    "no-else-return": 2, // disallow else after a return in an if
    "no-empty-function": 0, // disallow use of empty functions
    "no-empty-pattern": 2, // disallow use of empty destructuring patterns (recommended)
    "no-eq-null": 2, // disallow comparisons to null without a type-checking operator
    "no-eval": 2, // disallow use of eval()
    "no-extend-native": 2, // disallow adding to native types
    "no-extra-bind": 2, // disallow unnecessary function binding
    "no-extra-label": 2, // disallow unnecessary labels
    "no-fallthrough": 2, // disallow fallthrough of case statements (recommended)
    "no-floating-decimal": 2, // disallow the use of leading or trailing decimal points in numeric literals
    "no-implicit-coercion": 2, // disallow the type conversions with shorter notations
    "no-implicit-globals": 2, // disallow var and named functions in global scope
    "no-implied-eval": 2, // disallow use of eval()-like methods
    "no-invalid-this": 2, // disallow this keywords outside of classes or class-like objects
    "no-iterator": 2, // disallow usage of __iterator__ property
    "no-labels": 2, // disallow use of labeled statements
    "no-lone-blocks": 2, // disallow unnecessary nested blocks
    "no-loop-func": 2, // disallow creation of functions within loops
    "no-magic-numbers": 0, // disallow the use of magic numbers
    "no-multi-spaces": 2, // disallow use of multiple spaces (fixable)
    "no-multi-str": 2, // disallow use of multiline strings
    "no-native-reassign": 2, // disallow reassignments of native objects
    "no-new": 2, // disallow use of the new operator when not part of an assignment or comparison
    "no-new-func": 2, // disallow use of new operator for Function object
    "no-new-wrappers": 2, // disallows creating new instances of String,Number, and Boolean
    "no-octal": 2, // disallow use of octal literals (recommended)
    "no-octal-escape": 2, // disallow use of octal escape sequences in string literals, such as var foo = "Copyright \251";
    "no-param-reassign": 2, // disallow reassignment of function parameters
    "no-process-env": 0, // disallow use of process.env
    "no-proto": 2, // disallow usage of __proto__ property
    "no-redeclare": 2, // disallow declaring the same variable more than once (recommended)
    "no-return-assign": 2, // disallow use of assignment in return statement
    "no-script-url": 2, // disallow use of javascript: urls.
    "no-self-assign": 2, // disallow assignments where both sides are exactly the same (recommended)
    "no-self-compare": 2, // disallow comparisons where both sides are exactly the same
    "no-sequences": 2, // disallow use of the comma operator
    "no-throw-literal": 2, // restrict what can be thrown as an exception
    "no-unmodified-loop-condition": 2, // disallow unmodified conditions of loops
    "no-unused-expressions": 2, // disallow usage of expressions in statement position
    "no-unused-labels": 2, // disallow unused labels (recommended)
    "no-useless-call": 2, // disallow unnecessary .call() and .apply()
    "no-useless-concat": 2, // disallow unnecessary concatenation of literals or template literals
    "no-void": 2, // disallow use of the void operator
    "no-warning-comments": 0, // disallow usage of configurable warning terms in comments - e.g. TODO or FIXME
    "no-with": 2, // disallow use of the with statement
    "radix": 2, // require use of the second argument for parseInt()
    "require-await": 2, // disallow async functions which have no await expression
    "vars-on-top": 2, // require declaration of all vars at the top of their containing scope
    "wrap-iife": 2, // require immediate function invocation to be wrapped in parentheses
    "yoda": 2, // require or disallow Yoda conditions


    //
    // Strict Mode
    //
    // These rules relate to using strict mode.
    //
    "strict": 0, // controls location of Use Strict Directives. 0: required by babel-eslint


    //
    // Variables
    //
    // These rules have to do with variable declarations.
    //
    "init-declarations": 0, // enforce or disallow variable initializations at definition
    "no-catch-shadow": 2, // disallow the catch clause parameter name being the same as a variable in the outer scope
    "no-delete-var": 2, // disallow deletion of variables (recommended)
    "no-label-var": 2, // disallow labels that share a name with a variable
    "no-restricted-globals": 0, // restrict usage of specified global variables
    "no-shadow": 2, // disallow declaration of variables already declared in the outer scope
    "no-shadow-restricted-names": 2, // disallow shadowing of names such as arguments
    "no-undef": 2, // disallow use of undeclared variables unless mentioned in a /*global */ block (recommended)
    "no-undef-init": 2, // disallow use of undefined when initializing variables
    "no-undefined": 0, // disallow use of undefined variable
    "no-unused-vars": [2, {"argsIgnorePattern": "^_", "varsIgnorePattern": "^_"}], // disallow declaration of variables that are not used in the code (recommended)
    "no-use-before-define": [2, "nofunc"], // disallow use of variables before they are defined

    //
    // Node.js and CommonJS
    //
    // These rules are specific to JavaScript running on Node.js or using CommonJS in the browser.
    //
    "callback-return": 2, // enforce return after a callback
    "global-require": 0, // enforce require() on top-level module scope
    "handle-callback-err": 2, // enforce error handling in callbacks
    "no-buffer-constructor": 2, // disallow use of the Buffer() constructor
    "no-mixed-requires": 2, // disallow mixing regular variable and require declarations
    "no-new-require": 2, // disallow use of new operator with the require function
    "no-path-concat": 2, // disallow string concatenation with __dirname and __filename
    "no-process-exit": 0, // disallow process.exit()
    "no-restricted-imports": 0, // restrict usage of specified node imports
    "no-restricted-modules": 0, // restrict usage of specified node modules
    "no-sync": 1, // disallow use of synchronous methods

    //
    // Stylistic Issues
    //
    // These rules are purely matters of style and are quite subjective.
    //
    "array-bracket-spacing": [2, "never"], // enforce spacing inside array brackets (fixable)
    "block-spacing": [2, "always"], // disallow or enforce spaces inside of single line blocks (fixable)
    "brace-style": 2, // enforce one true brace style
    "camelcase": 0, // require camel case names
    "comma-spacing": [2, {"before": false, "after": true}], // enforce spacing before and after comma (fixable)
    "comma-style": [2, "last"], // enforce one true comma style
    "computed-property-spacing": [2, "never"], // require or disallow padding inside computed properties (fixable)
    "consistent-this": [2, "_this"], // enforce consistent naming when capturing the current execution context
    "eol-last": 2, // enforce newline at the end of file, with no multiple empty lines (fixable)
    "func-names": 0, // require function expressions to have a name
    "func-style": 0, // enforce use of function declarations or expressions
    "id-blacklist": 0, // blacklist certain identifiers to prevent them being used
    "id-length": 0, // this option enforces minimum and maximum identifier lengths (variable names, property names etc.)
    "id-match": 0, // require identifiers to match the provided regular expression
    "indent": [2, 2, {"SwitchCase": 1}], // specify tab or space width for your code (fixable)
    "jsx-quotes": [2, "prefer-double"], // specify whether double or single quotes should be used in JSX attributes (fixable)
    "key-spacing": [2, {"beforeColon": false, "afterColon": true}], // enforce spacing between keys and values in object literal properties
    "keyword-spacing": [2, {"before": true, "after": true}], // enforce spacing before and after keywords (fixable)
    "linebreak-style": [2, "unix"], // disallow mixed 'LF' and 'CRLF' as linebreaks
    "lines-around-comment": [2, {"beforeBlockComment": true, "afterBlockComment": false, "afterLineComment": false}], // enforce empty lines around comments
    "max-depth": [2, 3], // specify the maximum depth that blocks can be nested
    "max-len": [2, 100, 2], // specify the maximum length of a line in your program
    "max-nested-callbacks": [2, 3], // specify the maximum depth callbacks can be nested
    "max-params": [2, 5], // limits the number of parameters that can be used in the function declaration
    "max-statements": 0, // specify the maximum number of statement allowed in a function
    "new-cap": [2, {"newIsCap": true, "capIsNew": false}], // require a capital letter for constructors
    "new-parens": 2, // disallow the omission of parentheses when invoking a constructor with no arguments
    "newline-after-var": [2, "always"], // require or disallow an empty newline after variable declarations
    "newline-per-chained-call": 0, // enforce newline after each call when chaining the calls
    "no-array-constructor": 2, // disallow use of the Array constructor
    "no-bitwise": 2, // disallow use of bitwise operators
    "no-continue": 2, // disallow use of the continue statement
    "no-inline-comments": 0, // disallow comments inline after code
    "no-lonely-if": 2, // disallow if as the only statement in an else block
    "no-mixed-spaces-and-tabs": 2, // disallow mixed spaces and tabs for indentation (recommended)
    "no-multiple-empty-lines": [2, {"max": 2}], // disallow multiple empty lines
    "no-negated-condition": 2, // disallow negated conditions
    "no-nested-ternary": 2, // disallow nested ternary expressions
    "no-new-object": 2, // disallow the use of the Object constructor
    "no-plusplus": 0, // disallow use of unary operators, ++ and --
    "no-restricted-syntax": [2, "WithStatement"], // disallow use of certain syntax in code
    "no-spaced-func": 2, // disallow space between function identifier and application (fixable)
    "no-ternary": 0, // disallow the use of ternary operators
    "no-trailing-spaces": 1, // disallow trailing whitespace at the end of lines (fixable)
    "no-underscore-dangle": 2, // disallow dangling underscores in identifiers
    "no-unneeded-ternary": 2, // disallow the use of ternary operators when a simpler alternative exists
    "no-whitespace-before-property": 2, // disallow whitespace before properties
    "object-curly-spacing": [2, "never"], // require or disallow padding inside curly braces (fixable)
    "one-var": [2, "never"], // require or disallow one variable declaration per function
    "one-var-declaration-per-line": 2, // require or disallow an newline around variable declarations
    "operator-assignment": [2, "never"], // require assignment operator shorthand where possible or prohibit it entirely
    "operator-linebreak": [2, "after"], // enforce operators to be placed before or after line breaks
    "padded-blocks": [2, "never"], // enforce padding within blocks
    "quote-props": [2, "as-needed"], // require quotes around object literal property names
    "quotes": [2, "single"], // specify whether backticks, double or single quotes should be used (fixable)
    "require-jsdoc": 0, // Require JSDoc comment
    "semi": [0, "never"], // require or disallow use of semicolons instead of ASI (fixable)
    "semi-spacing": [2, {"before": false, "after": true}], // enforce spacing before and after semicolons (fixable)
    "sort-imports": 0, // sort import declarations within module
    "sort-vars": 0, // sort variables within the same declaration block
    "space-before-blocks": [2, "always"], // require or disallow a space before blocks (fixable)
    "space-before-function-paren": [2, {"anonymous": "always", "named": "never"}], // require or disallow a space before function opening parenthesis (fixable)
    "space-in-parens": [2, "never"], // require or disallow spaces inside parentheses (fixable)
    "space-infix-ops": 2, // require spaces around operators (fixable)
    "space-unary-ops": [2, {"words": true, "nonwords": false}], // require or disallow spaces before/after unary operators (fixable)
    "spaced-comment": [2, "always"], // require or disallow a space immediately following the // or /* in a comment
    "wrap-regex": 0, // require regex literals to be wrapped in parentheses
    //
    // ECMAScript 6
    //
    // These rules are only relevant to ES6 environments and are off by default.
    //
    "arrow-body-style": [2, "as-needed"], // require braces in arrow function body
    "arrow-parens": [2, "as-needed"], // require parens in arrow function arguments
    "arrow-spacing": 2, // require space before/after arrow function's arrow (fixable)
    "constructor-super": 2, // verify calls of super() in constructors (recommended)
    "generator-star-spacing": [2, "before"], // enforce spacing around the * in generator functions (fixable)
    "no-class-assign": 2, // disallow modifying variables of class declarations (recommended)
    "no-confusing-arrow": 0, // disallow arrow functions where they could be confused with comparisons
    "no-const-assign": 2, // disallow modifying variables that are declared using const (recommended)
    "no-dupe-class-members": 2, // disallow duplicate name in class members (recommended)
    "no-new-symbol": 2, // disallow use of the new operator with the Symbol object (recommended)
    "no-this-before-super": 2, // disallow use of this/super before calling super() in constructors (recommended)
    "no-useless-constructor": 2, // disallow unnecessary constructor
    "no-var": 2, // require let or const instead of var
    "object-shorthand": 2, // require method and property shorthand syntax for object literals
    "prefer-arrow-callback": 2, // suggest using arrow functions as callbacks
    "prefer-const": 2, // suggest using const declaration for variables that are never modified after declared
    "prefer-reflect": 0, // suggest using Reflect methods where applicable
    "prefer-rest-params": 2, // suggest using the rest parameters instead of arguments
    "prefer-spread": 2, // suggest using the spread operator instead of .apply()
    "prefer-template": 2, // suggest using template literals instead of strings concatenation
    "require-yield": 2, // disallow generator functions that do not have yield
    "template-curly-spacing": 2, // enforce spacing around embedded expressions of template strings (fixable)
    "yield-star-spacing": 2 // enforce spacing around the * in yield* expressions (fixable)
  }
  
}

What did you do? Please include the actual source code causing the issue, as well as the command that you used to run ESLint.

'use strict';

const loadData = async sampleDir => {
  const {foo} = await import(`${sampleDir}/x.js`)

  return foo()
}

const p = loadData('.')
p.then( x =>console.log(x))

eslint --env es6 fail.js

What did you expect to happen?
Successful parse. That program runs to completion under Node >= 12 (node --experimental-modules fail.js, where file x.js is the one liner export const foo = () => 'ok').

What actually happened? Please include the actual, raw output from ESLint.

eslint threw an exception, possibly confused by the await import syntax. That syntax is supported natively in node , as well as by Babel ES6.

TypeError: Cannot read property 'range' of null
Occurred while linting /home/hughw/eslint-get-token-before-bug/fail.js:4
    at SourceCode.getTokenBefore (/home/hughw/eslint/lib/source-code/token-store/index.js:298:18)
    at checkSpacingBefore (/home/hughw/eslint/lib/rules/template-curly-spacing.js:60:42)
    at TemplateElement (/home/hughw/eslint/lib/rules/template-curly-spacing.js:119:17)
    at /home/hughw/eslint/lib/linter/safe-emitter.js:45:58
    at Array.forEach (<anonymous>)
    at Object.emit (/home/hughw/eslint/lib/linter/safe-emitter.js:45:38)
    at NodeEventGenerator.applySelector (/home/hughw/eslint/lib/linter/node-event-generator.js:254:26)
    at NodeEventGenerator.applySelectors (/home/hughw/eslint/lib/linter/node-event-generator.js:283:22)
    at NodeEventGenerator.enterNode (/home/hughw/eslint/lib/linter/node-event-generator.js:297:14)
    at CodePathAnalyzer.enterNode (/home/hughw/eslint/lib/linter/code-path-analysis/code-path-analyzer.js:635:23)  

Are you willing to submit a pull request to fix this bug?
uncomfortable with my understanding of what needs to be done.

I hardened the code in lib/rules/template-curly-spacing.js to do nothing in case of null, but running again revealed the next error, again due to a null token. At that point I became uncomfortable that I wasn't understanding the bigger picture. So I'm submitting this bug for you guys.

[hwinkler]

Uploaded a small project with repro code: eslint-get-token-before-bug.tar.gz

[mdjermanovic]

Hi @hwinkler, thanks for the bug report and all the details!

I hardened the code in lib/rules/template-curly-spacing.js to do nothing in case of null, but running again revealed the next error, again due to a null token. At that point I became uncomfortable that I wasn't understanding the bigger picture. So I'm submitting this bug for you guys.

sourceCode.getFirstToken(node) in TemplateElement(node) {} should never return null.

I believe this is a problem in babel-eslint, ref babel/babel#10904.

It seems that TemplateElement node inside import() expression doesn't have the correct range, so sourceCode.getFirstToken can't find its Template token (which has the correct range).

I think this is planned to be fixed in babel-eslint v11 (cc @kaicataldo).

[hwinkler]

Thank you @mdjermanovic that appears to be precisely the problem.

It is not yet fixed in pre-release babel-eslint. So the workaround meantime is to construct the template string on one line and use it in the dynamic import on a second line.