Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Poor performance on some projects #389

Closed
jrparish opened this issue Mar 29, 2019 · 66 comments
Closed

Poor performance on some projects #389

jrparish opened this issue Mar 29, 2019 · 66 comments
Labels
bug Something isn't working package: eslint-plugin Issues related to @typescript-eslint/eslint-plugin

Comments

@jrparish
Copy link

I'm opening a new issue since the other (#243) is closed and might not have visibility, but please close if not appropriate.

We are trying to use typescript-eslint on a pretty large project with 1000+ files. The CLI is very slow (see stats below), which I am able to live with, but since starting to use this plugin all our developers have been experiencing system lockups when running VSCode with the eslint plugin.

Additional info from my comment on the closed issue (#243 (comment)) -

CLI stats:

tslint with project - 20s
eslint with project - 96s
eslint without project - 57s

How many files are there?
1000+

Are you linting a multi tsconfig repo?
No

What rules are you using?
recommended + airbnb + prettier

Are you running eslint in watch mode? Via your ide? Via CLI on your entire repo?

  • CLI, non-watch
  • VSCode

Versions

package version
@typescript-eslint/eslint-plugin 1.5.0
@typescript-eslint/parser 1.5.0
TypeScript 3.3.4000
ESLint 5.15.1
node 10.15.0
@jrparish jrparish added package: eslint-plugin Issues related to @typescript-eslint/eslint-plugin triage Waiting for maintainers to take a look labels Mar 29, 2019
@armano2
Copy link
Member

armano2 commented Mar 29, 2019

q: did you used prettier + tslint in your test?

@jrparish
Copy link
Author

@armano2 - Yes, we also use prettier with tslint.

@bradzacher bradzacher added bug Something isn't working and removed triage Waiting for maintainers to take a look labels Mar 30, 2019
@andrewisaburden
Copy link

We also have the same issue. Setting parserOptions.project to undefined and setting these rules to 'off' resolves the issue, but we would like those rules to be enabled.

'@typescript-eslint/no-for-in-array': 'off',
'@typescript-eslint/no-unnecessary-qualifier': 'off',
'@typescript-eslint/no-unnecessary-type-assertion': 'off',
'@typescript-eslint/promise-function-async': 'off',
'@typescript-eslint/restrict-plus-operands': 'off',

We are using the CLI. Not using prettier still results in the same long linting time.

package version
@typescript-eslint/eslint-plugin 1.5.0
@typescript-eslint/parser 1.5.0
TypeScript 3.3.4000
ESLint 5.15.3
node 11.7.0

Using TIMING=1 DEBUG=eslint:cli-engine shows long lint times per file, and there are no individual rules that account for the time taken, so it is presumably generating the AST per file and that's taking all the time.

@deser
Copy link

deser commented Apr 2, 2019

Circle CI can't complete my linting, fails by 10 minutes timeout :) With babel-eslint it was 2-3 mins.

P.S. With project: undefined it runs on the same config on the same CI 2 mins

@bbugh
Copy link

bbugh commented Apr 5, 2019

I am having this issue as well, and our lint is only 126 files. We do not use prettier or any of the other @typescript-eslint packages, I am only using @typescript-eslint/parser.

Without project setting: 3.64 seconds
With project setting: ~7 minutes (426.79 seconds) 😮


package version
@typescript-eslint/parser 1.6.0
TypeScript 3.4.1
ESLint 5.16.0
node 10.15.3
npm 6.4.1
yarn 1.13.0

Relevant .eslintrc.js config:

// .eslintrc.js
  "parser": "vue-eslint-parser",
  "parserOptions": {
    "parser": "@typescript-eslint/parser",
    "project": "./tsconfig.json",
    "extraFileExtensions": [".vue"]
  },
// tsconfig.json
{
  "compilerOptions": {
    "declaration": false,
    "downlevelIteration": true,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "lib": ["es6", "dom", "es2017", "es2017.object"],
    "module": "esnext",
    "moduleResolution": "node",
    "baseUrl": ".",
    "paths": {
      "@/*": ["app/javascript/*"],
      "#/*": ["app/javascript/modules/*"]
    },
    "sourceMap": true,
    "strict": true,
    "target": "esnext"
  },
  "include": ["app/javascript/**/*.ts"],
  "exclude": [
    "**/*.spec.ts",
    "node_modules",
    "vendor",
    "public"
  ],
  "compileOnSave": false,
  "resolve": {
    "extensions": [ ".ts", ".tsx", ".js", ".json"]
  }
}

@kamok
Copy link

kamok commented Apr 5, 2019

@bbugh How are you running eslint? For example, npx eslint 'src/**/*.{ts,vue}' is what I do.

I am also able to replicate the massive increase in time when adding project

eslintrc.json

{
    "parser": "vue-eslint-parser",
    "parserOptions": {
        "ecmaVersion": 2018,
        "sourceType": "module",
        "parser": "@typescript-eslint/parser",
        "extraFileExtensions": [".vue"]
    },
    "extends": [
        "plugin:vue/essential",
        "plugin:@typescript-eslint/recommended",
        "eslint:recommended",
        "airbnb-base"
    ],
    "plugins": [
        "@typescript-eslint"
    ],

tsconfig.json

{
    "compilerOptions": {
        "target": "es2016",
        "lib": [
            "dom",
            "es2016"
        ],
        "module": "es6",
        "moduleResolution": "node",
        "esModuleInterop": true,
        "isolatedModules": false,
        "experimentalDecorators": true,
        "noImplicitAny": true,
        "noImplicitThis": true,
        "strictNullChecks": true,
        "removeComments": true,
        "emitDecoratorMetadata": true,
        "suppressImplicitAnyIndexErrors": true,
        "allowSyntheticDefaultImports": true,
        "sourceMap": true,
        "allowJs": true,
        "baseUrl": ".",
        "paths": {
            "my-secret-project-name/*": ["src/*"],
            "test-unit-helpers/*": ["test/unit/helpers/*"]
        }
    },
    "include": [
        "./**/*",
        "global.d.ts"
    ],
    "exclude": [
        "build/**/*.ts"
    ]
}

For me it's a 1.5 minutes vs 15 minutes difference

@bradzacher
Copy link
Member

bradzacher commented Apr 6, 2019

Questions for @kamok, @bbugh, @deser, @andrewisaburden

  • How many files are there?
    • more files = more time spent running rules + more time parsing w/ typescript)
  • Approx how many lines of code are there across the codebase?
    • more lines = more time spent parsing w/ typescript
  • Are you linting a multi tsconfig repo?
    • more tsconfigs = more project roots to construct and parse from = more time parsing w/ typescript
  • Is it a mixed codebase (i.e js + ts linted together)?

A small perf impact is expected, and unavoidable, because typescript has to parse and analyse your entire project.
This perf impact should be relative to the size of your project, and should be about the same as a standard tsc build time (note this impact should only be incurred once at the start of every CLI run).

These times are looking longer than that expected perf impact though - hard to say exactly without more info / testing.

Question for all users in this thread:

  • Are you working on private codebases, or do you have public github repos?
    • Without real world codebases, it is hard for us to investigate the root cause without doing npm releases and waiting for feedback (which obv involves a lot of trial and error, and long wait times).
  • What is the runtime of your lint, both with and without type information?

It might be worth us looking into the .tsbuildinfo feature that was released in 3.4

@bbugh
Copy link

bbugh commented Apr 6, 2019

@kamok How are you running eslint?
Almost the same as you, we have some javascript files though:

yarn eslint app/javascript/**/*.{js,ts,vue}


@bradzacher

Thanks for taking a look at this!

I'm not actually sure what happened... in the midst of running experiments to reply to this message, it's now running significantly faster than before. Now it's about 30-45 seconds (with package set) instead of 7 minutes. I'm not sure what could have changed, as I haven't touched experimental branch since earlier when I posted this. I did run multiple tests that were 5+ long when I was testing, so I know it wasn't just a fluke.

I've tried to reproduce the earlier issue, and I can't now... Sorry, I know intermittent failures are the worst. Now I've tried it with and without package, cleaning cache, with and without --fix and it's either 3 seconds (no package) or 30 seconds (with package).

I'll go ahead and answer your questions anyway, since I did have the issue earlier.

How many files are there?

We have 126 files (js, vue, ts). 32 of those are .ts and another 15 are vue files with lang="ts".

Approx how many lines of code are there across the codebase?

TypeScript - 591
TypeScript Vue component - 2425
Non-TS Vue component - 5380
JavaScript - 1484

(Much of the vue code is html/css, so those numbers are inflated for these measurements, but I couldn't find an easy way to extract just the line count for the script tags)

Are you linting a multi tsconfig repo?

I have a tsconfig.json in my spec folder, but I confirmed that removing that file doesn't make the linting any faster.

Is it a mixed codebase (i.e js + ts linted together)?

Yes, as I'm sure is clear from the line item counts, this is a mostly javascript application that is incrementally adopting TypeScript.

Are you working on private codebases, or do you have public github repos?

Unfortunately, it's a private repo.

Happy to offer anything else I can. If anything changes on my end I'll report again.

teppeis added a commit to cybozu/duck that referenced this issue Apr 7, 2019
Rules which require type information are too slow.
Dislable them with removing parserOptions > project until the
performance issue is fixed.

Also @see typescript-eslint/typescript-eslint#389
@deser
Copy link

deser commented Apr 8, 2019

@bradzacher

  1. Private

image
It's mostly js files, number of ts files is small (approx 30).

Is it a mixed codebase (i.e js + ts linted together)?
Yes.

Maybe the issue with this parser is that it tries to parse js file as ts?

@karthikiyengar
Copy link

Can confirm exponentially poor performance with project references in a mixed JS + TS environments in comparison to https://github.com/fimbullinter/wotan which supports project refs natively.

@kamok
Copy link

kamok commented Apr 8, 2019

Running eslint with npx eslint 'src/**/*.{ts,vue}'

How many files are there?

git ls-files | grep ".ts$" | wc -l    
263
git ls-files | grep ".js$" | wc -l    
32
git ls-files | grep ".vue$" | wc -l    
88

Approx how many lines of code are there across the codebase?

git ls-files | grep "src/" |grep -Ei "ts$|js$|vue$" | xargs wc -l
...
39989 total

Are you linting a multi tsconfig repo?

Yes we have 3 tsconfigs
One in /e2e folder
One in called tsconfig.prod.json
One called tsconfig on root

Is it a mixed codebase (i.e js + ts linted together)?

Yes, but I think we're not linting js files according to the command I run

@bradzacher
Copy link
Member

@karthikiyengar (and really, anyone new who is going to comment in this thread) please leave some more information in regards to your project.

If you want to let us know that you too are experiencing slowness, then either:

  • 👍 the OP so that you don't create notification spam for anyone, or
  • comment, but answer the questions that have been asked, so that we can gather as much information as possible.

Just a heads up for people:

  • Passing multiple tsconfigs into the parserOptions.projects field will exponentially increase lint times, as each tsconfig is treated as a different project from the perspective of typescript's parser.
    • note that just having multiple tsconfigs will not affect perf, unless you specifically tell us about them in the eslint config.
  • Linting a project with our parser when using type information will be slower than a lint run without type information.
    • This is entirely expected due to having to wait for typescript to parse your codebase. Though we are working to reduce this time as much as possible.

Temporary workaround for those with mixed codebases where TS is the minority: consider creating a separate config, and use package.json scripts to run the type information requiring rules only on typescript files (i.e. like "lint": "eslint -c .eslintrc.js src/**/*.js && eslint -c ts.eslintrc.js src/**/*.ts")

We're looking into the slowness, and will post back when we have more information.

@jcollum-nike
Copy link

jcollum-nike commented Apr 18, 2019

I decided to work around this by just having two lint paths:

    "lint": "npm run lint:js && npm run lint:ts",
    "lint:js": "eslint \"src/**/*.js\"",
    "lint:ts": "eslint -c .tslintrc \"src/**/*.@(ts|tsx)\"",

Took total lint time from 130s to 18s.

Totally new to TS so maybe this is a "bad idea".

@domoritz
Copy link
Contributor

I just started typing to switch from tslint to eslint for vega-lite and performance is incredibly bad. Just linting a single file takes 10 seconds and I have not been able to lint my whole project yet.

I am using eslint and prettier together.

You can follow my progress at vega/vega-lite#4901.

@JamesHenry
Copy link
Member

@domoritz Please could you try separating your JS and TS linting into separate steps and let us know how you get on?

@domoritz
Copy link
Contributor

But I don't have any JavaScript in my project.

@domoritz
Copy link
Contributor

Eslint times out after 10 minutes ("No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.") in https://travis-ci.org/vega/vega-lite/jobs/522604028.

@heikomat
Copy link

@domoritz your codebase pretty big. running npm run eslintbase takes around 2 minutes 27 seconds on my macbook, reporting with 38400 errors, most of wich are about indentation.
However, disabling @typescript-eslint/no-for-in-array and removing project: "tsconfig.json", from the parerOptions made eslint finish on that same laptop in about 14 seconds:

Screenshot 2019-04-21 at 06 04 17

A wild guess of mine would be, that your travis is not running on SSDs, resulting in hugely increased io-time, but your problem is definitely with the @typescript-eslint-plugin and the rules that require type information (and therefore the project-config in the parserOptions)

@domoritz
Copy link
Contributor

@heikomat Thank you for looking into this! My macbook is 5 years old so I guess it's a bit slower ;-).

I will probably turn off the indentation checks since prettier takes care of them already so that should reduce the number of errors. Now as you point out the issue is using project: "tsconfig.json". I would love to be able to use linting rules that require the typechecker (as we are doing with tslint right now) but if linting takes multiple minutes, that's too long. Are you suggesting that I follow your suggestions temporarily until the performance issues are addressed or are you proposing this as a general solution (I suspect the former but want to make sure I'm not misunderstanding you)?

@heikomat
Copy link

@domoritz i too would prefer to be able to use rules that require type information. In my opinion however, the performance tradeoff is not worth the one or two rules that require type information. So my suggestion is indeed to disable these rules until handling type information becomes fast enough.

i think some performance impact is definitely understandable considering that gathering type information does take some time, but ~1000% is just way to much

@JamesHenry
Copy link
Member

Thanks @domoritz I have forked your repo to this org in order to capture at this moment in time. We have been hoping to have a large real-world project to use for testing, so this is super useful.

I have opened a PR to do some basic initial infrastructure to bring down the commit and run the commands: #458

@domoritz
Copy link
Contributor

@JamesHenry I'm glad I can help. We have used tslint until now without any performance issues. That means, it is possible to do linting with type checking in a reasonable time (not saying it's easy but it is possible). Let me know if you have any questions.

@dotlouis
Copy link

dotlouis commented Apr 25, 2019

Hello, I'm having the exact same issue here 😭

Command I use to lint the codebase:
eslint 'src/**/*.{js,jsx,ts,tsx}'

⏳ Time linting the codebase

  • without parserOptions.project : ~4s
  • with parserOptions.project: ~100s

How many files are there?

110 Total, 105 JS(x), 5 TS(x)
relatively small I'd say

Approx how many lines of code are there across the codebase?

Approximately 13388 loc

Are you linting a multi tsconfig repo?

No

Is it a mixed codebase (i.e js + ts linted together)?

Yes, We're migrating from JS to TS and transforming files one at a time

A small perf impact is expected, and unavoidable, because typescript has to parse and analyse your entire project.
This perf impact should be relative to the size of your project, and should be about the same as a standard tsc build time (note this impact should only be incurred once at the start of every CLI run).

Sure, this is the time it takes to run tsc on the project: ~10s

Are you working on private codebases, or do you have public github repos?

It's private, but let me know if I can provide you with any more help.

Without real world codebases, it is hard for us to investigate the root cause without doing npm releases and waiting for feedback (which obv involves a lot of trial and error, and long wait times).

I've taken a look at the --debug log. I extracted the pattern that seems to change between the two versions

without parserOptions.project full log on pastebin

eslint:linter Linting code for /Users/louis/Dev/website/website-front/src/util/index.js (pass 1) +0ms
eslint:linter Generating fixed text for /Users/louis/Dev/website/website-front/src/util/index.js (pass 1) +22ms
eslint:source-code-fixer Applying fixes +22ms
eslint:source-code-fixer shouldFix parameter was false, not attempting fixes +0ms

(on a side note I don't get why the log say Applying fix although the next line, it says it will not apply them)

with parserOptions.project full log on pastebin

eslint:linter Linting code for /Users/louis/Dev/website/website-front/src/util/index.js (pass 1) +2ms
eslint:linter Generating fixed text for /Users/louis/Dev/website/website-front/src/util/index.js (pass 1) +821ms
eslint:source-code-fixer Applying fixes +823ms
eslint:source-code-fixer shouldFix parameter was false, not attempting fixes +0ms

the second line takes as much as 37 times the duration (821ms vs 22ms), and this for every file linted.

What is the runtime of your lint, both with and without type information?

eslint --version 5.16.0
tsc --version 3.4.5

.eslintrc

{
  "parser": "@typescript-eslint/parser",
  "parserOptions": {
    "project": "./tsconfig.json"  // >> this is the line I removed for my tests.
  },
  "plugins": [
    "@typescript-eslint",
    "react",
    "react-hooks",
    "jsx-a11y",
    "cypress"
  ],
  "env": {
    "browser": true,
    "node": true,
    "es6": true,
    "cypress/globals": true
  },
  "extends": [
    "plugin:@typescript-eslint/recommended",
    "plugin:react/recommended",
    "plugin:jsx-a11y/recommended",
    "prettier",
    "prettier/@typescript-eslint",
    "prettier/react",
    "plugin:cypress/recommended"
  ],
  "rules": {
    "react/prop-types": "off",
    "react/jsx-uses-vars": "warn",
    "react-hooks/rules-of-hooks": "error",
    "jsx-a11y/no-autofocus": 0,
    "no-console": ["error", { "allow": ["info", "warn", "error"] }],
    "jsx-a11y/no-noninteractive-element-interactions": "warn",
    "jsx-a11y/click-events-have-key-events": "warn",
    "@typescript-eslint/no-use-before-define": "off",
    "@typescript-eslint/camelcase": "off",
    "@typescript-eslint/explicit-function-return-type": [
      "error",
      { "allowExpressions": true, "allowTypedFunctionExpressions": true }
    ],
    "@typescript-eslint/no-explicit-any": "off"
  }
}

tsconfig.json

{
  "compilerOptions": {
    "baseUrl": "src",
    "strict": true,
    "jsx": "preserve",
    "module": "esnext",
    "moduleResolution": "node",
    "lib": ["dom", "dom.iterable", "esnext", "scripthost"],
    "allowSyntheticDefaultImports": true,
    "noUnusedLocals": true,
    "noUnusedParameters": true,
    "pretty": true,
    "noEmit": true
  }
}

Thanks for your time 🙏 . Btw, we should rename the issue because it does not reflect what we're seing. There is a perf problem even though some projects aren't considered large

UPDATE: since the issue has been closed, the issue has been resolved for my case. Thanks 🎉

@deser
Copy link

deser commented Jul 23, 2019

We've identified a source of slowness, and we're looking into a way to handle it.

As a stop gap for anyone who is experience slowness (and not using vue... sorry, but vue parsing is its own kettle of fish i.e. #404), the following might help mitigate it for you:

Please make sure that you include all files to be linted in your tsconfig.

i.e. for the following lint command:

eslint '{src,test,typings}/**/*.ts'
you should have the following tsconfig:

{
  // insert your config here.....
  // a good way is by extending your base config to create tsconfig.eslint.json which you pass into your eslint config
  // "extends": "./tsconfig.json",
  "include": [
    "src/**/*.ts",
    "test/**/*.ts",
    "typings/**/*.ts",
  ],
}

For each file that is not included in your tsconfig, it will cause the parse to slow down by a large factor.

Again, we're looking into a fix for this now that we've identified a problem.
Thank you to everyone who has provided repos and diagnostics so far.

There is also the existing issue with eslint-plugin-import which is waiting to be merged (benmosher/eslint-plugin-import#1409) - you could use something like patch-package to fix it locally in the meantime.

Btw, I have mixed project (js\ts). Does it mean that I need to add js to the include of tsconfig?

@bradzacher
Copy link
Member

after making sure that files in tsconfig match to those which are linted seems performance has been fixed.

🎉 🎉 🎉

Btw, I have mixed project (js\ts). Does it mean that I need to add js to the include of tsconfig?

I believe so. Which is why I suggested creating a brand new tsconfig for eslint - so that you can include all of your files without messing with your build / ide experience.

@deser
Copy link

deser commented Jul 23, 2019

Many thanks. Is there feeling when #389 (comment) is going to be fixed? :)

@bradzacher
Copy link
Member

It'll be worked on soon™ (this week or next week potentially).

@ulrichb
Copy link
Contributor

ulrichb commented Jul 26, 2019

Here are my results in my codebase:

image

@typescript-eslint/no-misused-promises and @typescript-eslint/no-floating-promises are behaving pretty bad 😢

@waitingsong
Copy link

waitingsong commented Aug 1, 2019

It's my test:

  "dependencies": {
    "eslint": "^6.1.0",
    "@typescript-eslint/eslint-plugin": "^1.13.0",
    "@typescript-eslint/eslint-plugin-tslint": "^1.13.0",
    "@typescript-eslint/parser": "^1.13.0",
    "eslint-plugin-import": "^2.18.0",
    "eslint-plugin-unicorn": "^9.1.1"
  },

typescript: v3.5.3
node.js: v10.16.0

PC: amd 2700X, SSD
The project contains only one file, index.ts:

import * as ts from 'typescript'


export default function() {
  const transformer = <T extends ts.Node>(context: ts.TransformationContext) => (rootNode: T) => {
    function visit(node: ts.Node): ts.Node {
      console.log('Visiting ' + ts.SyntaxKind[node.kind])

      node = ts.visitEachChild(node, visit, context)
      if (node.kind === ts.SyntaxKind.BinaryExpression) {
        void 0
      }

      return node
    }
    return ts.visitNode(rootNode, visit)
  }

  return transformer
}

eslint.conf:

module.exports = {
  env: {
    browser: true,
    commonjs: true,
    es6: true,
    node: true,
  },
  extends: [
      './plugins/import.yml',
      'eslint:recommended',
      'plugin:@typescript-eslint/eslint-recommended',
      'plugin:@typescript-eslint/recommended'
  ],
  globals: {
    Atomics: 'readonly',
    document: 'readonly',
    globalThis: 'readonly',
    navigator: 'readonly',
    SharedArrayBuffer: 'readonly',
    window: 'readonly',
  },
  parser: '@typescript-eslint/parser',
  parserOptions: {
    ecmaFeatures: {
      globalReturn: false,
      impliedStrict: true,
    },
    ecmaVersion: 2019,
    project: 'tsconfig.json',
    sourceType: 'module',
  },
  plugins: ['@typescript-eslint', 'unicorn'],
  root: true,
}
 

Run eslint src/**/*.ts

real    0m36.691s
user    0m0.015s
sys     0m0.136s

Disable the following rules of ./plugins/import.yml

  import/namespace: 0
  import/no-cycle: 0
  import/no-deprecated: 0

cost !:

real    0m2.741s
user    0m0.015s
sys     0m0.120s

36.691s decreases to 2.741s !

And then remove entry parserOptions.project , it cost

real    0m1.879s
user    0m0.030s
sys     0m0.151s

wish helpful for someone having bad performance .

@bradzacher
Copy link
Member

I'm going to close this now.

We just released v2.0.0 which includes #760, so it will be much easier to tell when you have a config which will perform poorly.

For sure there are some things we can do to improve perf in the future, but the majority case should be handled now.

@fabb
Copy link

fabb commented Aug 25, 2019

In our project, the average eslint time is around 27s, which is ok. But we also use husky with lint-staged to run eslint and others on every commit. Even if we just commit one single file, it will always take around 12s just for the eslint part, which is too much for committing. (Worse, imagine rebases where this is executed for every single rebased commit).

I found out that the culprit are not single rules, but all rules that depend on type information. As a quick hack for resolving our commit times, I created a separate .eslintrc.githook.js:

module.exports = {
    extends: ['.eslintrc.js'],
    parserOptions: {
        // disable rules that depend on type information - these would slow down git committing too much, rather rely on PR branch builds for these
        project: null,
    },
    rules: {
        '@typescript-eslint/tslint/config': 'off',
    },
}

lint-staged is configured to use this configuration:

"eslint --fix --ext js,jsx,ts,tsx -c .eslintrc.githook.js"

For automatic PR builds, I continue to use the full eslint rule set, so violations will still break my builds, same as compile errors.

The same applies for the VSCode eslint plugin, it will also use the full rule set and show me violations in the editor.

While this works, it would be awesome if typescript-eslint could somehow cache the type information between runs, similar to typescript's new incremental mode.

dpinol added a commit to hubtype/botonic that referenced this issue Oct 14, 2019
jest 25 (npm i jest@next) is much faster than 24.x
don't specify parserOptions.project (typescript-eslint/typescript-eslint#389)
eslint_d eliminates eslint startup time
dpinol added a commit to hubtype/botonic that referenced this issue Oct 18, 2019
jest 25 (npm i jest@next) is much faster than 24.x
don't specify parserOptions.project (typescript-eslint/typescript-eslint#389)
eslint_d eliminates eslint startup time
dpinol added a commit to hubtype/botonic that referenced this issue Oct 22, 2019
jest 25 (npm i jest@next) is much faster than 24.x
don't specify parserOptions.project (typescript-eslint/typescript-eslint#389)
eslint_d eliminates eslint startup time
dpinol added a commit to hubtype/botonic that referenced this issue Oct 22, 2019
jest 25 (npm i jest@next) is much faster than 24.x
don't specify parserOptions.project (typescript-eslint/typescript-eslint#389)
eslint_d eliminates eslint startup time
@khaledosman
Copy link

khaledosman commented Oct 24, 2019

if you came here because your tslint/typescript setup is slow, I recommend you to try the following:

  1. Setup an .eslintignore file to ignore node_modules as well as build / non-typescript files
  2. Do the same thing for tsconfig.json via the "exclude" property to exclude node_modules or non typescript files.
  3. use --cache flag when running eslint, example: eslint --cache **/*.ts
  4. if you're using webpack also exclude node_modules/others from there, add fork-ts-checker-webpack-plugin to run type checking on a separate process. And use the new watch API for incremental builds:
{
      loader: 'ts-loader',
      exclude: [
            path.resolve(__dirname, 'node_modules')
         ],
      options: {
        transpileOnly: true,
        experimentalWatchApi: true,
      },
....
plugins: [
    new ForkTsCheckerWebpackPlugin({
      eslint: true,
      eslintOptions: {
        cache: true
      }
    })
  ]
  1. optimize your webpack build performance via these tips
  2. don't extend the expensive "plugin:@typescript-eslint/recommended-requiring-type-checking" configuration in your eslint config

guinii pushed a commit to hubtype/botonic that referenced this issue Jan 10, 2020
jest 25 (npm i jest@next) is much faster than 24.x
don't specify parserOptions.project (typescript-eslint/typescript-eslint#389)
eslint_d eliminates eslint startup time
@duncanleung
Copy link

@khaledosman Thanks for the tips!

I'm wondering, does typescript-eslint run through webpack?

Setting up webpack module loader exclude rules, and using fork-ts-checker-webpack-plugin sounds interesting, but I'm not seeing where typescript-eslint would use webpack configs and plugins?

I'm thinking that .eslintignore and .tsconfig.json includes/excludes should be the files handling what files/directories that typescript-eslint will parse through?

@khaledosman
Copy link

khaledosman commented Jan 10, 2020

@duncanleung right, fork-ts-checker has an argument to run eslint that is documented in their README.

In my personal setup I even commented out fork-ts-checker and left type-checking disabled for more performance, since I didn't really need it or linting because I was able to see the issues in my VSCode editor right away via their extensions. I also added a git commit hook via husky to lint the project before commits and make sure I don't commit something unlinted.

I just wrote this article which includes more details about my setup, if you're interested.

@johnnyreilly
Copy link
Contributor

Hey @duncanleung,

I work on fork-ts-checker-webpack-plugin and ts-loader.

I'm wondering, does typescript-eslint run through webpack?

If you turn on ESLint with fork-ts-checker-webpack-plugin it's an entirely plug and play experience. Behind the scenes ESLint is being invoked and it is powered by the same mechanisms as ESLint in VS Code etc.

I blogged about this and this may be helpful to you:

https://blog.johnnyreilly.com/2019/07/typescript-and-eslint-meet-fork-ts-checker-webpack-plugin.html

@duncanleung
Copy link

duncanleung commented Jan 13, 2020

@khaledosman @johnnyreilly thank you for your replies - my apologies; I didn't word my question correctly 🙈.

You are right - the fork-ts-checker-webpack-plugin README is very clearly written that it works with ESLint. Cheers for the awesome docs on that library 👍.

What I was originally intending to ask:

Is there was a way for a npm run lint script to take advantage of the optimizations in fork-ts-checker-webpack-plugin.

TL;DR:

After a bit more reading, I realize I can use fork-ts-checker-webpack-plugin and have webpack run ESLint in the background at all times through eslint-loader.

This can replace our gated git pre-push hook.

Original Problem - Trying to improve typescript-eslint speed during husky pre-push hook

To give some context:
My team currently ensures code quality through a gated git push with lint-prepush on husky pre-push.

The pre-push hook runs npm run lint and npm run test on newly committed files. Any failing lint / tests will prevent git push.

However, lint speed is slow, and I was trying to find ways to optimize the speed. I thought somehow I could run fork-ts-checker-webpack-plugin on this pre-push hook.

I realize now that since webpack can continually run eslint in the background through eslint-loader, this can achieve the same code quality goal.

I do like the explicit gated push if lint fails, but the improved lint speed might be a better developer experience overall.

Thanks again for everyone's feedback.

@johnnyreilly
Copy link
Contributor

johnnyreilly commented Jan 13, 2020

Hey @duncanleung

Happy to help! One thing that's worth clarifying:

After a bit more reading, I realize I can use fork-ts-checker-webpack-plugin and have webpack run ESLint in the background at all times through eslint-loader.

fork-ts-checker-webpack-plugin does not need eslint-loader to run ESLint. You can use fork-ts-checker-webpack-plugin to perform ESLinting and type checking. Alternatively you could use fork-ts-checker-webpack-plugin to just perform type checking and have eslint-loader running alongside doing linting. I suspect having fork-ts-checker-webpack-plugin performing both tasks is more performant but I haven't measured.

If you're looking for ways to speed up ESLint performance I suggest using the eslintOptions option with (I think) { cache: true }.

This will generate cached ESLint checks locally which your precommit hook can take advantage of as well I believe.

        new ForkTsCheckerWebpackPlugin({
          eslint: true,
          eslintOptions: {
            cache: true,
          }
        }

ericmarcos pushed a commit to hubtype/botonic that referenced this issue Feb 3, 2020
jest 25 (npm i jest@next) is much faster than 24.x
don't specify parserOptions.project (typescript-eslint/typescript-eslint#389)
eslint_d eliminates eslint startup time
vanbasten17 pushed a commit to hubtype/botonic that referenced this issue Feb 8, 2020
jest 25 (npm i jest@next) is much faster than 24.x
don't specify parserOptions.project (typescript-eslint/typescript-eslint#389)
eslint_d eliminates eslint startup time
@typescript-eslint typescript-eslint locked as resolved and limited conversation to collaborators Feb 21, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working package: eslint-plugin Issues related to @typescript-eslint/eslint-plugin
Projects
None yet
Development

No branches or pull requests