Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use git stashes for gitWorkflow #663

Merged
merged 37 commits into from
Oct 31, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
bfd2adc
feat: use git stashes for gitWorkflow
iiroj Jul 21, 2019
bdf0faf
ci: print git version number in AppVeyor
iiroj Jul 21, 2019
5c165fa
refactor: simplify gitWorkflow by creating only one stash
iiroj Jul 21, 2019
b008ab8
test: update test for restoring changes
iiroj Jul 21, 2019
30939b9
fix: retry failing apply with 3-way merge
iiroj Jul 21, 2019
8ec8bfb
refactor: improvements
iiroj Jul 22, 2019
340c1c2
refactor: GitWorkflow is a class
iiroj Jul 22, 2019
31e440e
test: increase jest timeout for slow CI runners
iiroj Jul 23, 2019
080f1c6
fix: try applying unstaged changes before handling errors
iiroj Aug 17, 2019
e8d5e27
test: add some runAll tests
iiroj Aug 17, 2019
7b798fd
test: add test for failing merge conflict resolution
iiroj Sep 10, 2019
89e6de0
refactor: use execa.command to avoid parsing of commands
iiroj Sep 11, 2019
959d9d9
fix: gitWorkflow handles active merge mode
iiroj Sep 11, 2019
66381f0
test: move mock files inside test directory
iiroj Sep 12, 2019
e535e76
test: rework usage of tmp module
iiroj Sep 12, 2019
d087155
refactor: no need to use path.resolve since path is normalized
iiroj Sep 12, 2019
af77588
refactor: check for merge using fs.access before fs.readfile
iiroj Sep 13, 2019
3d8c753
refactor: move file operations to separate file
iiroj Sep 13, 2019
45688b2
test: scope global jest timeout to only runAll.unmocked.spec
iiroj Sep 13, 2019
6239830
test: remove non-functioning, throwing tests
iiroj Sep 13, 2019
d2e4253
refactor: rename src/ to lib/ since it's not compiled
iiroj Sep 13, 2019
dded921
refactor: improve long argument warning indentation
iiroj Sep 13, 2019
647abd7
refactor: show helpful warning about git failures
iiroj Sep 13, 2019
05f53b3
test: do not use tmp for creation of tmp directories
iiroj Sep 13, 2019
b591ba8
test: add mock console
iiroj Sep 13, 2019
19e6c11
test: further increase timeout for long test
iiroj Sep 13, 2019
d20c5be
fix: keep untracked files around by backing them up
iiroj Sep 15, 2019
0018949
refactor: improvements based on PR review
iiroj Sep 26, 2019
e3da610
docs: remove bit about next release
iiroj Oct 2, 2019
50afea0
fix: workaround for stashing deleted files for git < 2.23
iiroj Oct 2, 2019
35b0616
test: should work when amending previous commit with unstaged changes
iiroj Oct 3, 2019
de82960
test: improve coverage of gitWorkflow
iiroj Oct 3, 2019
7d0379d
feat: automatically stage task modifications
iiroj Oct 21, 2019
4014f3c
feat: warn when task contains "git add"
iiroj Oct 23, 2019
473b93b
test: add test for supplying object config via node api
iiroj Oct 23, 2019
0111f48
fix: correctly restore untracked files from backup stash
iiroj Oct 29, 2019
2b57db0
fix: prevent Listr from hiding git add warning
iiroj Oct 30, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions .appveyor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ cache:
test_script:
- node --version
- yarn --version
- git --version
- yarn test

branches:
Expand Down
6 changes: 3 additions & 3 deletions .lintstagedrc.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"*.{js,json,md}": ["prettier --write", "git add"],
"*.js": ["npm run lint:base --fix", "git add"],
".*{rc, json}": ["jsonlint --in-place", "git add"]
"*.{js,json,md}": "prettier --write",
"*.js": "npm run lint:base --fix",
".*{rc, json}": "jsonlint --in-place"
}
79 changes: 27 additions & 52 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,30 +2,6 @@

Run linters against staged git files and don't let :poop: slip into your code base!

---

## 🚧 Help test `lint-staged@next`!

Version 10 of `lint-staged` is coming with changes that help it run faster on large git repositories and prevent loss of data during errors. Please help test the `next` version and report any inconsistencies in our [GitHub Issues](https://github.com/okonet/lint-staged/issues):

**Using npm**

npm install --save-dev lint-staged@next

**Using yarn**

yarn add -D lint-staged@next

### Notable changes

- A git stash is created before running any tasks, so in case of errors any lost changes can be restored easily (and automatically unless lint-staged itself crashes)
- Instead of write-tree/read-tree, `lint-staged@next` uses git stashes to hide unstaged changes while running tasks against staged files
- This results in a performance increase of up to 45x on very large repositories
- The behaviour of committing modifications during tasks (eg. `prettier --write && git add`) is different. The current version creates a diff of these modifications, and applies it against the original state, silently ignoring any errors. The `next` version leaves modifications of staged files as-is, and then restores all hidden unstaged changes as patch. If applying the patch fails due to a merge conflict (because tasks have modified the same lines), a 3-way merge will be retried. If this also fails, the entire commit will fail and the original state will be restored.
- **TL;DR** the `next` version will never skip committing any changes by tasks (due to a merge conflict), but might fail in very complex situations where unstaged changes cannot be restored cleanly. If this happens to you, we are very interested in a repeatable test scenario.

---

[![asciicast](https://asciinema.org/a/199934.svg)](https://asciinema.org/a/199934)

## Why
Expand All @@ -36,10 +12,10 @@ This project contains a script that will run arbitrary shell tasks with a list o

## Related blogs posts and talks

* [Make Linting Great Again](https://medium.com/@okonetchnikov/make-linting-great-again-f3890e1ad6b8#.8qepn2b5l)
* [Running Jest Tests Before Each Git Commit](https://benmccormick.org/2017/02/26/running-jest-tests-before-each-git-commit/)
* [AgentConf: Make Linting Great Again](https://www.youtube.com/watch?v=-mhY7e-EsC4)
* [SurviveJS Interview](https://survivejs.com/blog/lint-staged-interview/)
- [Make Linting Great Again](https://medium.com/@okonetchnikov/make-linting-great-again-f3890e1ad6b8#.8qepn2b5l)
- [Running Jest Tests Before Each Git Commit](https://benmccormick.org/2017/02/26/running-jest-tests-before-each-git-commit/)
- [AgentConf: Make Linting Great Again](https://www.youtube.com/watch?v=-mhY7e-EsC4)
- [SurviveJS Interview](https://survivejs.com/blog/lint-staged-interview/)

> If you've written one, please submit a PR with the link to it!

Expand Down Expand Up @@ -79,22 +55,22 @@ Options:
-h, --help output usage information
```

* **`--config [path]`**: This can be used to manually specify the `lint-staged` config file location. However, if the specified file cannot be found, it will error out instead of performing the usual search. You may pass a npm package name for configuration also.
* **`--relative`**: By default filepaths will be passed to the linter tasks as *absolute*. This flag makes them relative to `process.cwd()` (where `lint-staged` runs).
* **`--shell`**: By default linter commands will be parsed for speed and security. This has the side-effect that regular shell scripts might not work as expected. You can skip parsing of commands with this option.
* **`--quiet`**: By default `lint-staged` will print progress status to console while running linters. Use this flag to supress all output, except for linter scripts.
* **`--debug`**: Enabling the debug mode does the following:
* `lint-staged` uses the [debug](https://github.com/visionmedia/debug) module internally to log information about staged files, commands being executed, location of binaries, etc. Debug logs, which are automatically enabled by passing the flag, can also be enabled by setting the environment variable `$DEBUG` to `lint-staged*`.
* Use the [`verbose` renderer](https://github.com/SamVerschueren/listr-verbose-renderer) for `listr`.
- **`--config [path]`**: This can be used to manually specify the `lint-staged` config file location. However, if the specified file cannot be found, it will error out instead of performing the usual search. You may pass a npm package name for configuration also.
- **`--relative`**: By default filepaths will be passed to the linter tasks as _absolute_. This flag makes them relative to `process.cwd()` (where `lint-staged` runs).
- **`--shell`**: By default linter commands will be parsed for speed and security. This has the side-effect that regular shell scripts might not work as expected. You can skip parsing of commands with this option.
- **`--quiet`**: By default `lint-staged` will print progress status to console while running linters. Use this flag to supress all output, except for linter scripts.
- **`--debug`**: Enabling the debug mode does the following:
- `lint-staged` uses the [debug](https://github.com/visionmedia/debug) module internally to log information about staged files, commands being executed, location of binaries, etc. Debug logs, which are automatically enabled by passing the flag, can also be enabled by setting the environment variable `$DEBUG` to `lint-staged*`.
- Use the [`verbose` renderer](https://github.com/SamVerschueren/listr-verbose-renderer) for `listr`.

## Configuration

Starting with v3.1 you can now use different ways of configuring it:

* `lint-staged` object in your `package.json`
* `.lintstagedrc` file in JSON or YML format
* `lint-staged.config.js` file in JS format
* Pass a configuration file using the `--config` or `-c` flag
- `lint-staged` object in your `package.json`
- `.lintstagedrc` file in JSON or YML format
- `lint-staged.config.js` file in JS format
- Pass a configuration file using the `--config` or `-c` flag

See [cosmiconfig](https://github.com/davidtheclark/cosmiconfig) for more details on what formats are supported.

Expand Down Expand Up @@ -128,19 +104,19 @@ So, considering you did `git add file1.ext file2.ext`, lint-staged will run the

Linter commands work on a subset of all staged files, defined by a _glob pattern_. `lint-staged´ uses [micromatch](https://github.com/micromatch/micromatch) for matching files with the following rules:

* If the glob pattern contains no slashes (`/`), micromatch's `matchBase` option will enabled, so globs match a file's basename regardless of directory:
* **`"*.js"`** will match all JS files, like `/test.js` and `/foo/bar/test.js`
* **`"!(*test).js"`**. will match all JS files, except those ending in `test.js`, so `foo.js` but not `foo.test.js`
* If the glob pattern does contain a slash (`/`), it will match for paths as well:
* **`"/*.js"`** will match all JS files in the git repo root, so `/test.js` but not `/foo/bar/test.js`
* **`"foo/**/*.js"`** will match all JS files inside the`/foo`directory, so`/foo/bar/test.js`but not`/test.js`
- If the glob pattern contains no slashes (`/`), micromatch's `matchBase` option will enabled, so globs match a file's basename regardless of directory:
- **`"*.js"`** will match all JS files, like `/test.js` and `/foo/bar/test.js`
- **`"!(*test).js"`**. will match all JS files, except those ending in `test.js`, so `foo.js` but not `foo.test.js`
- If the glob pattern does contain a slash (`/`), it will match for paths as well:
- **`"/*.js"`** will match all JS files in the git repo root, so `/test.js` but not `/foo/bar/test.js`
- **`"foo/**/\*.js"`** will match all JS files inside the`/foo`directory, so`/foo/bar/test.js`but not`/test.js`

When matching, `lint-staged` will do the following

* Resolve the git root automatically, no configuration needed.
* Pick the staged files which are present inside the project directory.
* Filter them using the specified glob patterns.
* Pass absolute paths to the linters as arguments.
- Resolve the git root automatically, no configuration needed.
- Pick the staged files which are present inside the project directory.
- Filter them using the specified glob patterns.
- Pass absolute paths to the linters as arguments.

**NOTE:** `lint-staged` will pass _absolute_ paths to the linters to avoid any confusion in case they're executed in a different working directory (i.e. when your `.git` directory isn't the same as your `package.json` directory).

Expand All @@ -156,7 +132,7 @@ In advanced scenarios, where it is impossible to configure the linter task itsel

## What commands are supported?

Supported are any executables installed locally or globally via `npm` as well as any executable from your $PATH.
Supported are any executables installed locally or globally via `npm` as well as any executable from your \$PATH.

> Using globally installed scripts is discouraged, since lint-staged may not work for someone who doesn’t have it installed.

Expand Down Expand Up @@ -407,7 +383,7 @@ const lintStaged = require('lint-staged')
try {
const success = await lintStaged()
console.log(success ? 'Linting was successful!' : 'Linting failed!')
} catch(e) {
} catch (e) {
// Failed to load configuration
console.error(e)
}
Expand All @@ -426,7 +402,6 @@ const success = await lintStaged({

You can also pass config directly with `config` option:


```js
const success = await lintStaged({
config: {
Expand Down
2 changes: 1 addition & 1 deletion bin/lint-staged
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ require('please-upgrade-node')(

const cmdline = require('commander')
const debugLib = require('debug')
const lintStaged = require('../src')
const lintStaged = require('../lib')

const debug = debugLib('lint-staged:bin')

Expand Down
17 changes: 17 additions & 0 deletions lib/execGit.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
'use strict'

const debug = require('debug')('lint-staged:git')
const execa = require('execa')

module.exports = async function execGit(cmd, options = {}) {
debug('Running git command', cmd)
try {
const { stdout } = await execa('git', [].concat(cmd), {
...options,
cwd: options.cwd || process.cwd()
})
return stdout
} catch ({ all }) {
throw new Error(all)
}
}
51 changes: 51 additions & 0 deletions lib/file.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
'use strict'

const debug = require('debug')('lint-staged:file')
const fs = require('fs')

/**
* Check if file exists and is accessible
* @param {String} filename
* @returns {Promise<Boolean>}
*/
module.exports.checkFile = filename =>
new Promise(resolve => {
debug('Trying to access `%s`', filename)
fs.access(filename, fs.constants.R_OK, error => {
if (error) {
debug('Unable to access file `%s` with error:', filename)
debug(error)
} else {
debug('Successfully accesses file `%s`', filename)
}

resolve(!error)
})
})

/**
* @param {String} filename
* @returns {Promise<Buffer|Null>}
*/
module.exports.readBufferFromFile = filename =>
new Promise(resolve => {
debug('Reading buffer from file `%s`', filename)
fs.readFile(filename, (error, file) => {
debug('Done reading buffer from file `%s`!', filename)
resolve(file)
})
})

/**
* @param {String} filename
* @param {Buffer} buffer
* @returns {Promise<Void>}
*/
module.exports.writeBufferToFile = (filename, buffer) =>
new Promise(resolve => {
debug('Writing buffer to file `%s`', filename)
fs.writeFile(filename, buffer, () => {
debug('Done writing buffer to file `%s`!', filename)
resolve()
})
})
1 change: 0 additions & 1 deletion src/generateTasks.js → lib/generateTasks.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ const debug = require('debug')('lint-staged:gen-tasks')
* @param {boolean} [options.gitDir] - Git root directory
* @param {boolean} [options.files] - Staged filepaths
* @param {boolean} [options.relative] - Whether filepaths to should be relative to gitDir
* @returns {Promise}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should not be removed completely perhaps? @returns {array} - Array of tasks?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or even better with a shape of the task. but that's a nit

*/
module.exports = function generateTasks({
config,
Expand Down
File renamed without changes.