Node executable path always prepended to PATH when running a package.json script

[jasongrout]

Do you want to request a feature or report a bug?
Feature

What is the current behavior?

There is no way to eliminate the explicit prepending of the node executable directory to the PATH environment variable when running package.json scripts.

Npm has a --scripts-prepend-node-path cli switch and config option, which defaults to auto, which only prepends the node executable directory to the path if it is not in the path. Also, setting the option (i.e., to true) prepends the node executable path just before the existing PATH, but after all of the local .bin directories. See https://docs.npmjs.com/cli/run-script.

Yarn, on the other hand, always prepends the node executable path (

yarn/src/util/execute-lifecycle-script.js

Lines 145 to 147 in ae2de7b

	// Include the directory that contains node so that we can guarantee that the scripts
	// will always run with the exact same Node release than the one use to run Yarn
	pathParts.unshift(path.dirname(process.execPath));

). Furthermore, it redundantly prepends the node executable path to the very front of the new path (overriding any local .bin directories) if the --scripts-prepend-node-path switch is given (

yarn/src/util/execute-lifecycle-script.js

Lines 177 to 179 in ae2de7b

	if (config.scriptsPrependNodePath) {
	pathParts.unshift(path.join(path.dirname(process.execPath)));
	}

). This last override was recently added in #5787 to address #5776.

Yarn's behavior is problematic. For example, if node is installed at the system level (e.g., /usr/bin), this effectively prepends /usr/bin to the path with no recourse, which completely inverts and breaks a normal PATH ordering of having local directories override the global /usr/bin directory. We saw this break a package.json script that called out to python, for example (it can break python virtual environments). You can also imagine many other situations when inverting the PATH priority like this can break things.

Furthermore, the behavior introduced in #5787 seems problematic, in that the global node path now overrides any local .bin directories prepended to the path, directly contradicting the npm behavior (but having the same option name).

What is the expected behavior?
I think npm's behavior here makes sense:

1. By default, only prepend the node executable path to PATH if it is not already in PATH. And in that case, only prepend it just before the existing PATH, but after any local node .bin directories, etc (i.e., do the intelligent prepending at

   yarn/src/util/execute-lifecycle-script.js

   Lines 145 to 147 in ae2de7b

   	// Include the directory that contains node so that we can guarantee that the scripts
   	// will always run with the exact same Node release than the one use to run Yarn
   	pathParts.unshift(path.dirname(process.execPath));

   , and delete the prepending at

   yarn/src/util/execute-lifecycle-script.js

   Lines 177 to 179 in ae2de7b

   	if (config.scriptsPrependNodePath) {
   	pathParts.unshift(path.join(path.dirname(process.execPath)));
   	}

   ).
2. Make it possible to explicitly enable or disable this prepending using --scripts-prepend-node-path

CC @ivanov

Please mention your node.js, yarn and operating system version.
yarn 1.7.0
node 8.11.2
npm 5.6.0
macOS 10.13

[indirectlylit]

see also #5874

[jasongrout]

Thanks for the reference. That definitely seems like the same issue.

[jasongrout]

FYI, I think the node logic for auto-prepending the path is here: https://github.com/npm/npm/blob/4c65cd952bc8627811735bea76b9b110cc4fc80e/node_modules/npm-lifecycle/index.js#L163

[arcanis]

I'll fix this for the 1.9.0.

The goal of the faulty code is to force scripts executed through Yarn to use the same Yarn & npm than the ones currently executing Yarn - hijacking the PATH and affecting other binaries is an unfortunate side effects of the current implementation :(

[jasongrout]

Thanks!

[jasongrout]

@arcanis - We are trying to decide between downgrading yarn to avoid this bug, or waiting for the fix in 1.9.0. Do you happen to have any updates or timelines for a fix in 1.9.0?

[x-yuri]

I've been burnt by this trying to run ruby tests.

@jasongrout Consider the following workaround:

fix-path.sh:

#!/usr/bin/env bash
set -eu

s_path=$(printf "%s" "$PATH" | tr : \\n)
_IFS=$IFS
IFS=$'\n'
a_path=($s_path)
IFS=$_IFS

usr_bin=$(dirname -- "$(which node)")
n_usr_bin=$(egrep "^$usr_bin$" <(printf "%s" "$s_path") | wc -l)
r=()
for (( i = 0; i < ${#a_path[@]}; i++ )); do
    if [ "${a_path[$i]}" = "$usr_bin" ] && (( n_usr_bin > 1 )); then
        (( n_usr_bin-- ))
    else
        r+=("${a_path[$i]}")
    fi
done

PATH=$(
    for p in ${r[@]+"${r[@]}"}; do
        printf "%s\n" "$p"
    done | paste -sd:
)

"$@"

Then, in package.json:

"cmd1": "./fix-path.sh bin/rails test:system",
"cmd2": "./fix-path.sh bash -c 'echo $PATH' | tr : \\\\n"

[arcanis]

We are trying to decide between downgrading yarn to avoid this bug, or waiting for the fix in 1.9.0. Do you happen to have any updates or timelines for a fix in 1.9.0?

@jasongrout Hard to tell, I'm working on a large feature I really want to push in the 1.9.0 (and that includes the fix for this problem - which is a bit intricate so I'd prefer to avoid backporting it). I have hopes that I'll be able to publish it in a week or two.

[jasongrout]

Great, thanks for the update!

…

On Fri, Jun 29, 2018, 08:10 Maël Nison ***@***.***> wrote: We are trying to decide between downgrading yarn to avoid this bug, or waiting for the fix in 1.9.0. Do you happen to have any updates or timelines for a fix in 1.9.0? @jasongrout <https://github.com/jasongrout> Hard to tell, I'm working on a large feature I really want to push in the 1.9.0 (and that includes the fix for this problem - which is a bit intricate so I'd prefer to avoid backporting it). I have hopes that I'll be able to publish it in a week or two. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5935 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AALwZvKY4QrolwjSfaXDcNC3xlWNeEkBks5uBkN2gaJpZM4Ublla> .

[jasongrout]

@arcanis - I see yarn 1.9.0 was just released. Just curious, was a fix for this merged in 1.9.0?

[arcanis]

Unfortunately no, my feature isn't stable enough to have made the cut, and I don't have much time at the moment to work on a backport of this specific issue.

That said, if you're willing to work on a fix, we can merge it sooner (in short, what I do is: in execute-lifecycle-script, instead of adding the node binary path to the PATH, I instead create a temporary directory and symlink both node and yarn there. I then inject this temporary directory in the PATH).

[jasongrout]

Thanks for the update. I can't work on this in the next week, but if it's still open after that, I'll see about giving it a shot.

[jasongrout]

(as an update, it's not looking like I have very much available time to work on this, so anyone watching this: please go ahead and don't wait for me)

[bollwyvl]

Here's a very rough look at what I think you folks are saying (probably wrong):
bollwyvl@9288398#diff-939cc0d319704d4f0d98baf9167ba850R186

the tests do pass locally (on linux), but I don't think it's taking any effect right now, and haven't quite figured out how to test it.