Browser report fails when node_modules are read-only

[singles]

Description of a problem

In environments where node_modules are read-only browser report isn't created.

Examples of such environments: nix based builds, where node_modules land in /nix/store as read-only (555 or 444 rights).
Another one might be some stateless build systems, where node_modules is passed between build steps as an artifact and also might be read-only.

How to reproduce:

mkdir test && cd test
npm init -y && npm install backstopjs
./node_modules/.bin/backstop init
chmod -w ./node_modules/.bin/backstop/compare/output # simulate read-only case
./node_modules/.bin/backstop test
...

report | 2 Failed
       report | Writing browser report
       report | Failed writing report with error: undefined # <-- this

Potential cause

I dug a little into the internals and it looks that it's caused by

BackstopJS/core/command/report.js

Line 26 in ddfedcb

return fs.copy(config.comparePath, toAbsolute(config.html_report)).then(function () {

where fs.copy copies output with directory rights. Assuming I'm not wrong about how *nix systems handle permissions, in presented case, there's no write for output directory - thus no new directories html_report directory on the project side can be created as part of copy.

Workarounds

One workaround is to create directory structure before running tests, something like:

rm -rf backstop_data/html_report && mkdir -p backstop_data/html_report/fonts/assets

Summary

Not sure whether it should be considered as a bug, I just wanted to point out that current implementation might be causing some issues.

[lo1tuma]

I’ve stumbled upon the same problem. Additionally when creating the report fails the process exits with 0, which made our CI pipeline silently pass, even though the tests were actually failing.

[lo1tuma]

This might be an issue of fs-extra which preservers the permissions of copied files.

[XjSv]

I am also just running into this problem when packaging in using Electron. The files get packaged in \release\build\win-ia32-unpacked\resources\app.asar\node_modules\backstopjs\compare\output which is a read-only archive.

[XjSv]

I am using the code from #1400 for my Electron project and I am passing the comparePath in my config like so:

"comparePath": ${backstopDataDir}/compare/output,

It seems to be working at the moment.