Random build failures due to fs-extra defect #121
Comments
|
Same here: Cordova-Android 8.0.0 I only get this error on my jenkins server. Not on my local pc. But I don't use network paths. |
|
Any news / workaround / status? This problem is not 100% reproducible, sometimes it works. But "random" is the wrong word. It's about 80-99.9% that the build fails. This means: It's a critical must-fix bug and should get the highest priority. |
|
@Domvel Other than completely removing fs-extra, there is no way to fix this in Cordova AFAIK. This is unlikely to happen, because there has just been a move to introduce fs-extra into Cordova. This would need to be fixed upstream, but given the activity in jprichardson/node-fs-extra#657, it appears that this won't happen any time soon. This is just a huge effin mess and I agree that the builds fail more often than they pass. It is a major defect, but I don't see how to resolve it and it appears to not affect enough people to get enough attention. I urge you to join the discussion in the upstream ticket, as that seems to be the most promising place to find a solution for this issue. |
|
@oliversalzburg Thanks for the info. Is I don't quite understand the reason that fs-extra breaks the build. What does the message mean: "Source and destination must not be the same." I see over 2 occurrences in node-fs-extra sources in relation of copy a object. Is this a pseudo-error and could be removed? (copy.js and copy-sync.js in function "checkPaths") |
|
|
|
Have an other solutions to replace the node-fs-extra? |
|
@f5cs We've been using a |
|
fs-extra are about to release version 8 which apparently has a workaround for this issue. Will you guys use that once it's out? |
|
@dimitriscsd With "we", I didn't mean Cordova, I meant our company :D But I'm sure Cordova will be updated to use the latest version of fs-extra. Feel free to open a PR when the update is available to help speed up the process. |
|
@oliversalzburg understood. My "you guys" was more towards cordova devs than you specifically :) Thanks for your hook workaround suggestion though. I might have to give that a shot if this takes too long. Do you mind sharing more info on it? |
|
This is the hook file itself: #!/usr/bin/env node
"use strict";
const fs = require( "fs" );
const path = require( "path" );
const os = require( "os" );
if( os.platform() !== "win32" ) {
console.log( "fs-extra doesn't need to be patched on non-Windows OS. Skipping." );
process.exit( 0 );
}
function copyFile( source, target, cb ) {
let cbCalled = false;
const reader = fs.createReadStream( source );
const writer = fs.createWriteStream( target );
reader.on( "error", error => done( error ) );
writer.on( "error", error => done( error ) );
writer.on( "close", () => done() );
reader.pipe( writer );
function done( error ) {
if( !cbCalled ) {
cb( error );
cbCalled = true;
}
}
}
const copyFrom = path.resolve( __dirname, "fs-extra-patch/copy.js" );
const copyTo = path.resolve( __dirname, "../node_modules/fs-extra/lib/", "copy/copy.js" );
const copySyncFrom = path.resolve( __dirname, "fs-extra-patch/copy-sync.js" );
const copySyncTo = path.resolve( __dirname, "../node_modules/fs-extra/lib/", "copy-sync/copy-sync.js" );
console.log( `Copying '${copyFrom}' to '${copyTo}'.` );
copyFile( copyFrom, copyTo, Function.prototype );
console.log( `Copying '${copySyncFrom}' to '${copySyncTo}'.` );
copyFile( copySyncFrom, copySyncTo, Function.prototype );Relative to it, there is a folder named function checkPaths( src, dest ) {
try {
return fs.statSync( dest );
} catch( err ) {
if( err.code === "ENOENT" ) {
return notExist;
}
throw err;
}
}Hope it helps. |
If/when there is a version released with a fix, we'll see if we can pull it in without it being a breaking change (which pretty much just means, it needs to support node 6). |
jprichardson/node-fs-extra#667
jprichardson/node-fs-extra#667 seems to keep Node.js 6 support in package.json: [1] and tested green so far on AppVeyor: [2] I would personally favor incorporating this update in a minor release due to what I think are some non-trivial changes in jprichardson/node-fs-extra#667 |
|
@dpogue Why is node 6 required? Why not just update and use the latest. e.g. 10.x (LTS). I just wondering. |
According to semver rules, we cannot drop Node.js 6 support without making another major release. |
|
For context:
Currently ~45% of Cordova users are using node <10. |
|
fs-extra v8 has been released. Just a heads up. |
But this is like the Internet Explorer phenomenon. It's more a question of laziness and ignorance. I see it more as a task to bring the people in it. Using the latest versions. Especially for developers, it is important to always be up to date and not to stick to proven versions. For a long time that is not good. However, I think it's because these people have not yet updated. But there would be nothing wrong with that. Or are there known issues with Node 10+? |
|
@Domvel Not updating NodeJS isn't always linked to being lazy. Think about companies where you have hundreds of developers. Maybe dependencies you use no longer work in newer versions of Node. There are very good reasons not to update NodeJS constantly. |
|
@oliversalzburg This was not exactly what I meant. It's fine. Do not blindly update the productive environments. The question is, when do these people finally update? I mean only the people and companies that still use Internet Explorer. It's just wrong in every way. No discussion please. 😄 |
|
I just transferred this issue from cordova-common to cordova and closed #83 as a duplicate. Also discussed in the following places:
I hope to get the bugfix on cordova-common released next week, then I think we should update some other packages including cordova-lib, cordova-android, cordova-windows, and cordova-cli. Please feel free to contact us via email to dev@cordova.apache.org in case of any further delays. |
|
cordova-common@3.2.0 is now released with apache/cordova-common#70 merged (Thx! @brodybits) |
You are welcome.
I am still waiting for one more vote from a committer, will publish on npm on Friday if we get the vote in time.
I only mentioned some other issues and a PR where a similar problem was reported. I think the update in I would also like to apply the same update to some other Cordova packages, cannot promise when I will have time to do this. Here are the packages I think should be updated when someone gets a chance:
|
|
I just published cordova-common@3.2.0 on npm, blog post should show up pretty soon. I expect that this should resolve the reported issue for most if not all users. Here is the recommended procedure to update both the Cordova tooling and Cordova project:
This may be extra pedantic, but I would recommend this procedure to hopefully ensure a zero chance of using an old version of I would like to update the other packages ( I took about 2-3 days out of an extremely busy schedule to make this update, in response to the demand from the user community. |
|
Thanks much, @brodybits!
Is there any way I could help out in updating other packages? Would you like me to submit PRs targeting the updated packages? I see that the above packages have dependencies to both Again, thanks for your efforts! |
|
I have fixed the problem (for my Cordova build not working) by a much easier way guys. Please follow the below steps to update your
|
|
I want to add that if problem persist for Windows 10 users, just follow these steps :
This worked for me. |
|
Hi! I'm facing the same issue and I'm building on a mac. Android (8.0.0) and iOS (5.0.1) builds fail. |
Which version of Node.js are you using? I would highly recommend using the latest patch on Node.js version 10. |
|
My node version is 10.15.3. I pulled my project from my git repository again to have a clean start. Still the same error. Previous Android and iOS versions work fine. |
|
This issue continuously kills me and does so at random... |
|
This may be caused by symlinks, at least that was my case. The solution, other than upgrading as stated before, is to replace symbolic links with hard links. |
|
@gquagliano thx for this trigger, this solved it (Source and destination must not be the same.) for me when adding a new platform. I use a virtualbox for development and I have disks mapped as network share from my host where the projects are located. Moving the cordova project to the C drive fixed it for me. |
|
is this issue still open ? |
|
Closing this issue as obsolete. If you still have this issue, I'd recommend trying to reinstalling cordova and/or clearing your package-lock & node_modules to force npm to fetch the latest satisfactory dependencies, which should include the fixes from fs-extra upstream. |
Bug Report
Problem
What is expected to happen?
You are able to build your Cordova project reliably.
What does actually happen?
Builds randomly fail with the error Source and destination must not be the same.
Information
The random failures are caused by a defect in fs-extra, which neglects to respect an inode-related issue in Node itself.
Command or Code
Every command that copies files with
fs-extrais affected.cordova prepareis most likely to cause this.Environment, Platform, Device
Windows 10
Version information
Cordova 8.1.2 (cordova-lib@8.1.1)
Checklist
The text was updated successfully, but these errors were encountered: