Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unzipping progress goes to 200% #5967

Closed
goodliff opened this issue Dec 16, 2019 · 16 comments · Fixed by #6009 · May be fixed by qsays/grafana#1
Closed

Unzipping progress goes to 200% #5967

goodliff opened this issue Dec 16, 2019 · 16 comments · Fixed by #6009 · May be fixed by qsays/grafana#1
Assignees
@flotwig
Labels
type: regression A bug that didn't appear until a specific Cy version release v3.8.0 🐛 Issue present since 3.8.0

Comments

@goodliff
Copy link

goodliff commented Dec 16, 2019 •
edited by jennifer-shehane

Tried the last release (3.8.0) on docker to check the the unzipping change that was introduced by cli: unnzip Cypress using unzip utility on Linux #5851, when downloading and extracting cypress I noticed that the unzipping progress continued to 200% and time remaining went into negative figures.

Current behavior:

[14:09:00] Unzipping Cypress 200% -56s [title changed]

Desired behavior:

Ideally this would be accurate to the unzip progress.

Versions

Cypress package version: 3.8.0 | Cypress binary version: 3.8.0 | docker-compose version 1.24.1, build 4667896b
@GregLahaye
Copy link
Contributor

Experienced same issue with Cypress 3.8.0 on Windows 10 WSL Ubuntu

@jennifer-shehane
Copy link
Member

jennifer-shehane commented Dec 17, 2019 •
edited

😅 Yeah, that should not be happening. We have some tests around this, but no tests around the exact percentage display as far as I can tell.

Can you run the install with debug mode mode and print the entire set of logs here?

There are 2 pathways it could take for unzipping and this should tell us which pathway yours followed.

@jennifer-shehane jennifer-shehane added type: regression A bug that didn't appear until a specific Cy version release v3.8.0 🐛 Issue present since 3.8.0 labels Dec 17, 2019
@cypress-bot cypress-bot bot added the stage: needs investigating Someone from Cypress needs to look at this label Dec 17, 2019
@GregLahaye
Copy link
Contributor

Here is debug log from Windows 10 WSL Ubuntu
https://pastebin.com/aqWyJ0aX

@goodliff
Copy link
Author

Greg I'm a bit slow, did that take 14 minutes to unzip? I'm struggling to convince people to get it into our build cycles with a 4 minute (haven't checked the time since 3.8.0) unzip time. I'm assuming you're able to cache it between builds?

@GregLahaye
Copy link
Contributor

😄 yea it is a long time - this was on WSL (file I/O takes longer) and also yes - it will cache after first install so you should only need to install once (per version at least).

@flotwig
Copy link
Contributor

flotwig commented Dec 17, 2019

@GregLahaye @goodliff It does still unzip correctly, right? Just the progress indicator is wrong?

@flotwig
Copy link
Contributor

flotwig commented Dec 17, 2019 •
edited

If you guys can do me a favor and send me the output of unzip -v - it would be helpful.

The progress indicator works basically by counting the number of lines output and dividing that by the total number of file in the archive - it assumes 1 line per file output. But maybe there is some version of unzip that has different stdout conventions.

My unzip:

UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP.

Latest sources and executables are at ftp://ftp.info-zip.org/pub/infozip/ ;
see ftp://ftp.info-zip.org/pub/infozip/UnZip.html for other sites.

Compiled with gcc 9.2.0 for Unix (Linux ELF).

UnZip special compilation options:
        ACORN_FTYPE_NFS
        COPYRIGHT_CLEAN (PKZIP 0.9x unreducing method not supported)
        SET_DIR_ATTRIB
        SYMLINKS (symbolic links supported, if RTL and file system permit)
        TIMESTAMP
        UNIXBACKUP
        USE_EF_UT_TIME
        USE_UNSHRINK (PKZIP/Zip 1.x unshrinking method supported)
        USE_DEFLATE64 (PKZIP 4.x Deflate64(tm) supported)
        UNICODE_SUPPORT [wide-chars, char coding: UTF-8] (handle UTF-8 paths)
        LARGE_FILE_SUPPORT (large files over 2 GiB supported)
        ZIP64_SUPPORT (archives using Zip64 for large files supported)
        USE_BZIP2 (PKZIP 4.6+, using bzip2 lib version 1.0.6, 6-Sept-2010)
        VMS_TEXT_CONV
        WILD_STOP_AT_DIR
        [decryption, version 2.11 of 05 Jan 2007]

UnZip and ZipInfo environment options:
           UNZIP:  [none]
        UNZIPOPT:  [none]
         ZIPINFO:  [none]
      ZIPINFOOPT:  [none]

And it looks fine:

unzip

@flotwig flotwig self-assigned this Dec 17, 2019
@goodliff
Copy link
Author

Wow! Have you sped that up for demonstration purposes? :)

I'm just in the middle of something but when I'm free I'll try and get more information from docker (had to remove it from dependencies for the moment)

@flotwig
Copy link
Contributor

flotwig commented Dec 17, 2019

Nope, it should be this fast if you have unzip installed! That was supposed to be an improvement in 3.8.0

re: unzip info - Sounds good - lemme know. It's also possible that your system doesn't have unzip, which could be causing the regression.

@goodliff
Copy link
Author

goodliff commented Dec 17, 2019 •
edited

Quite possibly, it's running via docker-compose at the moment and I've tried adding unzip to that. I think it may be defaulting to the npm unzip, I believe we have that as a dev-dep.

> cypress@3.8.0 postinstall /usr/src/app/node_modules/cypress
> node index.js --exec install

Installing Cypress (version: 3.8.0)

[16:41:16]  Downloading Cypress     [started]
[16:41:16]  Downloading Cypress      0% 0s [title changed]
[16:41:16]  Downloading Cypress      1% 20s [title changed]
[16:41:17]  Downloading Cypress      1% 30s [title changed]
[16:41:17]  Downloading Cypress      1% 40s [title changed]
...
[16:41:59]  Downloading Cypress      96% 2s [title changed]
[16:41:59]  Downloading Cypress      97% 1s [title changed]
[16:42:00]  Downloading Cypress      98% 1s [title changed]
[16:42:00]  Downloading Cypress      99% 0s [title changed]
[16:42:00]  Downloading Cypress      100% 0s [title changed]
[16:42:01]  Downloaded Cypress      [title changed]
[16:42:01]  Downloaded Cypress      [completed]
[16:42:01]  Unzipping Cypress       [started]
[16:42:02]  Unzipping Cypress        0% 0s [title changed]
[16:42:08]  Unzipping Cypress        1% 684s [title changed]
[16:42:08]  Unzipping Cypress        1% 685s [title changed]
[16:42:08]  Unzipping Cypress        1% 686s [title changed]
...
[16:43:33]  Unzipping Cypress        97% 3s [title changed]
[16:43:34]  Unzipping Cypress        98% 2s [title changed]
[16:43:35]  Unzipping Cypress        99% 1s [title changed]
[16:43:35]  Unzipping Cypress        100% 0s [title changed]
[16:43:36]  Unzipping Cypress        101% -1s [title changed]
[16:43:36]  Unzipping Cypress        102% -2s [title changed]
[16:43:38]  Unzipping Cypress        103% -3s [title changed]
[16:43:38]  Unzipping Cypress        104% -4s [title changed]
...
[16:44:38]  Unzipping Cypress        198% -78s [title changed]
[16:44:39]  Unzipping Cypress        199% -78s [title changed]
[16:44:39]  Unzipped Cypress        [title changed]
[16:44:39]  Unzipped Cypress        [completed]
[16:44:39]  Finishing Installation  [started]
[16:44:39]  Finished Installation   /root/.cache/Cypress/3.8.0 [title changed]
[16:44:39]  Finished Installation   /root/.cache/Cypress/3.8.0 [completed]

You can now open Cypress by running: node_modules/.bin/cypress open

https://on.cypress.io/installing-cypress

I've chopped large bits out of this to make it more readable, I'm still working on getting -v on unzip but I'll have to rebuild which will take some time.

Looking at docker-compose it doesn't mention unzip so I think you might be right.

Off topic, I'm trying to workout whether a PR from 8 months ago got released, where is the best place to ask or should I just raise a ticket?

@goodliff
Copy link
Author

goodliff commented Dec 17, 2019 •
edited

Yep, looks like you're right, I installed the unzip via apt before the installation and it completed in 11 secs instead of the few minutes without it. It also calculated the progress correctly.

[17:17:38]  Unzipping Cypress       [started]
[17:17:38]  Unzipping Cypress        0% 0s [title changed]
...
[17:17:48]  Unzipping Cypress        44% 12s [title changed]
[17:17:48]  Unzipping Cypress        45% 12s [title changed]
[17:17:48]  Unzipping Cypress        46% 11s [title changed]
[17:17:48]  Unzipping Cypress        47% 11s [title changed]
[17:17:48]  Unzipping Cypress        48% 11s [title changed]
[17:17:48]  Unzipping Cypress        49% 10s [title changed]
[17:17:48]  Unzipping Cypress        50% 10s [title changed]
[17:17:49]  Unzipping Cypress        51% 10s [title changed]
[17:17:49]  Unzipping Cypress        52% 9s [title changed]
[17:17:49]  Unzipping Cypress        52% 10s [title changed]
[17:17:49]  Unzipping Cypress        100% 0s [title changed]
[17:17:49]  Unzipped Cypress        [title changed]
[17:17:49]  Unzipped Cypress        [completed]
[17:17:49]  Finishing Installation  [started]
[17:17:49]  Finished Installation   /root/.cache/Cypress/3.8.0 [title changed]
[17:17:49]  Finished Installation   /root/.cache/Cypress/3.8.0 [completed]

You can now open Cypress by running: node_modules/.bin/cypress open

@GregLahaye
Copy link
Contributor

GregLahaye commented Dec 18, 2019 •
edited by jennifer-shehane

Same thing here - didn't have unzip installed so it seems like Cypress fell back to unzipping with Node.js:

2019-12-17T10:57:23.765Z cypress:cli:unzip unzipping from /tmp/cypress.zip
2019-12-17T10:57:23.766Z cypress:cli:unzip into /home/greg/.cache/Cypress/3.8.0
2019-12-17T10:57:23.785Z cypress:cli:unzip zipFile entries count 39363
2019-12-17T10:57:23.786Z cypress:cli:unzip unzipping via `unzip`
2019-12-17T10:57:23.868Z cypress:cli:unzip unzipping with node.js (slow)
2019-12-17T10:57:23.884Z cypress:cli:unzip `unzip` failed { code: -2 }
2019-12-17T10:57:23.884Z cypress:cli:unzip unzipping with node.js (slow)

Installing unzip through apt fixes the issue.

@goodliff
Copy link
Author

Need anything else @flotwig?

@cypress-bot cypress-bot bot added stage: work in progress and removed stage: needs investigating Someone from Cypress needs to look at this labels Dec 19, 2019
@flotwig flotwig mentioned this issue Dec 19, 2019
Merged
2 tasks
@flotwig
Copy link
Contributor

flotwig commented Dec 19, 2019

@goodliff Nope, we're good, thanks for the details so far. I've reproduced it locally and I have a PR open to fix it here: #6009

@cypress-bot cypress-bot bot added stage: needs review The PR code is done & tested, needs review and removed stage: work in progress labels Dec 26, 2019
@cypress-bot
Copy link
Contributor

cypress-bot bot commented Dec 26, 2019

The code for this is done in cypress-io/cypress#6009, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

@cypress-bot cypress-bot bot added stage: pending release and removed stage: needs review The PR code is done & tested, needs review labels Dec 26, 2019
@cypress-io cypress-io locked as resolved and limited conversation to collaborators Dec 30, 2019
@jennifer-shehane
Copy link
Member

Released in 3.8.2

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@flotwig flotwig

Labels
type: regression A bug that didn't appear until a specific Cy version release v3.8.0 🐛 Issue present since 3.8.0
Projects
None yet
Milestone
No milestone
4 participants
@goodliff @flotwig @jennifer-shehane @GregLahaye