Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Height of screenshots with cy.screenshot() in Chrome headless new is 79px smaller than viewportHeight config that is set #27260

Closed
lgenzelis opened this issue Jul 10, 2023 · 18 comments · May be fixed by #28930
Assignees
Labels
browser: chrome existing workaround stage: wontfix Cypress does not regard this as an issue or will not implement this feature topic: screenshots 📸 type: regression A bug that didn't appear until a specific Cy version release v12.5.0 🐛

Comments

@lgenzelis
Copy link

Current behavior

Currently, calling cy.screenshot() during a test will use the viewport dimensions (specified in cypress.config.js) if the test is run using "headed" mode. However, when using headless mode:

  • viewportWidth is respected
  • viewportHeight is respected only if it's smaller than the "default". Otherwise, it's ignored.

That "default" height seems to depend on the browser being used. In cypress 12.14, the default was 720px for both chrome and firefox. However, in cypress 12.17, it's 720px for firefox, but 596px for chrome.

If you watch the video corresponding to the headless executions, sometimes you can even see, for a small fraction of a second, that the screen changes, removing all the Cypress UI elements, but also changing the viewport height.

cypress.bug.mp4

A straightforward workaround for this is adding this line before capturing the screenshot:

cy.get('html').then((html) => html.get(0).style.height = `${Cypress.config('viewportHeight')}px`);

Desired behavior

Cypress should respect the specified viewportHeight.

Test code to reproduce

https://github.com/lgenzelis/cypress-bug-screenshot-viewportheight-ignored

If you run this using npm run cypress:open, the screenshot is captured with the correct dimensions (specified in cypress.config.js). If run it using npm run cypress:run, though, the resulting screenshot is 400px wide (as expected), but only 596px tall.

Cypress Version

12.17.0

Node version

v18.14.0

Operating System

macos 13.4.1

Debug Logs

No response

Other

No response

@lgenzelis
Copy link
Author

After more investigation, my "fix" is no real fix, it just makes the problem less obvious. The viewport is still taking the default height (e.g. 596px for chrome), but cypress default 'fullPage' capture mode is making it look like it takes the correct viewport height. The problem only becomes apparent if there are any fixed elements in the UI (https://docs.cypress.io/api/commands/screenshot#Full-page-captures-and-fixedsticky-elements)

@lgenzelis
Copy link
Author

lgenzelis commented Jul 10, 2023

Alright, after reading this article (in particular, the distinction between viewport and window) I feel a little less flabbergasted. The only important thing to note, though, is that starting in cypress 12.15, chrome subtracts 124px from the window's height. I have no idea why, but the release notes for 12.15 mention that "Chrome versions 112 and above will now be run in the headless mode that matches the headed browser implementation". So, there's definitely something there.

In a nutshell, I don't know if this is a bug or not, but if you want to keep the same behavior as in before cypress 12.15, you need to add this to your cypress.config.ts (in e2e.setupNodeEvents):

on('before:browser:launch', (browser, launchOptions) => {
  if (browser.name === 'chrome') {
    launchOptions.args.push(`--window-size=1280,${720 + 124}`);
  }
  return launchOptions;
});

(note the magical +124 part 😅

@lmiller1990
Copy link
Contributor

I could be misunderstanding or doing it wrong, but I could not reproduce this. I tried making a page that is 738px high (4x 180px + 18px) and used Cypress 12.14, then Cypress 12.17. Both screenshots are the same - 738px high:

https://github.com/lmiller1990/cypress-test-tiny/compare/master...lmiller1990:cypress-test-tiny:issue-27260?expand=1

$ file before.png                                               (git)-[master]-
before.png: PNG image data, 1000 x 738, 8-bit/color RGBA, non-interlaced
$ file after.png                                                (git)-[master]-
after.png: PNG image data, 1000 x 738, 8-bit/color RGBA, non-interlaced

I tried with capture: 'viewport' and did see a 4px difference - weird. It's not the 124px you described, though. 🤔

Would be nice to find out what exactly changed, this could be a huge rabbit hole. Any recommendation for moving forward? We are using Percy for visual regression and none of our tests (500+ screenshots) had any changes... strange.

@lmiller1990 lmiller1990 removed their assignment Jul 12, 2023
@lgenzelis
Copy link
Author

lgenzelis commented Jul 12, 2023

That is really strange, @lmiller1990 . Adding that + 124 consistently brought the height back to 720 for me, as in before cypress 12.15. I reproduced it both locally (in macos) and in github actions running with ubuntu-latest. Two questions:

  • are you sure you're using chrome (in headless mode)? This doesn't happen with e.g. firefox (and I didn't try electron)
  • if so, are you using chrome 114? The change I refer to in the changelog for cypress 12.15 mentions "Chrome versions 112 and above"

@lmiller1990
Copy link
Contributor

Default is --headless, so definitely headless. Chrome 114.

image

Glad you found a fix, it would be nice if we could reproduce and narrow the issue down.

@jmeleropager
Copy link

I ran into same issue and after digging deeper I believe I found the source of the issue: The new headless mode of Chrome is "rendering" Chrome's UI alongside the viewport.

I've done a quick test, running these commands:

/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --headless=old --window-size=1280,800 --screenshot https://developer.chrome.com/ 

/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --headless=new --window-size=1280,800 --screenshot https://developer.chrome.com/ 

Results

Old headless mode (image resolution 2560x1600)
screenshot-old

New headless mode (image resolution 2560 × 1442)
screenshot-new

From previous images I can drop 2 conclusions:

  • Both modes are aware of underlying SO/hardware, because taking screenshot of 1280x800 window size resulted in images of double resolution, I guess it's because I'm using a Mac with retina display.
  • New headless mode image has a lower vertical resolution: 1442 instead of 1600, 158px less.

Trying to explain the difference in last point, I took a screenshot of the browser and measured chrome's UI

Captura de pantalla 2023-07-12 a la(s) 4 21 58 p m  1

And not for my surprise, yes, Google Chrome UI is 158px height (actually is 79px height, but because of retina display the image has 158px).

So yes, it seems new Google Chrome "headless" mode is actually rendering the UI or at least is making space for it.

@lgenzelis
Copy link
Author

I've just run those two commands in my computer as well and got the exact same results as @jmeleropager 🤯 So, the problem is caused indeed by Cypress 12.15 changing to --headless=new.

@lmiller1990
Copy link
Contributor

To clarify what headless: new is, see https://developer.chrome.com/articles/new-headless/.

As for this discovery; is the proposed fix to simply add a hard-coded value of 124 to the viewport height during a chrome headless=new screenshot? If this really fixes the issue and Cypress continues to work how you'd expect, this seems like a reasonable proposed fix to me.

@lmiller1990 lmiller1990 added pkg/driver This is due to an issue in the packages/driver directory pkg/server This is due to an issue in the packages/server directory labels Jul 14, 2023
@yktoo
Copy link

yktoo commented Jul 18, 2023

I suspect it's also related to #27291. The screenshotting mechanics seems totally broken:

  • the viewport settings are ignored
  • some screenshots contain Cypress sidebar, some inexplicably don't turns out the sidebar is only present when registering a failure, not when calling cy.screenshot()
  • the screenshots without the sidebar contain garbage, seemingly leftovers from previous screenshots

@lmiller1990
Copy link
Contributor

lmiller1990 commented Jul 18, 2023

@yktoo is this a general bug report of specific to chrome 114+ and the new headless mode (which seems to be the general consensus here in this issue).

If there's a separate bug, we should file a separate issue - if something is totally broken, this is a high priority (although all the screenshots in my projects look okay, it's definitely possible something regressed, although my understanding is nothing changed around there lately).

@yktoo
Copy link

yktoo commented Jul 19, 2023

@lmiller1990 Well it depends of course on the definition of "totally," but to me the combination of this one and #27291 renders screenshots pretty much unusable. The good news is failure screenshots are not affected (so it seems at least).

I believe these two bugs cover the existing problems.

@lmiller1990
Copy link
Contributor

#27291 looks like a huge problem - I don't see this on my apps, but we will look into what's happening there - a core feature like this should work perfectly (and our tests say it does... maybe we missed some cases).

@4knort
Copy link

4knort commented Sep 14, 2023

on('before:browser:launch', (browser, launchOptions) => {
  if (browser.name === 'chrome') {
    launchOptions.args.push(`--window-size=1280,${720 + 124}`);
  }
  return launchOptions;
});

this workaround doesn't work if we screenshot elements, not full page. Elements are smaller than they need to be, if our website is flexible.

@MelvinBaaijAviva
Copy link

I am also experiencing issues with screenshots (viewportwidth / viewportheight are not respected and or screenshot is made with a zoomlevel). Very annoying when using visual testing (for example with cypress-image-diff-js), especially on specific elements.
I hope you can have a look on this one and fix it @lmiller1990

@4knort did you already find a workaround or solution for your problem?

@hanna-becker
Copy link

Any updates here? We've been stuck with v12.14.0 because of this issue (upgrading breaks our snapshot tests).

@vaab
Copy link

vaab commented Dec 8, 2023

Same here, recently upgraded to cypress 13.5.1 over a cypress 12.11.0. And noticed that our tour of the app, used to make screenshots for mobile app stores, produced different sized screenshots. Only the height is affected. And yes, I'm using chrome 114, and could reproduce this on macosx and linux box on our CI, also:

The report at the end of the tests

  -  /.../.cypress/screenshots/spec.cy.ts/login.png                  (1242x1857)
  -  /.../.cypress/screenshots/spec.cy.ts/dashboard.png              (1242x2208)
  -  /.../.cypress/screenshots/spec.cy.ts/recipients.png             (1242x2208)
  -  /.../.cypress/screenshots/spec.cy.ts/pay.png                    (1242x2208)
  -  /.../.cypress/screenshots/spec.cy.ts/top-up.png                 (1242x2208)
  -  /.../.cypress/screenshots/spec.cy.ts/map.png                    (1242x1857)

Expected size: 1242x2208.

Two of them are not correctly sized (first and last), but actually all of them are screwed: on the ones that seem to have the right height, the actual bit below 1857 and down to the bottom is filled with garbage that seem to be previous screenshots or some cache of what was rendered before.

This seems pretty random as it occurs on the same run. Here, with my settings, the padding could be 117 (x3 which the scaleFactor). If this works for this resolution it feels clunky. And I tried with success the forcing of --headless=old. Which worked flawlessly.
So for me, this is the best workaround: you add in your cypress.config.ts:

on('before:browser:launch', (browser, launchOptions) => {
  if (browser.name === 'chrome') {
    launchOptions.args.push(`--headless=old`);
  }
  return launchOptions;
});

@jennifer-shehane jennifer-shehane added topic: screenshots 📸 v12.5.0 🐛 browser: chrome and removed pkg/driver This is due to an issue in the packages/driver directory pkg/server This is due to an issue in the packages/server directory labels Dec 17, 2023
@jennifer-shehane jennifer-shehane added the type: regression A bug that didn't appear until a specific Cy version release label Dec 27, 2023
@jennifer-shehane jennifer-shehane changed the title cy.screenshot() ignores viewport height when running in headless mode Height of screenshots with cy.screenshot() in Chrome headless new is 79px smaller than viewportHeight config that is set Dec 27, 2023
@emilyrohrbough
Copy link
Member

There are two things going on here:

Wont Fix - Browser Behavior

Cypress updated to use Chrome's headless=new flag when testing. Chrome introduced this flag to unify how the headed and headless browsers were rendered for a more accurate headless browser implementation. This reduction to the viewport height available to render browser content is outside of Cypress' control and could change in any Chrome version. With the headless=new flag, this is the correct behavior. The introduction of the headless=new flag likely should have been apart of v13.

Bug - Capture fullPage is incorrectly capturing as viewport

As reported, the default screenshot capture method of fullPage should be respecting the configured viewportHeight.
The current screenshot behavior is acting as capture: viewport instead of capture: fullPage.

The Chrome change impacting the viewport height, should ONLY be impacting cy.screenshot({ capture: 'viewport' }) / Cypress.screenshot.defaults({ capture: 'viewport' }). However as observed in your reproduction, the fullPage screenshot is not actually capturing a the full AUT, but what is visible within the viewport when the Cypress runner UI is removed.

The cy.screenshot command is correctly taking 1+ screenshots to account for the viewportHeight value, however, our styles are incorrect so they window is not scrollable, so when the scroll logic executes to capture content outside of the viewport, it does not scroll.

I quickly tested Cypress v9 and its was also broken, so this been an issue for a few versions. Fixing this will have a fairly large impact for anyone utilizing screenshots for visual regression. cc// @jennifer-shehane

Varying Heights per Run

@vaab Per #27260 (comment), I have been unable to reproduce this behavior. Please open a new issue specifically around this issue you are reporting.

@jennifer-shehane
Copy link
Member

After a thorough evaluation of this described issue, Cypress has decided that the new behavior from Chrome headless=new in this case is something that we will not be changing our behavior around.

With Chrome updating their headless browser to be the same codebase as the headed browser, this introduced Chrome's 'chrome' and UI like info bars taking up space within the window. This height is not under our control and it is able to change based on Chrome's implementation.

I did find this issue on Chromium's side, where someone was saying this behavior should be the same as headless=old and Chromium should change it. I'm not sure if they intend to act on this though: https://issues.chromium.org/issues/40275252

We could arbitrarily add some height calculations, but this won't solve anything since this height is able to change based on Chrome's implementation.

We'll be closing this issue.

@jennifer-shehane jennifer-shehane closed this as not planned Won't fix, can't repro, duplicate, stale Mar 25, 2024
@jennifer-shehane jennifer-shehane added the stage: wontfix Cypress does not regard this as an issue or will not implement this feature label Mar 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
browser: chrome existing workaround stage: wontfix Cypress does not regard this as an issue or will not implement this feature topic: screenshots 📸 type: regression A bug that didn't appear until a specific Cy version release v12.5.0 🐛
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants