-
-
Notifications
You must be signed in to change notification settings - Fork 161
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
False positive on images #98
Comments
What applies the compression? I think it is |
I have exactly this issue too. My particular pixel diff lands at around 10% which is too high of a threshold to be used as a viable workaround... I typically run the CI on linux hosts and development & "update snapshots" on MacOS, but they both use Electron to do so headlessly so I don't understand what could make any difference there. If it has something to do with compression, that would make more sense. |
I can confirm that it is failing on color - one large plain colored UI element in my example RGB |
I think it's a colour profile issue. On OSX I have changed mine to different settings and get different colours in the screenshots. It could be that your snapshots were generated with one colour profile and then your run is creating (and comparing) screenshots in another. This happens for us between locally on OSX and CI. I don't think there's a way to force the colour profile that Electron uses, only Chrome, so we are seeing differences between the two as well. |
I too am getting false positives. In the images above, I've zoomed in to show there is actually a slight pixel difference in some of the lettering on my page. It's not limited to text only, though. Other elements on the page are showing diffs without code changes. I'm running this in a Bitbucket pipeline after generating images locally - both the pipeline and local installations are running the same (latest) version of Cypress & Electron. Are we any closer to an answer on what is causing this? |
I think my issue above can be disregarded - it's a separate issue. On closer inspection, it looks like it's CSS / Operating System related, i.e the scrollbar is rounded in one image and square in the other, which I think is caused by my Mac. Edit: my workaround for, for anyone else who encounters it, was to use docker both locally when creating my screenshots and in my pipeline (bitbucket), and used the same docker image on both. I followed the docker setup in the cypress docs:- |
I have the same issue and as a workaround, I manage to use a docker image to generate the base screenshots and use the same docker fie with my CI to do the visual testing |
We're experiencing a similar issue. We do use Docker images (running the tests on AWS). Out issue is that text is showing in slightly different positions (letters are a few pixels to the right or left compared to the baseline images). There are no changes to the files being tested, that is no CSS or HTML changes. This only happens on some runs of some tests but enough to block us from letting the tests act as blockers for allowing a PR to be merged. |
Could someone who has come up with a workaround kindly post more details of their approach? Would really appreciate it! |
Actually I may have found one - adding |
Hi, we have the exact issue, @mcintyret tried your suggestion setting this flag but the new images are the same, nothing changes. How do you set the env var maybe I'm doing something wrong... Edit: |
Same happening here. When I run the snapshots in CI, there is a difference in the colors. I am using electron, and tried the |
@MatiasArriola |
Same issue here, trying to use docker now. |
@sebinsua |
Seemed that the following code fixes the color difference issue, notice that launchOptions has to be returned:
|
Often we see false positives on images used, for example:
To me it feels/looks like there's some compression on the baseline, which is not applied when comparing the screenshots.
The text was updated successfully, but these errors were encountered: