[BUG] Headless webkit is not loading PayPal popup

[gersanco]

System info

• Playwright Version: [v1.38.0]
• Operating System: [macOS 13.5.2]
• Browser: [WebKit]
• Other info:

Source code

• I provided exact source code that allows reproducing the issue locally.

Link to the GitHub repository with the repro

https://github.com/gersanco/playwright-webkit-paypal-popup

Steps

• Have video and trace set as "on"
• Run the test

Expected

The PayPal popup must be displayed with info.

Actual

It doesn't pass with Webkit when is headless

While I was preparing the project to report this, I discovered that if video and trace are not in the config or as 'on-first-retry' they pass.

Screen.Recording.2023-09-21.at.13.33.53.mov

[yury-s]

All the tests are timing out when I run them locally, do we need NEXT_PUBLIC_PAYPAL_CLIENT_ID? Can you reduce the repro, so that it does not depend on PayPay token or share a test token that we can use?

While I was preparing the project to report this, I discovered that if video and trace are not in the config or as 'on-first-retry' they pass.

Enabling those features just makes the test slower but should not cause semantic changes in the behavior, so there might be a race in the test which is just more likely to occur when those options are set. It can also be that the server returns a different content based on some feature detection, try comparing the traces.

[gersanco]

Hi @yury-s you can try with this NEXT_PUBLIC_PAYPAL_CLIENT_ID="AQwSpfPuE0nN08Hk6MvNkFCphq5ThZlCs1bcUC-PEtLRhYSFd9WvgKRmyPJ7gWlrbHKM6DwGssnK8p-m"

Enabling those features just makes the test slower but should not cause semantic changes in the behavior, so there might be a race in the test which is just more likely to occur when those options are set. It can also be that the server returns a different content based on some feature detection, try comparing the traces.

but as you can see in the video that I attached, it works when is not headless and the only browser that doesn't work is webkit.

[yury-s]

This is the same problem as #27073 and was fixed by #27098. Try with npm i @playwright/test@1.38.1, the problem should be gone.