Upgrade to Puppeteer 11 #4093

limonte · 2021-11-07T11:40:34Z

This PR upgrades to Puppeteer 11 which brings us better and faster work with out-of-process iframes, such as iframes inside Gmail page. In order to work with iframes, e.g. check their content, we don't need to open iframe in a new tab anymore 🎉

Related issues:

Puppeteer v10:

const urls = await gmailPage.getFramesUrls(['/chrome/elements/pgp_block.htm'], { sleep: 1 });
const pgpBlockPage = await browser.newPage(t);
await pgpBlockPage.page.goto(urls[0]);
await pgpBlockPage.waitForContent('@pgp-block-content', '...');

Puppeteer v11:

const pgpBlockFrame = await gmailPage.getFrame(['pgp_block.htm']);
await pgpBlockPage.waitForContent('@pgp-block-content', '...');

Also, this PR tackles flaky live gmail tests

Tests (delete all except exactly one):

Does not need tests (refactor only, docs or internal changes)

To be filled by reviewers

I have reviewed that this PR... (tick whichever items you personally focused on during this review):

addresses the issue it closes (if any)
code is readable and understandable
is accompanied with tests, or tests are not needed
is free of vulnerabilities
is documented clearly and usefully, or doesn't need documentation

…future

…w tab

limonte · 2021-11-08T20:49:15Z

@rrrooommmaaa this task was outside of milestones, I took care of it because the live Gmail tests are failing on the master branch. So I killed two birds with one stone - fixed tests and upgraded puppeteer. Upgrading allowed less LoC and live Gmail tests in CI now take around 30s less.

rrrooommmaaa · 2021-11-09T05:49:56Z

test/source/browser/controllable.ts

@@ -273,11 +273,11 @@ abstract class ControllableBase {
    await this.waitAndClick(`@ui-modal-${type}-${clickBtn}`);
  }

-  public waitAndClick = async (selector: string, { delay = 0.1, confirmGone = false, retryErrs = false, sleepWhenDone }:
-    { delay?: number, confirmGone?: boolean, retryErrs?: boolean, sleepWhenDone?: number } = {}) => {
+  public waitAndClick = async (selector: string, { delay = 0.1, timeout = 10000, confirmGone = false, retryErrs = false, sleepWhenDone }:


I find it a bit strange, that delay is in seconds, and timeout is in millis.
Also, can we use a constant here, like TIMEOUT_ELEMENT_APPEAR / 2?
Is that the intent?

The intent here was to fail earlier than before (the previous timeout was 20s). But let me roll back this change as not important and potentially confusing.

Changed to TIMEOUT_ELEMENT_APPEAR, thanks 👍

test/source/tests/page-recipe/setup-page-recipe.ts

test/source/tests/gmail.ts

tomholub · 2021-11-09T10:19:35Z

Thank you for this change, and Roman for the review. Good to see the Gmail tests simplified - it really was not ideal before.

* Upgrade to Puppeteer 11 * unsuccessful attempt to refactor createSecureDraft, add todo for the future * Refactor pageHasSecureDraft() to use getFrame() instead of opening new tab * use getFrame() in Thunderbird tests * workaround sending in the 'secure reply btn, reply draft' test * Simplify openGmailPage() * fail faster - add timeout param to waitAndClick() * wip * handle 'Node is either not clickable' error * wait longer for @action-step0 and @action-step1 * wait longer for @input-compatibility-fix-expire-years * wip * always delete local draft after sending * cleanup * rename mock live test * log * fix 'secure reply btn, reply draft' test * cleanup * do not rely on sleep timeouts * typo * timeout in seconds * let composeBox: Controllable | undefined

…4109) * test cancelling passphrase dialog in compose * Change the 'keyup' events handlers to 'input' because text content can be changed with the mouse (#4100) * Change the 'keyup' handlers to 'input' because text content can be changed with the mouse * trigger 'input' events * Upgrade to Puppeteer 11 (#4093) * Upgrade to Puppeteer 11 * unsuccessful attempt to refactor createSecureDraft, add todo for the future * Refactor pageHasSecureDraft() to use getFrame() instead of opening new tab * use getFrame() in Thunderbird tests * workaround sending in the 'secure reply btn, reply draft' test * Simplify openGmailPage() * fail faster - add timeout param to waitAndClick() * wip * handle 'Node is either not clickable' error * wait longer for @action-step0 and @action-step1 * wait longer for @input-compatibility-fix-expire-years * wip * always delete local draft after sending * cleanup * rename mock live test * log * fix 'secure reply btn, reply draft' test * cleanup * do not rely on sleep timeouts * typo * timeout in seconds * let composeBox: Controllable | undefined * #4052 passphrase dialog for non-primary S/MIME signing * Do not show post-it reminder for EKM (#4103) * Do not show post-it reminder for EKM * upd tests * enterPp.expectPostitNoteReminder * Remove expectPostitNoteReminder and related check * Fix the 'Reply' button behavior, align it with Gmail (#4107) * Fix the 'Reply' button behavior, align it with Gmail * copy before iterating * Add tests * add clearRecipientsForReply() to initComposeBox() * Revert * fix the reply button behavior * await promise * Add live gmail test for the reply icon button * simpler namings (#4111) * align 'reply all' behavior with Gmail Co-authored-by: Roman <rrrooommmaaa@mail.ru> * issue #3885 add checkbox per email for attester key submission (#3907) * add checkbox per email for attester key submission * fixes tslint error * fixes tslint error * add xss-escaped comment to pass pattern checks * Added test for issue-3885 selectable email aliases to submit on attester * Added a private key with two UIDs * refactor data-test naming and add multi email alias account to google-endpoint.ts * fix tslint error * fix failing test on setup-page-recipe * Added test for importing key with multiple email alias (incomplete) * fixes tslint formatting error * complete test for importing key with multiple email alias * rename data-test and class container * rename data-test and class container * separate the test to CONSUMER-MOCK test variant * move the render display function to key-import-ui.ts * emails for checkboxes are default to 'unchecked' state * remove accidental console.log * Added automatic check/uncheck when an email is present. * fixes tslint by adding interface property * add event of keyup paste and change to manipulate checkboxes * change button color from gray to green when valid private key * remove checkEmailAliasIfPresent and uses fillOnly * use fillOnly completely * bring back checkEmailAliasIfPresent and wrap it in fillOnly * rename css class name to avoid interfering with any className based checking. * added key with multiple aliases * simplified code and move data-test to label input * corrected any type to string * remove spagetti code and better checking for submitkeyforaddrs * exclude email (uncheck checkbox) before submitting * attester pubkey for multi alias user (failing) * added test "setup - imported key from a file with multiple alias" * added test if excluded email was submitted from the attester * collect submitted keys from attester * patch data-test (selector) trasnformer to match/replace any provided selector * move data-test directly to the input * fix inconsistency in checking detected email alias * manipulate test key and added 1 UID * complete neccessary tests by checking default detected key states * refactor email alias process [floating-promise-error] in constructor setup.ts (key-import-ui initPrvImportSrcForm) * remove comment * parse aliases via already rendered input checkbox * patched 'saveKeysAndPassPhrase' on setup.ts * fix conflict * fix conflict * fixes eslint * added callback when performing tests * remove unnecessary undefined initialization * fix pubkey definition * uncheck the checkbox when submit pubkey was set to false * better flow for pubkey submission * check for checkbox state in the first run * fix test title typo * reverted back changes [proposed solution] * clean up setup procedure Co-authored-by: Roman Shevchenko <rrrooommmaaa@mail.ru> Co-authored-by: Mart Gil Robles <mart@Marts-MacBook-Air.local> Co-authored-by: Tom J <6306961+tomholub@users.noreply.github.com> Co-authored-by: Tom <tom@flowcrypt.com> Co-authored-by: Tom J <tom@holub.me> * issue #4097 add warning when manually importing public keys (#4110) * issue #4097 add warning when manually importing public keys * Improve wording, make import buttons orange * more flexible google mock * allow opening a draft compose box based from inbox.ts in debug mode * fix * added filePath option for addKeyTest * test loading draft with a forgotten non-primary passphrase * merge fix * merge fix * test fix * remark Co-authored-by: Limon Monte <limon.monte@gmail.com> Co-authored-by: martgil <46025304+martgil@users.noreply.github.com> Co-authored-by: Mart Gil Robles <mart@Marts-MacBook-Air.local> Co-authored-by: Tom J <6306961+tomholub@users.noreply.github.com> Co-authored-by: Tom <tom@flowcrypt.com> Co-authored-by: Tom J <tom@holub.me>

limonte added 9 commits November 7, 2021 13:35

Upgrade to Puppeteer 11

264ad58

unsuccessful attempt to refactor createSecureDraft, add todo for the …

37ae533

…future

Refactor pageHasSecureDraft() to use getFrame() instead of opening ne…

a47e638

…w tab

use getFrame() in Thunderbird tests

b2a2e56

workaround sending in the 'secure reply btn, reply draft' test

52770d7

Simplify openGmailPage()

c22ddb7

fail faster - add timeout param to waitAndClick()

90b170b

wip

2a1e72c

handle 'Node is either not clickable' error

7999b26

limonte force-pushed the puppeteer-11-run-gmail-tests branch 2 times, most recently from f8a561e to c9b187d Compare November 7, 2021 16:04

wait longer for @action-step0 and @action-step1

20d6d7c

limonte force-pushed the puppeteer-11-run-gmail-tests branch from c9b187d to 20d6d7c Compare November 7, 2021 16:17

limonte added 3 commits November 7, 2021 18:21

wait longer for @input-compatibility-fix-expire-years

65829ae

wip

09e0283

always delete local draft after sending

ac25d44

limonte force-pushed the puppeteer-11-run-gmail-tests branch 2 times, most recently from f1b502d to f730676 Compare November 7, 2021 20:59

cleanup

f80ab4b

limonte force-pushed the puppeteer-11-run-gmail-tests branch from f730676 to f80ab4b Compare November 7, 2021 21:10

rename mock live test

f069f69

limonte force-pushed the puppeteer-11-run-gmail-tests branch 4 times, most recently from 6128ec5 to 613873a Compare November 7, 2021 22:37

log

254ff2f

limonte force-pushed the puppeteer-11-run-gmail-tests branch from 613873a to 254ff2f Compare November 7, 2021 22:53

limonte added 3 commits November 8, 2021 01:37

fix 'secure reply btn, reply draft' test

cc2f6f9

cleanup

dfd8693

do not rely on sleep timeouts

5b223c2

typo

728e967

limonte force-pushed the puppeteer-11-run-gmail-tests branch from 140f4fb to 728e967 Compare November 8, 2021 19:40

limonte changed the title ~~[WIP] Upgrade to Puppeteer 11~~ Upgrade to Puppeteer 11 Nov 8, 2021

limonte marked this pull request as ready for review November 8, 2021 19:51

limonte requested a review from rrrooommmaaa as a code owner November 8, 2021 19:51

rrrooommmaaa suggested changes Nov 9, 2021

View reviewed changes

limonte added 2 commits November 9, 2021 10:26

timeout in seconds

78cb573

let composeBox: Controllable | undefined

15acf39

rrrooommmaaa approved these changes Nov 9, 2021

View reviewed changes

rrrooommmaaa merged commit 994c4d7 into master Nov 9, 2021

rrrooommmaaa deleted the puppeteer-11-run-gmail-tests branch November 9, 2021 14:45

tomholub added this to the 8.1.6: S/MIME, drafts improvements milestone Nov 9, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade to Puppeteer 11 #4093

Upgrade to Puppeteer 11 #4093

limonte commented Nov 7, 2021 •

edited

limonte commented Nov 8, 2021

rrrooommmaaa Nov 9, 2021

limonte Nov 9, 2021

limonte Nov 9, 2021

tomholub commented Nov 9, 2021

Upgrade to Puppeteer 11 #4093

Upgrade to Puppeteer 11 #4093

Conversation

limonte commented Nov 7, 2021 • edited

To be filled by reviewers

limonte commented Nov 8, 2021

rrrooommmaaa Nov 9, 2021

Choose a reason for hiding this comment

limonte Nov 9, 2021

Choose a reason for hiding this comment

limonte Nov 9, 2021

Choose a reason for hiding this comment

tomholub commented Nov 9, 2021

limonte commented Nov 7, 2021 •

edited