Speed up and improve tests

[rrrooommmaaa]

This PR speeds up live gmail tests by testing iframes directly,
also makes more strict checks that no leftover original attachment remain

close #4929 (pattern getFramesUrls and then browser.newPage was removed except where necessary) -- main bulk was updated in #5069

---

Tests (delete all except exactly one):

• Internal test improvements

---

To be filled by reviewers

I have reviewed that this PR... (tick whichever items you personally focused on during this review):

• addresses the issue it closes (if any)
• code is readable and understandable
• is accompanied with tests, or tests are not needed
• is free of vulnerabilities
• is documented clearly and usefully, or doesn't need documentation

[rrrooommmaaa]

@sosnovsky Why sometimes there is only one artefact when 2 tests are failed?

It generally looks as if gmail live tests run in parallel (with another instance of test suite?)
Strange interference that never happens when a single test is run.
This primarily concerns drafts, as drafts are deleted at the beginning of almost each test

[sosnovsky]

@sosnovsky Why sometimes there is only one artefact when 2 tests are failed?

Looks like we're storing only 1 file in Artifacts folder even when multiple are written:

Should be fixed too, here is code in semaphore.yml - https://github.com/FlowCrypt/flowcrypt-browser/blob/master/.semaphore/semaphore.yml#L90

[sosnovsky]

It generally looks as if gmail live tests run in parallel (with another instance of test suite?)

It shouldn't be possible, I think, as live tests run only in master and they can't interfere.
What about running drafts test first and then all other tests?

[rrrooommmaaa]

It shouldn't be possible, I think, as live tests run only in master and they can't interfere.

Do you know why we have pool of 3 browsers for gmail live tests? Does this make sense?

[sosnovsky]

Do you know why we have pool of 3 browsers for gmail live tests? Does this make sense?

Not sure, maybe previously we ran live tests simultaneously, but for now we don't need these 3 browsers, just 1 will be enough.
There is a related task, where pool size should equal to --concurrency param value - #4955

[rrrooommmaaa]

Not sure, maybe previously we ran live tests simultaneously, but for now we don't need these 3 browsers, just 1 will be enough.

Changing this value to 1 dramatically increased the run time to over an hour

[rrrooommmaaa]

I think most of the updates are already incorporated into PR #5069.
After completion of it, I'll review this one, yes.

[sosnovsky]

@rrrooommmaaa probably now you can work on this one, as live tests are still failing

[rrrooommmaaa]

We can also use getFrame directly (#4929) here if we don't need debug flag

flowcrypt-browser/test/source/tests/page-recipe/gmail-page-recipe.ts

Lines 14 to 16 in cbb2aff

	const urls = await gmailPage.getFramesUrls(['/chrome/elements/compose.htm'], { sleep: 1 });
	expect(urls.length).to.equal(1);
	return await browser.newPage(t, `${urls[0]}&debug=___cu_true___`);

[rrrooommmaaa]

It'll be run serially anyway. I can leave this serial as precaution or delete it

[sosnovsky]

We can also use getFrame directly (#4929) here if we don't need debug flag

flowcrypt-browser/test/source/tests/page-recipe/gmail-page-recipe.ts

Lines 14 to 16 in cbb2aff

	const urls = await gmailPage.getFramesUrls(['/chrome/elements/compose.htm'], { sleep: 1 });
	expect(urls.length).to.equal(1);
	return await browser.newPage(t, `${urls[0]}&debug=___cu_true___`);

Is this debug flag used somewhere in tests or we can remove it without breaking tests? It'll be better to switch to using direct getFrame in #4929

[rrrooommmaaa]

Is this debug flag used somewhere in tests or we can remove it without breaking tests?

Actually, as richtext option is disabled in Chrome, the debug flag is used to allow it in the test environment.
So I renamed the method to more descriptive openSecureComposeWithRichTextWorkaround

[sosnovsky]

Actually, as richtext option is disabled in Chrome, the debug flag is used to allow it in the test environment. So I renamed the method to more descriptive openSecureComposeWithRichTextWorkaround

Then let's leave it for now, until we'll find another way to enable richtext option on Chrome for tests (or it'll be ready to be enabled by default on Chrome too)

[sosnovsky]

Looks good 👍