test(DIST-136): Increase resilience for visual tests in firefox #2
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -26,6 +26,16 @@ module.exports = function () { | |
| // For Visual Tests requires more waiting time | ||
| waitForVisualElement (cssLocator) { | ||
| this.waitForVisible({ css: cssLocator }, 10) | ||
| }, | ||
| async waitForIFrameMessage () { | ||
| this.executeAsyncScript(function (done) { | ||
| const interval = setInterval(function () { | ||
| if (window.document.title === 'form-ready') { | ||
| clearInterval(interval) | ||
| done() | ||
| } | ||
| }, 100) | ||
| }) | ||
|
Comment on lines
+29
to
+38
There was a problem hiding this comment. We cannot use directly window object with codeceptjs and webdriver, so we use |
||
| } | ||
| }) | ||
| } | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,74 +1,85 @@ | ||
| const { resizeWindow } = require('../codeceptjs-utils') | ||
|
|
||
| const clickOnLink = async (I, n = 1) => { | ||
| const cssLinkSelector = `[data-mode="${n}"]` | ||
| await I.click({ css: cssLinkSelector }) | ||
| } | ||
|
|
||
| const waitForIFrame = async (I) => { | ||
| await I.waitForVisible({ css: 'iframe' }, WAITING_TIME) | ||
| await I.waitForIFrameMessage() | ||
| } | ||
| const WAITING_TIME = 5 | ||
|
|
||
antoniofull marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| Feature('Embed Widget') | ||
|
|
||
| Scenario('Basic Widget @desktop', async (I) => { | ||
| await I.amOnPage('widget.html') | ||
| await waitForIFrame(I) | ||
antoniofull marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| await I.eyesCheck('Embed Basic Widget') | ||
| }) | ||
|
|
||
| Scenario('Full Page Widget @desktop', async (I) => { | ||
| await I.amOnPage('full.html') | ||
| await waitForIFrame(I) | ||
| await I.eyesCheck('Embed Full Page Widget') | ||
| }) | ||
|
|
||
| Scenario('Basic Widget @mobile', async (I) => { | ||
| await resizeWindow(I) | ||
| await I.amOnPage('widget.html') | ||
| await waitForIFrame(I) | ||
| await I.eyesCheck('Embed Basic Widget - Mobile') | ||
| }) | ||
|
|
||
| Scenario('Full Page Widget @mobile', async (I) => { | ||
| await resizeWindow(I) | ||
| await I.amOnPage('full.html') | ||
| await waitForIFrame(I) | ||
| await I.eyesCheck('Embed Full Page Widget - Mobile') | ||
| }) | ||
|
|
||
| Scenario('Popup Widget @desktop', async (I) => { | ||
| await I.amOnPage('popup.html') | ||
| await clickOnLink(I, 1) | ||
| await waitForIFrame(I) | ||
| await I.eyesCheck('Popup Widget') | ||
| }) | ||
|
|
||
| Scenario('Popup Widget @mobile', async (I) => { | ||
| await resizeWindow(I) | ||
| await I.amOnPage('popup.html') | ||
| await clickOnLink(I, 1) | ||
| await waitForIFrame(I) | ||
| await I.eyesCheck('Popup Widget') | ||
| }) | ||
|
|
||
| Scenario('Drawer Widget @desktop', async (I) => { | ||
| await I.amOnPage('popup.html') | ||
| await clickOnLink(I, 2) | ||
| await waitForIFrame(I) | ||
| await I.eyesCheck('Drawer Widget') | ||
| }) | ||
|
|
||
| Scenario('Drawer Widget @mobile', async (I) => { | ||
| await resizeWindow(I) | ||
| await I.amOnPage('popup.html') | ||
| await clickOnLink(I, 2) | ||
| await waitForIFrame(I) | ||
| await I.eyesCheck('Drawer Widget - Mobile') | ||
| }) | ||
|
|
||
| Scenario('Right Drawer Widget @desktop', async (I) => { | ||
| await I.amOnPage('popup.html') | ||
| await clickOnLink(I, 3) | ||
| await waitForIFrame(I) | ||
| await I.eyesCheck('Popup Widget') | ||
| }) | ||
|
|
||
| Scenario('Right Drawer Widget @mobile', async (I) => { | ||
| await resizeWindow(I) | ||
| await I.amOnPage('popup.html') | ||
| await clickOnLink(I, 3) | ||
| await waitForIFrame(I) | ||
| await I.eyesCheck('Right Drawer Widget - Mobile') | ||
| }) | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Runs firefox visual tests in parallel on travis (speed up tests)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, we can run tests in parallel for each browser 👍 I suggest you run
yarn test:visual:mobileseparately as well.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good I wasn't sure if creating many jobs in parallel would cause issues, thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I dont think it would. Each job needs to clone the repo and install dependencies - if the task is short it usually should not have a separate job. If it takes longer than cloning and installing and has no dependencies it can be a separate job.
I think I would also split functional tests per browser to make it faster.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 give me 10 minutes and I'll push the changes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One build failed because both tests were running in parallel, I wonder if makes more sense to run one job with both tests in parallel (as @Gotusso set it up) or separated? I am not sure about it. Thoughts @mathio ? (right now I split them and removed the parallel flag)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe try without the
--parallelflag for cypress? But I would check with Franco first. So for now feel free to keep this as is.Edit: I re-run the tests and they passed 🤷♂️ When run in separate Travis jobs it saves ~30 seconds of total run time.