Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add compareAfterShot config #385

Open
wants to merge 16 commits into
base: main
Choose a base branch
from

Conversation

SrBrahma
Copy link
Contributor

@SrBrahma SrBrahma commented Mar 31, 2024

Implementation of #372 (reply in thread)

All Submissions:

New Feature Submissions:

  1. Was the new feature discussed with core maintainers?
  2. Does your submission pass tests?
  3. Have you lint your code locally before submission?
  4. Does the change require an update in documentation? Have you updated the documentation in case it's true?

.eslintrc Outdated Show resolved Hide resolved
Comment on lines +346 to +352
shotItems: [
...storybookShotItems,
...pageShotItems,
...ladleShotItems,
...histoireShotItems,
...customShotItems,
],
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a code suggestion, I believe we could have a const shotItems at the beginning of the function and .push after each is obtained, so we don't have this here at the bottom.

It would also be good if each of them were in their own function as this function is quite big

Comment on lines -104 to -117
if (shallGenerateMeta()) {
log.process(
'info',
'general',
`Writing meta file with ${
Object.entries(comparisonResults).length
} items.`,
);
writeFileSync(
`${path.join(config.imagePathCurrent, 'meta')}.json`,
JSON.stringify(comparisonResults, null, 2),
);
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -131,7 +131,8 @@
"n/file-extension-in-import": "off",
"no-lone-blocks": "off",
"capitalized-comments": "off",
"unicorn/prefer-ternary": "off"
"unicorn/prefer-ternary": "off",
"no-await-in-loop": "off"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There were multiple disable-next-line for this rule.

@SrBrahma SrBrahma marked this pull request as ready for review March 31, 2024 20:31
Comment on lines +13 to +19
compareAfterShot: true,
flakynessRetries: 10,
// These times are greatly reduced due to compareAfterShot!
waitBetweenFlakynessRetries: 200,
waitBeforeScreenshot: 0,
waitForFirstRequest: 0,
waitForLastRequest: 0,
Copy link
Contributor Author

@SrBrahma SrBrahma Apr 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is where this PR really shines!

With the example-storybook-v8 with the default configs, it takes 10s to run -- and there are no retries.

If we use just the default flakynessRetries, it takes 20s as each page is screenshotted twice.

With this PR, we can lower the delays a lot -- actually even more than the values here and even with shotConcurrency: 20 -- without being afraid of temporary failures as they aren't a big deal, as we can simply retry how many times we want.

With compareAfterShot and this config, it takes 2s to run lostpixel, without any retry actually made.

It allows us to be less conservative with the delays to have a better performance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant