vrtStart and vrtStop with Cypress when running tests in parallel

[alexniculae]

As an example, let's consider we're running some Cypress tests together with some VRT Visual Tests, in parallel on X current threads (let's say 2 for an easy example).

Given the example set up for vrtStart and vrtStop from here, I've created some before and after hooks for each spec where we have VRT Visual Tests.

In case:

• spec 1 starts on parallel runner 1, and the before hook starts VRT
• spec 2 also starts soon after this and will run the vrtStart command as well, on parallel runner 2
• will there be any conflicts in VRT? (the start command was issued twice, from 2 different machines, but on the same VRT project)
  • it's probably good to mention that all specs in the Cypress project would use the same VRT project

What about vrtStop? Let's say:

• spec 1 runs vrtStart on parallel runner 1
• spec 2 runs vrtStart as well, but on parallel runner 2
• spec 1 finishes and runs vrtStop on parallel runner 1
• spec 2 is still running and has Visual Tests to execute (still on parallel runner 2)
  • will spec 2 be impacted by the vrtStop command issued by spec 1 on parallel runner 1?

[pashidlos]

thanks for your question

in order to sync parallel threads you should use ciBuildId as unique key
this will not start new build as soon as exists one

stop only could be called several times for the same build with no functional impact, its only visual representation on UI

[alexniculae]

Thank you for sharing this @pashidlos.

Following some details from our discussion here it is my understanding then that:

• vrtStart will just create the new build
• vrtStop will remove the build element loading bar from the DOM
• the project can function just as well without the vrtStop command
  • only that the loading bar will never be removed from the build element

Is VRT then able to create a new build without even requiring the vrtStart command? Or will this cause an error, as the build does not exist?
   - if vrtStart is mandatory, then an option would be to run vrtStart instantly followed by vrtStop
   - this would then create the new build but dismiss the build element loading bar from the DOM
   - could you please confirm if this is a valid workaround for using vrtStart && vrtStop?

I am looking into a solution to allow multiple (parallel) threads to record Visual Tests in the same VRT 'build', at the same time - in theory this should work I guess, will just put more stress on the VRT engine

Another solution would be to create dedicated jobs in the CI just for starting and stopping VRT before and after running the tests (which seems a bit much if not currently needed for a technical reason).

Update: or I can just leave it as suggested in the example projects, and call vrtStart before and vrtStop after each spec, as it would not change anything for the runs anyway (would just generate a lot of unrequired calls to the VRT backend).

[pashidlos]

   - if vrtStart is mandatory, then an option would be to run vrtStart instantly followed by vrtStop

yes, start is mandatory but you could call stop in teardown the only thing that it's gonna be called multiple times in cypress
like in example https://github.com/Visual-Regression-Tracker/examples-js/blob/master/agent-cypress/cypress/integration/vrt.spec.js

I am looking into a solution to allow multiple (parallel) threads to record Visual Tests in the same VRT 'build', at the same time - in theory this should work I guess, will just put more stress on the VRT engine

feel free to use multiple threads as Im using them as well
adding ciBuildId solving that issue so no additional workaround is needed
https://github.com/Visual-Regression-Tracker/frontend/blob/master/.github/workflows/workflow.yml#L30

no worries about multiple call so backend as it should sustain the load for such low resource consuming calls
currently only submit test run with really big image could make any noticeable load