Report generation broken if multiple browsers run in parallel

[dpoetzsch]

Describe the bug
If multiple browsers run in parallel the generated report is often broken. I guess this is because multiple processes are accessing the report.json file in parallel, leading to all kinds of race conditions.

To Reproduce
Steps to reproduce the behavior:

1. Run multple browsers in parallel
2. The generated report often only contains results from one browser.

Expected behavior

I'd expect all matching results to be present in the report.json. Alternatively, it would be totally ok, if each browser generates its own report.json.

Desktop (please complete the following information):

• OS: Fedora Linux 31
• Browser Chrome + Firefox (both headless)
• Version 1.0.0

Additional context
Add any other context about the problem here.

[ennjin]

Hi, I checked parallel performance and got the correct result. You can checkout for master branch, build the project and run e2e job.

[dpoetzsch]

Interesting. I actually cannot reproduce the behavior with the tests in this repository as well.

However, I can reproduce it reliably with my test suite. Sadly this is a private repo, so I cannot provide access to it, but I can show you what I mean in a screencast: https://streamable.com/x0gih.

Something I did notice though, is that your report contains a browser property, while mine doesn't. I will continue investigating...

EDIT: Just checked, the browser property is probably not it. Maybe it is a cucumber thing?