feat: introduce retry option for tests

[antfu]

close #1057

[sheremet-va]

Can we show anything about retries in terminal for debugging?

[antfu]

What would the format be like in your mind? Like ✅ test name (x3) or ✅ test name (retry x3)?

[sheremet-va]

What would the format be like in your mind? Like ✅ test name (x3) or ✅ test name (retry x3)?

I think we should give brief description about errors, not just the counter.

[antfu]

Can you share a more detailed layout/text design, or directly push/pr to this branch?

[sheremet-va]

Can you share a more detailed layout/text design, or directly push/pr to this branch?

tbh this question is why I didn't start this PR by myself 😄

I feel like we need some kind of indicator, but I have poor imagination 😃

[antfu]

I will start with indicators and leave it for future improvements then

[sheremet-va]

Can we add a global option for this?

[antfu]

I am not yet sure if we want the global option for this tbh. We already support describe level and it should be enough for now I think.

[lnhrdt]

I am not yet sure if we want the global option for this tbh. We already support describe level and it should be enough for now I think.

Today I looked for a way to configure retries globally and I came across this PR. I was hoping to configure test retries globally in CI (where resources are limited and it makes my tests flakier). I had assumed there would be an option in my vite configuration where I could configure it based on something like process.env.CI. Would you reconsider providing a global configuration option @antfu?

[xsjcTony]

Yeah, that would be awesome if we can have this option globally since CI is just unstable sometimes. The approach that Playwright takes is just ✨

[giovannivocale-molar]

Is there an option to specify retry in the global settings for vitest, so that automatically retries X number of times if a test is failing?

[sheremet-va]

Is there an option to specify retry in the global settings for vitest, so that automatically retries X number of times if a test is failing?

I would recommend opening an enhancement issue if you need something,