Add aggregate_failures option

[sheremet-va]

Clear and concise description of the problem

I think it might be useful to allow aggregating failures inside a single test block.

Suggested solution

RSpec, Ruby testing tool, has an interesting API called "aggregate_failures". It allows users to group expectations inside a single test instead of aborting the test after the first expect. From the docs:

RSpec::Expectations provides aggregate_failures, an API that allows you to group a set of expectations and see all the failures at once, rather than it aborting on the first failure.

The metadata form is quite convenient, but may not work well for end-to-end tests that have multiple distinct steps. For example, consider a spec for an HTTP client workflow that (1) makes a request, (2) expects a redirect, (3) follows the redirect, and (4) expects a particular response. You probably want the expect(response.status).to be_between(300, 399) expectation to immediately abort if it fails, because you can't perform the next step (following the redirect) if that is not satisfied. For these situations, we encourage you to use the aggregate_failures block form to wrap each set of expectations that represents a distinct step in the test workflow.

This API can also be useful for displaying errors in type checking context, since it's possible to have several type errors inside a single test. Also docs give a lot of scenarios, where this behaviour is useful.

The proposed API is:

test('name', () => {
  expect(1).toBe(2)
  expect('a').toBe('b')
}, { aggregateFailures: true }) // false by default

Alternative

No response

Additional context

This API cannot work with 3rd party assertions, since they throw error, different from chai, and cannot be prevented.

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

[sheremet-va]

Superseded by #3435