Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#735 Removed throwing new AssertionFailedError when test failed #788

Conversation

Edmnikishin
Copy link

Suggestion for: #735

My point is just throw an exception of what happend, because this annotation (in my opinion) is used to get rid of flaky tests (in api/e2e test especially). Or if it too flaky/broken, so we should fix it.

Thing what annoyed me in my practice, that when you try to make an allure report you'll have same fail issue (Which is not true) like "Test execution #3 (of up to 3 with at least 1 successes) failed ~ test fails - see cause for details"
And second is that you need to dive to previous exception/allure step to know what happened

So what do i mean. Here is how it works right now (without my PR)

Here how it'll work with mine PR

And that's how it'll work if we throw MultipleFailuresError instead AssertionFailedError. In my opinion it bloats fail message and when test failed you want to know cause. And if you want to know what retry it is you can just look at previous retry attempts


PR checklist

The following checklist shall help the PR's author, the reviewers and maintainers to ensure the quality of this project.
It is based on our contributors guidelines, especially the "writing code" section.
It shall help to check for completion of the listed points.
If a point does not apply to the given PR's changes, the corresponding entry can be simply marked as done.

Documentation (general)

  • There is documentation (Javadoc and site documentation; added or updated)
  • There is implementation information to describe why a non-obvious source code / solution got implemented
  • Site documentation has its own .adoc file in the docs folder, e.g. docs/report-entries.adoc
  • Site documentation in .adoc file references demo in src/demo/java instead of containing code blocks as text
  • Only one sentence per line (especially in .adoc files)
  • Javadoc uses formal style, while sites documentation may use informal style

Documentation (new extension)

  • The docs/docs-nav.yml navigation has an entry for the new extension
  • The package-info.java contains information about the new extension

Code (general)

  • Code adheres to code style, naming conventions etc.
  • Successful tests cover all changes
  • There are checks which validate correct / false usage / configuration of a functionality and there are tests to verify those checks
  • Tests use AssertJ or our own PioneerAssert (which are based on AssertJ)

Code (new package)

  • The new package is exported in module-info.java
  • The new package is also present in the tests
  • The new package is opened for reflection to JUnit 5 in module-info.java
  • The new package is listed in the contribution guide

Contributing

  • A prepared commit message exists
  • The list of contributions inside README.adoc mentions the new contribution (real name optional)

@Michael1993
Copy link
Member

Has the same problem as discussed in #735 - it hides why previous executions failed.

@Edmnikishin
Copy link
Author

Edmnikishin commented Nov 9, 2023

Okay, so i've tested and pushed it with adding previous errors and it doesn't look bad, actually nice :)
Here what i've got in allure
image

And here is console
image

@nipafx
Copy link
Member

nipafx commented Nov 9, 2023

Thank you for the contribution, @Edmnikishin. I ended up creating my own change that does indeed throw the MultipleFailuresError mentioned in #735. I also changed how exceptions for aborted tests are rethrown and made sure the original exception's message comes first to ease parsing for tools (like you did here.

I'll close this PR.

@nipafx nipafx closed this Nov 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants