AssertionFailedException: What's the benefit of the expected value?

[mmichaelis]

What I like about IDEs knowing about actual and expected: They can clearly show them in console and provide a "diff" function. Currently they have to parse the exception message and guess expected and actual value from it.

But this only makes sense, if we expect a certain value... but already fails for negation, such as not null, or not equal to, ... With the current implementation of AssertionFailedError I see no other option than to leave the expected value empty. But then IDEs and other reporting tools will have less options to highlight the expectation.

So perhaps it is better to misuse the expected value in these cases and provide a descriptive string like "not null", "not equal to". Example:

Actual: Foo
Expected: not one of [Foo, Bar]

An IDE might consider it is nice to provide a "diff" button. But in this case the diff would not be of any value.

At least the use case of "expected" should be clearly defined so that assertion framework developers deal with it in the very same way as IDE developers expect it to be.

Side Note: Perhaps it might make sense to even provide some on-demand-diff hook if the assertion frameworks knows better how to display the difference.

[mmichaelis]

Also the actual state can be much more complex than to put it into a simple value:

Phase	Test	On Mismatch
Initialization	Expect Environment: Windows	TestSkippedException reporting actual environment
Preparation	Expect: HTML textfield becomes visible	TestAbortedException reporting actual visibility of textfield (either does not exist or is hidden). Possibly error message enhanced by complete DOM and Screenshot.
Assertion	Expect: Text entered to the textfield occurs inside some other div	AssertionFailedException reporting current state of div. Possibly error message enhanced by complete DOM and Screenshot, too.

It would be great if upon failure I could directly open the accompanying files like HTML dump or the screenshot from the following sources:

• build output on console - in many consoles you can just click the URL
• from inside the IDE with one click
• in CI environments like Hudson/Jenkins (btw. perhaps these projects should be contacted, too) I would like to click a link in the web overview to see the files

So what we really need is a bunch of debug information upon failure. Even more mission critical for integration and system tests where you are often not able to reproduce the failure locally. So perhaps an exception could carry just an extra object DebugInformation which is:

• either actual/expected value
• an actual value and some descriptive missed expectation
• file information to accompanying files providing more debug information (screenshots, DOM, environment properties, ...)

This DebugInformation could be contained in any of the existing exceptions.

The more and more and I think about a DebugInformationObject the more I like it. We could even have specialized DebugInformationObjects. One for console output (the default, I guess), one for Eclipse, one for IntelliJ Idea, one for Jenkins, ... Just like a clipboard it could have different flavors and the assertion framework developer might decide to support different flavors as debug information.

Ok, think I should now wait and see what you think about this idea. Is it worth to give it a try?

[marcphilipp]

It's certainly true that the current AssertionFailedError lacks support for anything but the trivial equals mismatch. I think something along the lines of your proposal makes sense! I didn't get the last part regarding specialized objects for different IDEs. How would that look in terms of code?