Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

containAllInAnyOrder fails to detect mismatch in element counts, as long as the overall counts of elements match #3881

Closed
AlexCue987 opened this issue Feb 16, 2024 · 2 comments · Fixed by #3907
Labels
assertions 🔍 Related to the assertion mechanisms within the testing framework. bug 🐛 Issues that report a problem or error in the code.

Comments

@AlexCue987
Copy link
Member

AlexCue987 commented Feb 16, 2024

latest from master
Presumably containAllInAnyOrder should detect mismatch in element counts, and sometimes it does, as follows:

        "detect count mismatch" {
            sequenceOf("apple", "apple", "orange") should containAllInAnyOrder(
                sequenceOf("apple", "apple", "orange", "orange")
            )
        }
-----------
Sequence should contain the values of [apple, apple, orange, orange] in any order, but was [apple, apple, orange]

However, the following test passes, which seems counter-intuitive to me:

        "fail to detect count mismatch" {
            sequenceOf("apple", "apple", "orange") should containAllInAnyOrder(
                sequenceOf("apple", "orange", "orange")
            )
        }

Interestingly, when I tried to fix the issue by comparing counts of elements, it broke two existing tests, as follows:

        succeed("for subset, same count with nulls") {
            sampleData.sparse.shouldContainAllInAnyOrder(sampleData.nulls)
         }
----
Sequence should contain the values of [null, null, null, null] in any order, but was [null, null, null, 3].
----
         fail("for subset, same count with nulls") {
            sampleData.sparse.shouldNotContainAllInAnyOrder(sampleData.nulls)
         }
------
Expected exception java.lang.AssertionError but no exception was thrown.
// this test matches [null, null, null, null] with [null, null, null, 3] - these seems different to me and as such the test should pass

My understanding is that these two tests are not validating correct behavior, and as such they should be fixed.
My understanding is that these two tests should not be passing in the first place.
If this is correct, I have a fix and can raise a PR.

@LeoColman LeoColman added bug 🐛 Issues that report a problem or error in the code. assertions 🔍 Related to the assertion mechanisms within the testing framework. labels Mar 3, 2024
@LeoColman
Copy link
Member

I believe you're right. Feel free to submit a PR :)

@AlexCue987
Copy link
Member Author

I believe you're right. Feel free to submit a PR :)

there you go:

#3907

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
assertions 🔍 Related to the assertion mechanisms within the testing framework. bug 🐛 Issues that report a problem or error in the code.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants