Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix output precision in WithinRelMatcher #2846

Merged
merged 2 commits into from
Apr 15, 2024
Merged

Fix output precision in WithinRelMatcher #2846

merged 2 commits into from
Apr 15, 2024

Conversation

ianhbell
Copy link
Contributor

Description

The WithinRelMatcher does not listen to the precision specification; it just naively pipes to stringstream. If you specify the precision, that has no impact on the outputted values when the match fails.

This fix makes the WithinRelMatcher listen to the precision (https://github.com/catchorg/Catch2/blob/devel/docs/tostring.md#floating-point-precision) specified by: Catch::StringMaker<float>::precision

@ianhbell ianhbell changed the title Update catch_matchers_floating_point.cpp Fix output precision in WithinRelMatcher Mar 19, 2024
@horenmar
Copy link
Member

If this is changed to pass tests, I am willing to merge it.

@ianhbell
Copy link
Contributor Author

What sort of tests do you have in mind?

@horenmar
Copy link
Member

Well, right now the change does not pass CI because of the output format change. See contributing.md for how to run the tests locally.

@ianhbell
Copy link
Contributor Author

I see - this breaks quite a few tests.

@ianhbell
Copy link
Contributor Author

I tried to fix the tests, but the tests are failing on my machine.

I tried to run on my (Apple M1) machine:

(base) XXX:Catch2 ihb$ ./tools/scripts/buildAndTest.sh

This fails:

[ 45%] Building CXX object tests/CMakeFiles/SelfTest.dir/SelfTest/IntrospectiveTests/RandomNumberGeneration.tests.cpp.o
/Users/ihb/Documents/Code/ianhbellCatch2/Catch2/tests/SelfTest/IntrospectiveTests/RandomNumberGeneration.tests.cpp:503:32: error: unused variable 'lowest' [-Werror,-Wunused-const-variable]
        static constexpr float lowest = -256.125f;
                               ^
/Users/ihb/Documents/Code/ianhbellCatch2/Catch2/tests/SelfTest/IntrospectiveTests/RandomNumberGeneration.tests.cpp:548:52: error: unused variable 'expected' [-Werror,-Wunused-const-variable]
    constexpr float uniform_fp_test_params<float>::expected[];
                                                   ^
/Users/ihb/Documents/Code/ianhbellCatch2/Catch2/tests/SelfTest/IntrospectiveTests/RandomNumberGeneration.tests.cpp:525:33: error: unused variable 'lowest' [-Werror,-Wunused-const-variable]
        static constexpr double lowest = -234582.9921875;
                                ^
/Users/ihb/Documents/Code/ianhbellCatch2/Catch2/tests/SelfTest/IntrospectiveTests/RandomNumberGeneration.tests.cpp:549:54: error: unused variable 'expected' [-Werror,-Wunused-const-variable]
    constexpr double uniform_fp_test_params<double>::expected[];

and then a bunch of other tests fail:

The following tests FAILED:
	  1 - RunTests (Not Run)
	  2 - List::Tests::Output (Not Run)
	  3 - List::Tests::Quiet (Not Run)
	  4 - List::Tests::ExitCode (Not Run)
	  5 - List::Tests::XmlOutput (Not Run)
	  6 - List::Tags::Output (Not Run)
	  7 - List::Tags::ExitCode (Not Run)
	  8 - List::Tags::XmlOutput (Not Run)
	  9 - List::Reporters::Output (Not Run)
	 10 - List::Reporters::ExitCode (Not Run)
	 11 - List::Reporters::XmlOutput (Not Run)
	 12 - List::Listeners::Output (Not Run)
	 13 - List::Listeners::ExitCode (Not Run)
	 14 - List::Listeners::XmlOutput (Not Run)
	 15 - NoAssertions (Not Run)
	 16 - TestSpecs::CombiningMatchingAndNonMatchingIsOk-1 (Not Run)
	 17 - TestSpecs::CombiningMatchingAndNonMatchingIsOk-2 (Not Run)
	 18 - TestSpecs::NoMatchedTestsFail (Not Run)
	 19 - TestSpecs::OverrideFailureWithNoMatchedTests (Not Run)
	 20 - TestSpecs::OverrideAllSkipFailure (Not Run)
	 21 - TestSpecs::NonMatchingTestSpecIsRoundTrippable (Not Run)
	 22 - Warnings::UnmatchedTestSpecIsAccepted (Not Run)
	 23 - Warnings::MultipleWarningsCanBeSpecified (Not Run)
	 24 - TestSpecs::WarnUnmatchedTestSpecFailsWithUnmatchedTestSpec (Not Run)
	 25 - UnmatchedOutputFilter (Not Run)
	 26 - FilteredSection-1 (Not Run)
	 27 - FilteredSection-2 (Not Run)
	 28 - FilteredSection::GeneratorsDontCauseInfiniteLoop-1 (Not Run)
	 29 - FilteredSection::GeneratorsDontCauseInfiniteLoop-2 (Not Run)
	 30 - ApprovalTests (Failed)
	 31 - RegressionCheck-1670 (Not Run)
	 32 - VersionCheck (Not Run)
	 33 - LibIdentityTest (Not Run)
	 34 - FilenameAsTagsTest (Not Run)
	 35 - FilenameAsTagsMatching (Not Run)
	 36 - EscapeSpecialCharactersInTestNames (Not Run)
	 37 - NegativeSpecNoHiddenTests (Not Run)
	 38 - TestsInFile::SimpleSpecs (Not Run)
	 39 - TestsInFile::EscapeSpecialCharacters (Not Run)
	 40 - TestsInFile::InvalidTestNames-1 (Not Run)
	 41 - TagAlias (Not Run)
	 42 - RandomTestOrdering (Failed)
	 44 - Benchmarking::SkipBenchmarkMacros (Not Run)
	 45 - Benchmarking::FailureReporting::ThrowingBenchmark (Not Run)
	 46 - Benchmarking::FailureReporting::FailedAssertion (Not Run)
	 47 - Benchmarking::FailureReporting::FailMacro (Not Run)
	 48 - Benchmarking::FailureReporting::ShouldFailIsRespected (Not Run)
	 49 - ErrorHandling::InvalidTestSpecExitsEarly (Not Run)
	 50 - MultiReporter::CapturingReportersDontPropagateStdOut (Not Run)
	 51 - MultiReporter::NonCapturingReportersPropagateStdout (Not Run)
	 52 - Outputs::DashAsOutLocationSendsOutputToStdout (Not Run)
	 53 - Reporters::DashAsLocationInReporterSpecSendsOutputToStdout (Not Run)
	 54 - Reporters::ReporterSpecificColourOverridesDefaultColour (Not Run)
	 55 - Reporters::UnrecognizedOptionInSpecCausesError (Not Run)
	 56 - Colours::ColourModeCanBeExplicitlySetToAnsi (Not Run)
	 57 - Reporters::JUnit::NamespacesAreNormalized (Not Run)
	 63 - Reporters:Filters:compact (Not Run)
	 64 - Reporters:RngSeed:compact (Not Run)
	 65 - Reporters:Filters:console (Not Run)
	 66 - Reporters:RngSeed:console (Not Run)
	 67 - Reporters:Filters:JUnit (Not Run)
	 68 - Reporters:RngSeed:JUnit (Not Run)
	 69 - Reporters:Filters:SonarQube (Not Run)
	 70 - Reporters:RngSeed:SonarQube (Not Run)
	 71 - Reporters:Filters:TAP (Not Run)
	 72 - Reporters:RngSeed:TAP (Not Run)
	 73 - Reporters:Filters:XML (Not Run)
	 74 - Reporters:RngSeed:XML (Not Run)
	 75 - Reporters:Filters:JSON (Not Run)
	 76 - Reporters:RngSeed:JSON (Not Run)
	 77 - TestShardingIntegration (Failed)
	 78 - TestSharding::OverlyLargeShardIndex (Not Run)
	 79 - MinDuration::SimpleThreshold (Not Run)
	 80 - MinDuration::DurationOverrideYes (Not Run)
	 81 - MinDuration::DurationOverrideNo (Not Run)
	 82 - CATCH_CONFIG_PREFIX_ALL (Not Run)
	 83 - CATCH_CONFIG_DISABLE-1 (Not Run)
	 84 - CATCH_CONFIG_DISABLE-2 (Not Run)
	 85 - CATCH_CONFIG_DISABLE_EXCEPTIONS-1 (Not Run)
	 86 - CATCH_CONFIG_BAZEL_REPORTER-1 (Failed)
	 87 - NO_CATCH_CONFIG_BAZEL_REPORTER-1 (Failed)
	 88 - BazelEnv::TESTBRIDGE_TEST_ONLY (Not Run)
	 89 - BazelEnv::Sharding (Failed)
	 90 - CATCH_CONFIG_DISABLE_EXCEPTIONS-2 (Not Run)
	 91 - CATCH_CONFIG_DISABLE_EXCEPTIONS-3 (Not Run)
	 92 - CATCH_CONFIG_DISABLE_EXCEPTIONS-4 (Not Run)
	 93 - DeferredStaticChecks (Not Run)
	 94 - FallbackStringifier (Not Run)
	 95 - CATCH_CONFIG_DISABLE_STRINGIFICATION (Not Run)
	 96 - PartialTestCaseEvents (Failed)
	 97 - BenchmarksInCumulativeReporter (Not Run)
	 98 - Reporters::registration-is-case-preserving (Not Run)
	 99 - Reporters::selection-is-case-insensitive (Not Run)
	100 - Reporters::CapturedStdOutInEvents (Not Run)
	101 - MultiReporter::NoncapturingListenerDoesntCauseStdoutPassThrough (Not Run)
	102 - MultiReporter::CapturingListenerCausesStdoutCapture (Not Run)
	103 - Reporters::PreferencesForPassingAssertionsIsRespected (Not Run)
	104 - MultiReporter::PreferencesForPassingAssertionsIsRespected (Not Run)
	105 - ListenersGetEventsBeforeReporters (Not Run)
	106 - CustomArgumentsForReporters (Not Run)
	107 - DuplicatedTestCases::SameNameAndTags (Not Run)
	108 - DuplicatedTestCases::SameNameDifferentTags (Not Run)
	109 - DuplicatedTestCases::DuplicatedTestCaseMethods (Not Run)
	110 - DuplicatedTestCases::DuplicatedTestCaseMethodsDifferentFixtures (Not Run)
	111 - Reporters::RegistrationErrorsAreCaught (Not Run)
	112 - ReporterEvents::AssertionStartingHappensBeforeAssertionIsEvaluated (Not Run)
	113 - TestSpecs::EmptySpecWithNoTestsFails (Not Run)
	114 - TestSpecs::OverrideFailureWithEmptySpec (Not Run)
	115 - List::Listeners::WorksWithoutRegisteredListeners (Not Run)
	116 - TestSpecs::SkippingAllTestsFails (Not Run)
	117 - AmalgamatedFileTest (Not Run)

@horenmar
Copy link
Member

horenmar commented Apr 6, 2024

Oh right, the warning issue in #2845.

I should have that fixed in a bit.

@horenmar
Copy link
Member

horenmar commented Apr 6, 2024

Done. The build on M1 mac should no longer cause warning and thus fail due to Werror.

@ianhbell
Copy link
Contributor Author

I think I have fixed the tests, waiting on the tests to run to be sure.

@codecov Codecov
Copy link

codecov bot commented Apr 14, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.09%. Comparing base (029fe3b) to head (5a5382f).

Additional details and impacted files 
@@           Coverage Diff           @@
##            devel    #2846   +/-   ##
=======================================
  Coverage   91.09%   91.09%           
=======================================
  Files         197      197           
  Lines        8382     8382           
=======================================
  Hits         7635     7635           
  Misses        747      747

@ianhbell
Copy link
Contributor Author

Ready for merge I think

@horenmar
Copy link
Member

Yep, thanks.

@horenmar horenmar merged commit 53ddf37 into catchorg:devel Apr 15, 2024
81 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@ianhbell @horenmar