You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The "simulator" we use in some of our randomized tests uses a random seed. The idea is that if a random test fails, we could take that seed and reproduce the exact failure conditions again to debug them. Originally, this seed was printed to stdout, which would be captured by the test runner, then later validated. Unfortunately, when I used the same code for benchmarking, stdout was not captured. I had to suppress this message.
Ideally, if we are benchmarking, we would suppress this output, but in regular test runs we wouldn't. It is possible that we could test debug_assertions for this on that basis that benchmarks run in release mode and tests run in debug mode, but I'm not 100% confident that this is the right way to do that.
The text was updated successfully, but these errors were encountered:
This is harder than it sounds.
The "simulator" we use in some of our randomized tests uses a random seed. The idea is that if a random test fails, we could take that seed and reproduce the exact failure conditions again to debug them. Originally, this seed was printed to stdout, which would be captured by the test runner, then later validated. Unfortunately, when I used the same code for benchmarking, stdout was not captured. I had to suppress this message.
Ideally, if we are benchmarking, we would suppress this output, but in regular test runs we wouldn't. It is possible that we could test
debug_assertions
for this on that basis that benchmarks run in release mode and tests run in debug mode, but I'm not 100% confident that this is the right way to do that.The text was updated successfully, but these errors were encountered: