Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output seed to stdout for tests #1645

Open
martinthomson opened this issue Feb 12, 2024 · 0 comments
Open

Output seed to stdout for tests #1645

martinthomson opened this issue Feb 12, 2024 · 0 comments

Comments

@martinthomson
Copy link
Member

This is harder than it sounds.

The "simulator" we use in some of our randomized tests uses a random seed. The idea is that if a random test fails, we could take that seed and reproduce the exact failure conditions again to debug them. Originally, this seed was printed to stdout, which would be captured by the test runner, then later validated. Unfortunately, when I used the same code for benchmarking, stdout was not captured. I had to suppress this message.

Ideally, if we are benchmarking, we would suppress this output, but in regular test runs we wouldn't. It is possible that we could test debug_assertions for this on that basis that benchmarks run in release mode and tests run in debug mode, but I'm not 100% confident that this is the right way to do that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant