Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark test for MachineComplexityEvaluator #26

Open
jonessha opened this issue Aug 22, 2022 · 0 comments
Open

Benchmark test for MachineComplexityEvaluator #26

jonessha opened this issue Aug 22, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@jonessha
Copy link
Contributor

Add a benchmark test for MachineComplexityEvaluator to detect performance regressions in machine complexity evaluation.

@[baldawar]
don't think its worth blocking on, but probably worth adding a benchmark test around this class to ensure evaluator doesn't grossly become inefficient after refactoring. for now, could be github issue that we address later.

@[jonessha]
This is a good idea. Note that we partially have this through testEvaluateBeyondMaxComplexity in MachineComplexityEvaluatorTest. This makes sure that we cap evaluation at max complexity. If we don't, the test will more or less spin forever. So this provides some protection. Still a good idea to have a dedicated benchmark test though. Some consumers of this library might run with a really high max complexity and be really sensitive to latency regressions.

Discussion from #18

@jonessha jonessha added the enhancement New feature or request label Aug 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant