Skip to content

Commit

Permalink
Unflake BenchmarkSampler_CheckWithHook (#1016)
Browse files Browse the repository at this point in the history
BenchmarkSampler_CheckWithHook was randomly failing esp. under load because under load, Zap sampler can end up dropping less than the expected amount (esp. with Hooks). Moreover, we were making a single comparison after all the benchmarks had run, which causes the time dilation effect to aggregate over all the tests, making it more likely to fail.

This changes the benchmark to compare the dropped and sampled ratio in between test runs and test the dropped/sampled ratio. The change was verified by running the benchmark 500 times on a MacBook Pro i7 model as well as devpod (96 core Linux) and confirming no failures happened.

Fix #996
Internal Ref: GO-858
  • Loading branch information
sywhang committed Oct 15, 2021
1 parent 10d89a7 commit 3d55364
Showing 1 changed file with 6 additions and 4 deletions.
10 changes: 6 additions & 4 deletions zapcore/sampler_bench_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -274,10 +274,12 @@ func BenchmarkSampler_CheckWithHook(b *testing.B) {
}
}
})
// We expect to see 1000 dropped messages for every sampled per settings,
// with a delta due to less 1000 messages getting dropped after initial one
// is sampled.
assert.Greater(b, dropped.Load()/1000, sampled.Load()-1000)
dropped.Store(0)
sampled.Store(0)
})
}
// We expect to see 1000 dropped messages for every sampled per settings,
// with a delta due to less 1000 messages getting dropped after initial one
// is sampled.
assert.Greater(b, dropped.Load()/1000, sampled.Load()-1000)
}

0 comments on commit 3d55364

Please sign in to comment.