PERF-3888 Add low-priority background operation to sinusoidal workload #841
Conversation
…sinusoidal workload
| Active: [1] | ||
| NopInPhasesUpTo: *NumPhases | ||
| PhaseConfig: | ||
| # Add a large set of expired documents at start to make sure TLL monitor doesn't catch up in future phases. |
There was a problem hiding this comment.
Did we confirm in the FTDC data that we were indeed always behind on TTL deletes?
There was a problem hiding this comment.
Based on the FTDC data, the TTL monitor is deleting fewer documents than it was deleting after the load phase, but @haleyConnelly is there another metric I should be using besides deletedDocuments to measure this?
There was a problem hiding this comment.
Hm, in general there should be a queueLength of non-zero to show low priority operations are at least queueing. Compared to the load phase, aren't there more threads the TTL is competing with in phase 2?
|
I'm not observing "queueLength" in the FTDC data, but I am seeing a lot of other "concurrentTransactions write lowPriority" metrics. Is queueLength nested in another metric? |
| OperationMetricsName: ReadDocs | ||
| ThrowOnFailure: false | ||
|
|
||
| - TemplateName: UpdatesActorWithSleepTemplate |
There was a problem hiding this comment.
Updating indexed documents definitely adds more stress to the system. Though the "eviction worker thread active" doesn't exceed 4, it is consistently at 4, which would imply that they're doing a lot of work? Let me know if you have other suggestions.
With this phase, we are able to see pretty decent improvements in average throughput and latency.
There was a problem hiding this comment.
How big were the throughput/latency improvements? And how about the percentile latencies?
| OperationMetricsName: ReadDocs | ||
| ThrowOnFailure: false | ||
|
|
||
| - TemplateName: UpdatesActorWithSleepTemplate |
There was a problem hiding this comment.
How big were the throughput/latency improvements? And how about the percentile latencies?
| Active: [1] | ||
| NopInPhasesUpTo: *NumPhases | ||
| PhaseConfig: | ||
| # Add a large set of expired documents at start to make sure TLL monitor doesn't catch up in future phases. |
There was a problem hiding this comment.
Hm, in general there should be a queueLength of non-zero to show low priority operations are at least queueing. Compared to the load phase, aren't there more threads the TTL is competing with in phase 2?
sys perf evergreen patch