Simplify tracking of task counts stats in TaskQueue #16423
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
The current model of tracking task counts for the
TaskCountStatsMonitor
is too limiting and verbose.Changes
TaskCountStatsProvider
CoordinatorRunStats
toDruidRunStats
as it is now being extensively used outside the coordinator.TaskQueue
.getRunningTaskCount
,getWaitingTaskCount
, etc.DruidRunStats
instance which is reset upon invocation ofgetQueueStats()
taskType
to task count metricsClasses to review
TaskQueue
TaskCountStatsProvider
TaskCountStatsMonitor
TaskMaster
Further work
Design considerations
The few advantages of the previous design were that:
DruidRunStats
.getStats()
method which returns a packagedDruidRunStats
, it is possible for the concrete implementation to return any or no metrics inside theDruidRunStats
. Still thinking of a clean way to enforce this. I suppose we could add logic in the monitor to ensure that all required metrics have been sent by the implementation.This PR has: