Measuring incident response time

[mrpea]

Hi - I'm wondering if the Dispatch team (or others) are measuring the time between when an incident is created and when the "owner" of the incident starts working on it.

This is typically easier to measure in PagerDuty where you have automated incident creation and then a person needs to acknowledge the incident.

It could also be a field for recording the timing of incident root cause. I'm trying to understand the time difference between when something breaks and when people start to fix it.

Any ideas?

[mvilanova]

Hey @mrpea we have thought about this in the context of measuring if we're meeting our SLAs for responding to incidents and our approach was going to be recording the timestamp of when the incident commander messages the incident channel for the first time and calculate the delta between that and when the incident is opened, but we haven't had the time to implement it. What are your thoughts about this approach?

[mrpea]

I like this approach. Additionally we have incidents identified but not opened until a later time. In that scenario the created is essentially the same as the acknowledged but we would probably manually change the created time to the "detected" time.

Do you think it would be possible to add an acknowledged_at column and UI field for manual entry now and iterate towards auto-calculating the value in the future? If so is this something your team would be willing to take on?

[mrpea]

@mvilanova Any thoughts on this one? Being totally honest I feel like going low-tech and iterating works well for us as we can manually backfill the data or hit API as needed.

[mvilanova]

Yes, I think it would make sense to have a dedicated column and field, so ICs can manually change it if necessary. Cc'ing @whitdog47 @metroid-samus for viz.

[mrpea]

@mvilanova What's the best way to move forward on this? Is it something you can take on or should we submit a PR? Also given I haven't made changes to the db or models yet, any example PRs off the top of your head?

[metroid-samus]

Hi, the Dispatch team is working on updating the cost model from the current passive aggregation strategy to a new event-based cost aggregation, which should cover your use case.