Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request - Include timestamp of 1st failure event in CircuitBreakerOnFailureRateExceededEvent #2145

Open
laksnv opened this issue Apr 12, 2024 · 5 comments

Comments

@laksnv
Copy link
Contributor

laksnv commented Apr 12, 2024

I'd like the ability to know how long it takes for Circuit Breaker to trip: T2 - T1

T2: Timestamp when failure ratio exceeds threshold
T1: Timestamp of first observed failure

If this data was made available through CircuitBreakerOnFailureRateExceededEvent, it would be perfect for clients to record how long detection took. Let me know what you think.

I couldn't find a similar request from the past; please let me know if I missed any.

@RobWin
Copy link
Member

RobWin commented Apr 15, 2024

Hi,
this kind of information is currently not available. A CircuitBreaker does not store any timestamps.
From the design perspective we wanted to enable it with Event Sourcing. Clients can consume Events and track any other kind of metric/information which is important for them.

@laksnv
Copy link
Contributor Author

laksnv commented Apr 15, 2024

Thanks for getting back!

Even if clients rely on maintaining a record of failure events, they still will need to maintain a sliding window to determine the first failure in the window. Am I understanding it right?

@RobWin
Copy link
Member

RobWin commented Apr 15, 2024

That's a good point.
Only option is to consume the CircuitBreakerStateTransitionEvent and then calculate backwards based on the failure rate threshold. But it's cumbersome.
But storing the timestamp in the sliding window would introduce a lot of changes.

@laksnv
Copy link
Contributor Author

laksnv commented Apr 17, 2024

We just need to keep track of timestamp of first reported failure, if any, for every second in the sliding window and include this detail when creating a snapshot. CircuitBreaker.Metrics will expose a method to get details about failure timestamps (whether getting the earliest reported failure timestamp or similar). When publishing rate exceeded events, timestamp detail could be included:

publishCircuitFailureRateExceededEvent(getName(), circuitBreakerMetrics.getFailureRate());
.

What else is needed?

@laksnv
Copy link
Contributor Author

laksnv commented Apr 18, 2024

It may be better to only expose earliest failure timestamp on demand i.e. only when generating error events.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants