Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document how to mark an observation as an error if the exception was handled #29848

Closed
jonatan-ivanov opened this issue Jan 18, 2023 · 3 comments
Assignees
Labels
in: web Issues in web modules (web, webmvc, webflux, websocket) theme: observability An issue related to observability and tracing theme: web-error-handling An issue related to web error handling type: documentation A documentation task
Milestone

Comments

@jonatan-ivanov
Copy link
Member

jonatan-ivanov commented Jan 18, 2023

Affects: 6.0.3 (6.0.x)

With the current version (6.0.3) (and Boot 3.1), if I throw out an error from an MVC controller method and if I have an @ExceptionHandler, the error is not reported to the observation, i.e.: observation.error(...) is not called.

The issue is reproducible with a simple Boot project (initializer) and adding a @RestController with an error handler:

@GetMapping("/")
public String trouble() {
	throw new IllegalStateException("Noooooo!");
}

@ExceptionHandler(Throwable.class)
ProblemDetail handleThrowable(HttpServletRequest request, Throwable error) {
	ProblemDetail problemDetail = ProblemDetail.forStatus(INTERNAL_SERVER_ERROR);
	problemDetail.setTitle(INTERNAL_SERVER_ERROR.getReasonPhrase());
	problemDetail.setDetail(error.toString());
	return problemDetail;
}

After I call the endpoint and see the error response (500), if I check the metrics (or traces), I can see that the exception was never reported to the observation (e.g.: by checking the /actuator/metrics endpoint of the tracing backend).
Here's what I see in the metrics endpoint (I simplified the JOSN):

exception: [ "none" ]
method: [ "GET" ]
error: [ "none" ]
uri: [ "/" ]
outcome: [ "SERVER_ERROR" ]
status: [ "500" ]

Both the exception and the error tags should contain the error (IllegalStateException) but they don't.

After debugging this, I think the issue is that the exception attribute is not set in mvc, if I set it manually in the @ExceptionHandler, the error is reported as expected:

request.setAttribute(ERROR_EXCEPTION_ATTRIBUTE, error);

If you need a reproducer on top of the initializer link and the snippets, you can use apps in micrometer-samples repo, e.g.: this one or this one but they already contain the workaround you need to remove it.

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged or decided on label Jan 18, 2023
@jonatan-ivanov
Copy link
Member Author

I think this is slightly related to #29398.

@bclozel bclozel added theme: web-error-handling An issue related to web error handling in: web Issues in web modules (web, webmvc, webflux, websocket) theme: observability An issue related to observability and tracing labels Jan 19, 2023
@bclozel
Copy link
Member

bclozel commented Jan 19, 2023

This was originally done on purpose in Spring Boot, as developers often handle business exceptions and handle them at the web layer, but don't consider those as "errors" in the observability sense. For example, a missing user profile results in a 404 but you probably don't want to clutter your dashboards with those errors just because a 404 happened. Right now, only exceptions that are bubbling up to the servlet filter are recorded as errors.

At the Spring MVC/WebFlux level, it's hard to know if an error can be considered worthy of an observation key-value, since we only know that an error has been handled at the MVC level, or not. With the ProblemDetail support, error handling is somehow expanded and we can reconsider this approach.

Currently, we advise developers to register exceptions with the observation directly.

@jonatan-ivanov
Copy link
Member Author

jonatan-ivanov commented Jan 19, 2023

I see. Can we have a section about this both in Boot's and Framework's docs (or Boot can just link to Framework's)? Since if the users want to observe and report errors happening in their apps they have a choice of not having a custom error handler or implement one. In the first case, they will bump into #29398, in the second case, the problem that is described above.

@bclozel bclozel added type: documentation A documentation task and removed status: waiting-for-triage An issue we've not yet triaged or decided on labels Jan 31, 2023
@bclozel bclozel added this to the 6.0.x milestone Jan 31, 2023
@bclozel bclozel self-assigned this Jan 31, 2023
@bclozel bclozel modified the milestones: 6.0.x, 6.0.5 Feb 2, 2023
@bclozel bclozel changed the title Errors are not signaled on Observations if using an @ExceptionHandler Document how to mark an observation as an error if the exception was handled Feb 2, 2023
@bclozel bclozel closed this as completed in 708f600 Feb 2, 2023
mdeinum pushed a commit to mdeinum/spring-framework that referenced this issue Jun 29, 2023
The `ServerHttpObservationFilter` implementations record observations
for processed HTTP exchanges. The `Observation.Context` contains various
metadata contributed by the observation convention. The instrumentation
can also mark the observation as an error by setting any `Throwable` on
the context.

Because the instrumentation is done as filters, only exceptions reaching
the filter can be considered. Any error handled at a lower level by the
Framework can, or cannot be considered as an error for an observation.

This commit documents how a web application should opt-in for
considering a handled exception as an error for the current observation.

Closes spring-projectsgh-29848
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in: web Issues in web modules (web, webmvc, webflux, websocket) theme: observability An issue related to observability and tracing theme: web-error-handling An issue related to web error handling type: documentation A documentation task
Projects
None yet
Development

No branches or pull requests

3 participants